Compare commits

..

28 Commits

Author SHA1 Message Date
Romain Vimont
9ac57d6e71 hid_mouse_initialization 2022-01-01 20:05:17 +01:00
Romain Vimont
82ac944d23 refactor_hid_keyboard_init 2022-01-01 20:00:33 +01:00
Romain Vimont
b33e9ca3a7 hidmousecli 2022-01-01 19:48:44 +01:00
Romain Vimont
d140537e93 hid_mouse 2022-01-01 19:48:44 +01:00
Romain Vimont
9f80b91d72 Add CLAMP() macro 2022-01-01 19:48:44 +01:00
Romain Vimont
7dba4a9e98 relative-mouse-mode 2022-01-01 19:48:44 +01:00
Romain Vimont
9ec4017910 relative_mode 2022-01-01 19:48:44 +01:00
Romain Vimont
a97151707e Add relative mouse motion in event
This will allow the mouse processor to handle relative motion easily.
2022-01-01 19:48:44 +01:00
Romain Vimont
e84cdf963c Destroy acksync immediately on error
If AOA or HID keyboard may not be initialized for some reason, acksync
is useless.
2022-01-01 19:47:47 +01:00
Romain Vimont
4fcace8ab0 Remove duplicate boolean
The AOA initialization state is already tracked by aoa_hid_initialized.
2022-01-01 19:45:50 +01:00
Romain Vimont
e9bef7e880 Reorder controller and HID initialization
This allows to merge two "#ifdef HAVE_AOA_HID" blocks to simplify.
2022-01-01 19:38:27 +01:00
Romain Vimont
eb23a2afc8 Move input_manager into screen
The input_manager is strongly tied to the screen, it could not work
independently of the specific screen implementation.

To implement a user-friendly HID mouse behavior, some SDL events
will need to be handled both by the screen and by the input manager. For
example, a click must typically be handled by the input_manager so that
it is forwarded to the device, but in HID mouse mode, the first click
should be handled by the screen to capture the mouse (enable relative
mouse mode).

Make the input_manager a descendant of the screen, so that the screen
decides what to do on SDL events.

Concretely, replace this structure hierarchy:

     +- struct scrcpy
        +- struct input_manager
        +- struct screen

by this one:

     +- struct scrcpy
        +- struct screen
           +- struct input_manager
2022-01-01 19:01:27 +01:00
Romain Vimont
c7038da147 Use separate struct for input manager params
This avoids to directly pass the options instance (which contains more
data than strictly necessary), and limit the number of parameters for
the init function.
2022-01-01 19:01:27 +01:00
Romain Vimont
9e4773fd24 Pass buttons state in scroll events
A scroll event might be produced when a mouse button is pressed (for
example when scrolling while selecting a text). For consistency, pass
the actual buttons state (instead of 0).

In practice, it seems that this use case does not work properly with
Android event injection, but it will work with HID mouse.
2022-01-01 19:01:27 +01:00
Romain Vimont
dada6883d6 Make some mouse processors ops optional
Do not force all mouse processors implementations to implement scroll
events or touch events.
2022-01-01 19:01:27 +01:00
Romain Vimont
65087fcf57 Make process_text() optional
Not all key processors support text injection (HID keyboard does not
support it).

Instead of providing a dummy op function, set it to NULL and check on
the caller side before calling it.
2022-01-01 19:01:27 +01:00
Romain Vimont
71c2af1aed Apply buttons mask if not --forward-all-clicks
If --forward-all-clicks is not set, then only left clicks are forwarded.
For consistency, also mask the buttons state in other events.
2022-01-01 19:01:27 +01:00
Romain Vimont
4ab8775ed0 Reorder mouse processor ops
Group the mouse events callbacks before the touch event callback.
2022-01-01 19:01:27 +01:00
Romain Vimont
396bbc1a84 Simplify mouse injection implementation
The static functions are now so simple they become unnecessary: the
control message may be initialized directly instead.
2022-01-01 19:01:27 +01:00
Romain Vimont
f92e9edd1d Make some event conversions infallible
When the implementation handles all possible input values, it may never
fail.
2022-01-01 19:01:27 +01:00
Romain Vimont
07e46dabe4 Use scrcpy input events for mouse processors
Pass scrcpy input events instead of SDL input events to mouse
processors.

These events represent exactly what mouse processors need, abstracted
from any visual orientation and scaling applied on the SDL window.

This makes the mouse processors independent of the "screen" instance,
and the implementation source code independent of the SDL API.
2022-01-01 19:01:27 +01:00
Romain Vimont
abc5bcc6df Use scrcpy input events for key processors
Pass scrcpy input events instead of SDL input events to key processors.

This makes the source code of key processors independent of the SDL API.
2022-01-01 19:01:27 +01:00
Romain Vimont
2ba699902e Use common sc_action in input manager
Now that the scrcpy input events API exposes a sc_action enum, use the
same from the input manager.
2022-01-01 19:01:27 +01:00
Romain Vimont
191339987f Add intermediate input events layer
This aims to make the key/mouse processors independent of the "screen",
instead of processing SDL events themselves.

In particular, these scrcpy events are not impacted by any UI window
scaling or rotation (contrary to SDL events).
2022-01-01 19:01:27 +01:00
Romain Vimont
8cd245214d Rename SC_MOD_* to SC_SHORTCUT_MOD_*
This will avoid conflicts with new SC_MOD_* constants.
2022-01-01 19:01:27 +01:00
Romain Vimont
cc59906a38 Remove actions bitset
The input manager exposed functions taking an "actions" parameter,
containing a bitmask-OR of ACTION_UP and ACTION_DOWN.

But they are never called with both actions simultaneously anymore, so
simplify.

Refs 964b6d2243
Refs d0739911a3
2022-01-01 19:01:27 +01:00
Romain Vimont
7bd3da79b6 Expose V4L2 option on all platforms
This allows to report a meaningful error message if an unsupported
feature is used on another platform. This is consistent with the
behavior of -K/--hid-keyboard.
2022-01-01 19:01:27 +01:00
Romain Vimont
d76bf4c50c Fail on unsupported HID option
If the feature is not supported on the platform, fail during command
line parsing instead of using a fallback.
2022-01-01 19:01:26 +01:00
8 changed files with 19 additions and 65 deletions

View File

@@ -31,8 +31,6 @@ Its features include:
- device screen [as a webcam (V4L2)](#v4l2loopback) (Linux-only)
- [physical keyboard simulation (HID)](#physical-keyboard-simulation-hid)
(Linux-only)
- [physical mouse simulation (HID)](#physical-mouse-simulation-hid)
(Linux-only)
- and more…
## Requirements
@@ -817,35 +815,6 @@ a physical keyboard is connected).
[Physical keyboard]: https://github.com/Genymobile/scrcpy/pull/2632#issuecomment-923756915
#### Physical mouse simulation (HID)
Similarly to the physical keyboard simulation, it is possible to simulate a
physical mouse. Likewise, it only works if the device is connected by USB, and
is currently only supported on Linux.
By default, scrcpy uses Android mouse events injection, using absolute
coordinates. By simulating a physical mouse, a mouse pointer appears on the
Android device, and relative mouse motion, clicks and scrolls are injected.
To enable this mode:
```bash
scrcpy --hid-mouse
scrcpy -M # short version
```
You could also add `--forward-all-clicks` to [forward all mouse
buttons][forward_all_clicks].
[forward_all_clicks]: #right-click-and-middle-click
When this mode is enabled, the computer mouse is "captured" (the mouse pointer
disappears from the computer and appears on the Android device instead).
Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
(disable or enable) the mouse capture. Use one of them to give the control of
the mouse back to the computer.
#### Text injection preference

View File

@@ -96,8 +96,6 @@ The keyboard layout must be configured (once and for all) on the device, via Set
However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
Also see \fB\-\-hid\-mouse\fR.
.TP
.B \-\-legacy\-paste
Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+Shift+v).
@@ -122,18 +120,6 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
Default is 0 (unlimited).
.TP
.B \-M, \-\-hid\-mouse
Simulate a physical mouse by using HID over AOAv2.
In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
It may only work over USB, and is currently only supported on Linux.
Also see \fB\-\-hid\-keyboard\fR.
.TP
.B \-\-no\-clipboard\-autosync
By default, scrcpy automatically synchronizes the computer clipboard to the device clipboard before injecting Ctrl+v, and the device clipboard to the computer clipboard whenever it changes.

View File

@@ -178,8 +178,7 @@ static const struct sc_option options[] = {
"directly: `adb shell am start -a "
"android.settings.HARD_KEYBOARD_SETTINGS`.\n"
"However, the option is only available when the HID keyboard "
"is enabled (or a physical keyboard is connected).\n"
"Also see --hid-mouse.",
"is enabled (or a physical keyboard is connected).",
},
{
.shortopt = 'h',
@@ -224,8 +223,7 @@ static const struct sc_option options[] = {
"LAlt, LSuper or RSuper toggle the capture mode, to give "
"control of the mouse back to the computer.\n"
"It may only work over USB, and is currently only supported "
"on Linux.\n"
"Also see --hid-keyboard.",
"on Linux.\n",
},
{
.shortopt = 'm',
@@ -408,8 +406,8 @@ static const struct sc_option options[] = {
"frames. This increases latency to compensate for jitter.\n"
"This option is similar to --display-buffer, but specific to "
"V4L2 sink.\n"
"Default is 0 (no buffering).\n"
"This option is only available on Linux.",
"This feature is only available on Linux.\n"
"Default is 0 (no buffering).",
},
{
.shortopt = 'V',

View File

@@ -54,6 +54,7 @@ enum sc_mod {
SC_MOD_NUM = KMOD_NUM,
SC_MOD_CAPS = KMOD_CAPS,
SC_MOD_SCROLL = KMOD_SCROLL,
};
enum sc_action {

View File

@@ -171,6 +171,7 @@ send_keycode(struct controller *controller, enum android_keycode keycode,
if (!controller_push_msg(controller, &msg)) {
LOGW("Could not request 'inject %s'", name);
return;
}
}
@@ -222,6 +223,7 @@ press_back_or_turn_screen_on(struct controller *controller,
if (!controller_push_msg(controller, &msg)) {
LOGW("Could not request 'press back or turn screen on'");
return;
}
}
@@ -632,6 +634,14 @@ static void
input_manager_process_mouse_motion(struct input_manager *im,
const SDL_MouseMotionEvent *event) {
uint32_t mask = SDL_BUTTON_LMASK;
if (im->forward_all_clicks) {
mask |= SDL_BUTTON_MMASK | SDL_BUTTON_RMASK;
}
//if (!(event->state & mask)) {
// // do not send motion events when no click is pressed
// return;
//}
if (event->which == SDL_TOUCH_MOUSEID) {
// simulated from touch events, so it's a duplicate
return;
@@ -653,11 +663,7 @@ input_manager_process_mouse_motion(struct input_manager *im,
assert(im->mp->ops->process_mouse_motion);
im->mp->ops->process_mouse_motion(im->mp, &evt);
// vfinger must never be used in relative mode
assert(!im->mp->relative_mode || !im->vfinger_down);
if (im->vfinger_down) {
assert(!im->mp->relative_mode); // assert one more time
struct sc_point mouse =
screen_convert_window_to_frame_coords(im->screen, event->x,
event->y);
@@ -786,9 +792,8 @@ input_manager_process_mouse_button(struct input_manager *im,
// In other words, the center of the rotation/scaling is the center of the
// screen.
#define CTRL_PRESSED (SDL_GetModState() & (KMOD_LCTRL | KMOD_RCTRL))
if (event->button == SDL_BUTTON_LEFT &&
((down && !im->vfinger_down && CTRL_PRESSED) ||
(!down && im->vfinger_down))) {
if ((down && !im->vfinger_down && CTRL_PRESSED)
|| (!down && im->vfinger_down)) {
struct sc_point mouse =
screen_convert_window_to_frame_coords(im->screen, event->x,
event->y);

View File

@@ -13,6 +13,8 @@
#include "trait/key_processor.h"
#include "trait/mouse_processor.h"
struct screen; // forward declaration to avoid cycle
struct input_manager {
struct controller *controller;
struct screen *screen;

View File

@@ -58,11 +58,6 @@ convert_touch_action(enum sc_touch_action action) {
static void
sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
const struct sc_mouse_motion_event *event) {
if (!event->buttons_state) {
// Do not send motion events when no click is pressed
return;
}
struct sc_mouse_inject *mi = DOWNCAST(mp);
struct control_msg msg = {

View File

@@ -83,7 +83,6 @@ unwrap(sc_socket socket) {
#endif
}
#ifndef HAVE_SOCK_CLOEXEC // avoid unused-function warning
static inline bool
sc_raw_socket_close(sc_raw_socket raw_sock) {
#ifndef _WIN32
@@ -92,7 +91,6 @@ sc_raw_socket_close(sc_raw_socket raw_sock) {
return !closesocket(raw_sock);
#endif
}
#endif
#ifndef HAVE_SOCK_CLOEXEC
// If SOCK_CLOEXEC does not exist, the flag must be set manually once the