Compare commits

..

31 Commits

Author SHA1 Message Date
Romain Vimont
43aff4af73 Document HID mouse in README 2022-01-04 17:41:40 +01:00
Romain Vimont
cba84f6999 Add support for HID mouse 2022-01-04 17:41:40 +01:00
Romain Vimont
ed2e45ee29 Refactor AOA/HID keyboard initialization
This paves the way to add support for HID mouse initialization.
2022-01-04 17:41:40 +01:00
Romain Vimont
aee1b39790 Add CLAMP() macro 2022-01-04 17:41:40 +01:00
Romain Vimont
17d01b5bf7 Add UI/UX support for relative mouse mode
In relative mouse mode, the mouse pointer must be "captured" from the
computer.

Toggle (disable/enable) relative mouse mode using any of the hardcoded
capture keys:
 - left-Alt
 - left-Super
 - right-Super

These capture keys do not conflict with shortcuts, since a shortcut is
always a combination of the MOD key and some other key, while the
capture key triggers an action only if it is pressed and released alone.

The relative mouse mode is also automatically enabled on any click in
the window, and automatically disabled on focus lost (it is possible to
lose focus even without the mouse).
2022-01-04 17:41:40 +01:00
Romain Vimont
40fca82b60 Forward all motion events to mouse processors
The decision to not send motion events when no click is pressed is
specific to Android mouse injection. Other mouse processors (e.g. for
HID mouse) will need to receive all events.
2022-01-04 17:41:40 +01:00
Romain Vimont
643293752d Provide relative mouse motion vector in event
This will allow the mouse processor to handle relative motion easily.
2022-01-04 17:41:40 +01:00
Romain Vimont
b5855e5deb Add relative mode flag to mouse processors
The default mouse injection works in absolute mode: it forwards clicks
at a specific position on screen.

To support HID mouse, add a flag to indicate that the mouse processor
works in relative mode: it forwards mouse motion vectors, without any
absolute reference to the screen.
2022-01-04 17:41:40 +01:00
Romain Vimont
924375487e Pass buttons state in scroll events
A scroll event might be produced when a mouse button is pressed (for
example when scrolling while selecting a text). For consistency, pass
the actual buttons state (instead of 0).

In practice, it seems that this use case does not work properly with
Android event injection, but it will work with HID mouse.
2022-01-04 17:41:40 +01:00
Romain Vimont
7121a0dc53 Destroy acksync immediately on error
If AOA or HID keyboard may not be initialized for some reason, acksync
is useless.
2022-01-04 17:41:40 +01:00
Romain Vimont
f04812fc71 Remove duplicate boolean
The AOA initialization state is already tracked by aoa_hid_initialized.
2022-01-04 17:41:40 +01:00
Romain Vimont
5ce1ccde85 Reorder controller and HID initialization
This allows to merge two "#ifdef HAVE_AOA_HID" blocks to simplify.
2022-01-04 17:41:40 +01:00
Romain Vimont
6102a0b5bb Move input_manager into screen
The input_manager is strongly tied to the screen, it could not work
independently of the specific screen implementation.

To implement a user-friendly HID mouse behavior, some SDL events
will need to be handled both by the screen and by the input manager. For
example, a click must typically be handled by the input_manager so that
it is forwarded to the device, but in HID mouse mode, the first click
should be handled by the screen to capture the mouse (enable relative
mouse mode).

Make the input_manager a descendant of the screen, so that the screen
decides what to do on SDL events.

Concretely, replace this structure hierarchy:

     +- struct scrcpy
        +- struct input_manager
        +- struct screen

by this one:

     +- struct scrcpy
        +- struct screen
           +- struct input_manager
2022-01-04 17:41:35 +01:00
Romain Vimont
2b34e1224e Use separate struct for input manager params
This avoids to directly pass the options instance (which contains more
data than strictly necessary), and limit the number of parameters for
the init function.
2022-01-04 15:14:38 +01:00
Romain Vimont
a9d23400cd Remove unused enum value requiring SDL 2.0.18
Refs b8fed50639
Fixes #2924 <https://github.com/Genymobile/scrcpy/issues/2924>
2022-01-04 15:11:33 +01:00
Romain Vimont
cca3c953da Enable virtual finger only on left click
The pinch-to-zoom feature must only be enabled with Ctrl+left_click.
2022-01-02 00:00:33 +01:00
Romain Vimont
57f1655d4b Make some mouse processors ops optional
Do not force all mouse processors to implement scroll events or touch
events.
2022-01-01 23:34:56 +01:00
Romain Vimont
bc674721dc Make process_text() optional
Not all key processors support text injection (HID keyboard does not
support it).

Instead of providing a dummy op function, set it to NULL and check on
the caller side before calling it.
2022-01-01 23:31:01 +01:00
Romain Vimont
63e29b1782 Apply buttons mask if not --forward-all-clicks
If --forward-all-clicks is not set, then only left clicks are forwarded.
For consistency, also mask the buttons state in other events.
2022-01-01 23:31:01 +01:00
Romain Vimont
3c15cbdaf8 Reorder mouse processor ops
Group the mouse events callbacks before the touch event callback.
2022-01-01 23:31:01 +01:00
Romain Vimont
96e0e89740 Simplify mouse injection implementation
The static functions are now so simple that they become unnecessary: the
control message may be initialized directly instead.
2022-01-01 23:30:55 +01:00
Romain Vimont
a1f2f5fbd3 Make some event conversions infallible
When the implementation handles all possible input values, it may never
fail.
2022-01-01 23:28:45 +01:00
Romain Vimont
9460bdd87b Use scrcpy input events for mouse processors
Pass scrcpy input events instead of SDL input events to mouse
processors.

These events represent exactly what mouse processors need, abstracted
from any visual orientation and scaling applied on the SDL window.

This makes the mouse processors independent of the "screen" instance,
and the implementation source code independent of the SDL API.
2022-01-01 23:28:45 +01:00
Romain Vimont
b4b638e8fe Use scrcpy input events for key processors
Pass scrcpy input events instead of SDL input events to key processors.

This makes the source code of key processors independent of the SDL API.
2022-01-01 23:28:45 +01:00
Romain Vimont
e4396e34c2 Use common sc_action in input manager
Now that the scrcpy input events API exposes a sc_action enum, use the
same from the input manager.
2022-01-01 23:28:45 +01:00
Romain Vimont
b8fed50639 Add intermediate input events layer
This aims to make the key/mouse processors independent of the "screen",
by processing scrcpy-specific input events instead of SDL events.

In particular, these scrcpy events are not impacted by any UI window
scaling or rotation (contrary to SDL events).
2022-01-01 23:28:45 +01:00
Romain Vimont
d540c72e7c Rename SC_MOD_* to SC_SHORTCUT_MOD_*
This will avoid conflicts with new SC_MOD_* constants.
2022-01-01 23:28:45 +01:00
Romain Vimont
cd5891fee6 Remove actions bitset
The input manager exposed functions taking an "actions" parameter,
containing a bitmask-OR of ACTION_UP and ACTION_DOWN.

But they are never called with both actions simultaneously anymore, so
simplify.

Refs 964b6d2243
Refs d0739911a3
2022-01-01 23:28:45 +01:00
Romain Vimont
26ee7ce566 Expose V4L2 option on all platforms
This allows to report a meaningful error message if an unsupported
feature is used on an incompatible platform. This is consistent with the
behavior of -K/--hid-keyboard.
2022-01-01 23:28:45 +01:00
Romain Vimont
ba28d817fb Fail on unsupported HID option
If the feature is not supported on the platform, fail during command
line parsing instead of using a fallback.
2022-01-01 23:28:45 +01:00
Romain Vimont
37124e1452 Avoid unused function warning
If HAVE_SOCK_CLOEXEC is not defined, then sc_raw_socket_close() is never
used. Add an #ifndef block to remove the warning.
2022-01-01 23:27:34 +01:00
8 changed files with 65 additions and 19 deletions

View File

@@ -31,6 +31,8 @@ Its features include:
- device screen [as a webcam (V4L2)](#v4l2loopback) (Linux-only)
- [physical keyboard simulation (HID)](#physical-keyboard-simulation-hid)
(Linux-only)
- [physical mouse simulation (HID)](#physical-mouse-simulation-hid)
(Linux-only)
- and more…
## Requirements
@@ -815,6 +817,35 @@ a physical keyboard is connected).
[Physical keyboard]: https://github.com/Genymobile/scrcpy/pull/2632#issuecomment-923756915
#### Physical mouse simulation (HID)
Similarly to the physical keyboard simulation, it is possible to simulate a
physical mouse. Likewise, it only works if the device is connected by USB, and
is currently only supported on Linux.
By default, scrcpy uses Android mouse events injection, using absolute
coordinates. By simulating a physical mouse, a mouse pointer appears on the
Android device, and relative mouse motion, clicks and scrolls are injected.
To enable this mode:
```bash
scrcpy --hid-mouse
scrcpy -M # short version
```
You could also add `--forward-all-clicks` to [forward all mouse
buttons][forward_all_clicks].
[forward_all_clicks]: #right-click-and-middle-click
When this mode is enabled, the computer mouse is "captured" (the mouse pointer
disappears from the computer and appears on the Android device instead).
Special capture keys, either <kbd>Alt</kbd> or <kbd>Super</kbd>, toggle
(disable or enable) the mouse capture. Use one of them to give the control of
the mouse back to the computer.
#### Text injection preference

View File

@@ -96,6 +96,8 @@ The keyboard layout must be configured (once and for all) on the device, via Set
However, the option is only available when the HID keyboard is enabled (or a physical keyboard is connected).
Also see \fB\-\-hid\-mouse\fR.
.TP
.B \-\-legacy\-paste
Inject computer clipboard text as a sequence of key events on Ctrl+v (like MOD+Shift+v).
@@ -120,6 +122,18 @@ Limit both the width and height of the video to \fIvalue\fR. The other dimension
Default is 0 (unlimited).
.TP
.B \-M, \-\-hid\-mouse
Simulate a physical mouse by using HID over AOAv2.
In this mode, the computer mouse is captured to control the device directly (relative mouse mode).
LAlt, LSuper or RSuper toggle the capture mode, to give control of the mouse back to the computer.
It may only work over USB, and is currently only supported on Linux.
Also see \fB\-\-hid\-keyboard\fR.
.TP
.B \-\-no\-clipboard\-autosync
By default, scrcpy automatically synchronizes the computer clipboard to the device clipboard before injecting Ctrl+v, and the device clipboard to the computer clipboard whenever it changes.

View File

@@ -178,7 +178,8 @@ static const struct sc_option options[] = {
"directly: `adb shell am start -a "
"android.settings.HARD_KEYBOARD_SETTINGS`.\n"
"However, the option is only available when the HID keyboard "
"is enabled (or a physical keyboard is connected).",
"is enabled (or a physical keyboard is connected).\n"
"Also see --hid-mouse.",
},
{
.shortopt = 'h',
@@ -223,7 +224,8 @@ static const struct sc_option options[] = {
"LAlt, LSuper or RSuper toggle the capture mode, to give "
"control of the mouse back to the computer.\n"
"It may only work over USB, and is currently only supported "
"on Linux.\n",
"on Linux.\n"
"Also see --hid-keyboard.",
},
{
.shortopt = 'm',
@@ -406,8 +408,8 @@ static const struct sc_option options[] = {
"frames. This increases latency to compensate for jitter.\n"
"This option is similar to --display-buffer, but specific to "
"V4L2 sink.\n"
"This feature is only available on Linux.\n"
"Default is 0 (no buffering).",
"Default is 0 (no buffering).\n"
"This option is only available on Linux.",
},
{
.shortopt = 'V',

View File

@@ -54,7 +54,6 @@ enum sc_mod {
SC_MOD_NUM = KMOD_NUM,
SC_MOD_CAPS = KMOD_CAPS,
SC_MOD_SCROLL = KMOD_SCROLL,
};
enum sc_action {

View File

@@ -171,7 +171,6 @@ send_keycode(struct controller *controller, enum android_keycode keycode,
if (!controller_push_msg(controller, &msg)) {
LOGW("Could not request 'inject %s'", name);
return;
}
}
@@ -223,7 +222,6 @@ press_back_or_turn_screen_on(struct controller *controller,
if (!controller_push_msg(controller, &msg)) {
LOGW("Could not request 'press back or turn screen on'");
return;
}
}
@@ -634,14 +632,6 @@ static void
input_manager_process_mouse_motion(struct input_manager *im,
const SDL_MouseMotionEvent *event) {
uint32_t mask = SDL_BUTTON_LMASK;
if (im->forward_all_clicks) {
mask |= SDL_BUTTON_MMASK | SDL_BUTTON_RMASK;
}
//if (!(event->state & mask)) {
// // do not send motion events when no click is pressed
// return;
//}
if (event->which == SDL_TOUCH_MOUSEID) {
// simulated from touch events, so it's a duplicate
return;
@@ -663,7 +653,11 @@ input_manager_process_mouse_motion(struct input_manager *im,
assert(im->mp->ops->process_mouse_motion);
im->mp->ops->process_mouse_motion(im->mp, &evt);
// vfinger must never be used in relative mode
assert(!im->mp->relative_mode || !im->vfinger_down);
if (im->vfinger_down) {
assert(!im->mp->relative_mode); // assert one more time
struct sc_point mouse =
screen_convert_window_to_frame_coords(im->screen, event->x,
event->y);
@@ -792,8 +786,9 @@ input_manager_process_mouse_button(struct input_manager *im,
// In other words, the center of the rotation/scaling is the center of the
// screen.
#define CTRL_PRESSED (SDL_GetModState() & (KMOD_LCTRL | KMOD_RCTRL))
if ((down && !im->vfinger_down && CTRL_PRESSED)
|| (!down && im->vfinger_down)) {
if (event->button == SDL_BUTTON_LEFT &&
((down && !im->vfinger_down && CTRL_PRESSED) ||
(!down && im->vfinger_down))) {
struct sc_point mouse =
screen_convert_window_to_frame_coords(im->screen, event->x,
event->y);

View File

@@ -13,8 +13,6 @@
#include "trait/key_processor.h"
#include "trait/mouse_processor.h"
struct screen; // forward declaration to avoid cycle
struct input_manager {
struct controller *controller;
struct screen *screen;

View File

@@ -58,6 +58,11 @@ convert_touch_action(enum sc_touch_action action) {
static void
sc_mouse_processor_process_mouse_motion(struct sc_mouse_processor *mp,
const struct sc_mouse_motion_event *event) {
if (!event->buttons_state) {
// Do not send motion events when no click is pressed
return;
}
struct sc_mouse_inject *mi = DOWNCAST(mp);
struct control_msg msg = {

View File

@@ -83,6 +83,7 @@ unwrap(sc_socket socket) {
#endif
}
#ifndef HAVE_SOCK_CLOEXEC // avoid unused-function warning
static inline bool
sc_raw_socket_close(sc_raw_socket raw_sock) {
#ifndef _WIN32
@@ -91,6 +92,7 @@ sc_raw_socket_close(sc_raw_socket raw_sock) {
return !closesocket(raw_sock);
#endif
}
#endif
#ifndef HAVE_SOCK_CLOEXEC
// If SOCK_CLOEXEC does not exist, the flag must be set manually once the