Feature Request - absolute mouse

[jrsmile]

Would it be possible to adapt the mouse hid class to send absolute coordinates like a touchscreen?

[benjaminaigner]

Dear @jrsmile ,
should be possible.
I cannot estimate if and when we will include this, joystick support and stable connections have a higher priority.
If you want to try it on your own:
This is the HID report map:

esp32_mouse_keyboard/main/ble_hid/hid_device_le_prf.c

Line 44 in f7ab88a

static const uint8_t hidReportMap[] = {

There we need to replace the relative mouse by an absolute positioned.
Here is an example for this report map: https://forums.obdev.at/viewtopic.php?t=2559

[jrsmile]

i have seen a map here: HERE unfortunately i know to little to modify the code the correct way.
would the link help with the implementation?

[ChrisVeigl]

hi,
we added absolute mouse position and joystick reports to our HID-actuator several years ago - it worked. mybe this is helpful:

https://github.com/asterics/AsTeRICS/blob/master/CIMs/HID_actuator/Mouse_Keyboard_Joystick/Descriptors.c
https://github.com/asterics/AsTeRICS/blob/master/CIMs/HID_actuator/Mouse_Keyboard_Joystick/Descriptors.h

[benjaminaigner]

@ChrisVeigl , @jrsmile THX for the suggestions, based on this I will implement an additional report with an absolute mouse.

[jrsmile]

thank you very much, looking forward to it.
i will use it to build a firmware test tool.
best possible outcome would be a keyboard,mouse,touchscreen device and a ble serial device.
being able to test with a mobile phone as gui via ble a machine that has a bluetooth dongle attached is my target :-) i wonder if pairing with two devices is possible. if not usb serial will be suficient too.

[benjaminaigner]

Ah, sounds nice!
I've started a new branch with the absolute mouse included.
If there is time this week, I will test it.

Regarding the serial device:
This is a feature we won't implement by ourselves, but we could need that :-)

Pairing with multiple devices:
Pairing is possible, connecting should be too.
I think it is necessary to store different HID connection IDs for multiple devices, but this is also something thats not on our agenda.

[benjaminaigner]

@jrsmile

In the branch "absoluteMouse", there is a basic implementation of an absolute mouse report.
I've some input on Android, but I think it is not completely correct.

Could you please test it and report back?

[zuozhehao]

Is there any progress change.

[benjaminaigner]

@zuozhehao

I have a branch with the absolute mouse descriptor for testing.
Please try it out and report back. If everything is working, I'll merge the changes and make it available via config files.

[xcarcelle]

@benjaminaigner @zuozhehao thanks for the absolute mouse branch/progress and any help needed on test or code i will try to support this issue.

• I have tested the last branch commit on an Android tablet running 9.0 and it acts weird (like keys from a keyboard pushed very fast) when trying the absolute command from the uart in esp-idf (running make monitor flash)
• The mouse click also does not seem to work
  Do you have some unit tests (devices, os, test) you want to run to validate this current state ?

[benjaminaigner]

Dear all,
I'm sorry this feature takes so long.
Absolute mouse is not our top priority & due to COVID my workload regarding lectures is rising dramatically.

Still, I try my best to support you and see what we can do about the absolute mouse.

Current status:
@xcarcelle I've seen the same behaviour on my Android device.

I wanted to try my "real" absolute mouse (touchscreen) on iOS for @zuozhehao , but iPads do not accept USB accessory with 100mA current. Unfortunately I have no USB-C OTG cable available to test it on Android.

I will keep you updated, today I can spare some time for further testing.

[benjaminaigner]

OK, I tested following absolute HID descriptors:

https://www.codeproject.com/Articles/1001891/A-USB-HID-Keyboard-Mouse-Touchscreen-emulator-with

Variant 0: works on Debian, but not on iOS / Android
Variant 1: works on Android, but not on Debian / iOS
Variant 2: didn't get it to work on all platforms

https://www.schoeldgen.de/avr/

(USB Interface for Tektronix 4957 tablet using V-USB)
Works on Debian, not on iOS/Android
(most promising for iOS, because the Assistive Touch indicator went from grey to black)

Next try: remove all other reports except the absolute mouse, maybe this helps for iOS.
Does anybody has an alternative absolute mouse descriptor to test?