New approach to multitouch using DIDs and bitmasked events

[Rafi Rubin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 07/02/10 21:09, Chase Douglas wrote:
> On Fri, 2010-07-02 at 19:36 +1000, Peter Hutterer wrote:
>> Hi Chase,
>>
>> I'm replying to this summary now just to get some of the higher-level 
>> issues clarified. I haven't looked at the code yet.
>>
>> On 2/07/10 08:36 , Chase Douglas wrote:
>>> Last week I posted some patches for xf86-input-evdev to enable
>>> multitouch through array-indexed valuators. However, I wasn't quite
>>> satisfied with the results. First, depending on how many touch
>>> properties your device presents there may be only enough room in the
>>> valuators for five or six simultaneous touches. There are products
>>> readily available that support more than this amount. Second, it still
>>> sent touches as valuators of a device tied to a pointer. This seems ok
>>> for a device like the Magic Mouse, but for most multitouch devices we
>>> need to separate touches from X pointers due to the X pointer
>>> architecture. One other small issue was that we have a nice new
>>> optimized slotted events protocol from the kernel that doesn't send data
>>> for touches and properties that haven't changed. However, in the evdev
>>> patches we essentially undid all of that and passed all the properties
>>> for every touch in every event.
>>>
>>> To remedy the pointer architecture issues I turned to the concept of
>>> direct input devices (DIDs). Peter Hutterer brought up the idea of DIDs
>>> in an email here:
>>>
>>> http://lists.x.org/archives/xorg-devel/2010-June/009757.html
>>>
>>> I have implemented this DIDs approach by creating a new internal event
>>> type: ET_DirectMotion. These events are mostly just like ET_Motion
>>> events, except they cannot be turned into core or XI1 events.
>>>
>>> I heuristically determine if a device is a DID by checking whether it
>>> has valuators for ABS_MT_POSITION_X or ABS_MT_POSITION_Y, but none of
>>> ABS_X, ABS_Y, REL_X, or REL_Y. This check is performed at valuator class
>>> device initialization. Any motion events that a DID posts are sent as
>>> ET_DirectMotion events. The ET_DirectMotion events are transformed into
>>> normal motion events before they are sent to an XI2 client, so XI2
>>> clients will see touches as normal pointer motion events. Further, a DID
>>> starts off as a floating device, unattached to any master pointer
>>> device. Any attempt to attach it to a master will fail.
>>
>> This is a good start. Eventually, we should have DIDs exposed as a new 
>> device class on the protocol though. Likewise, I don't think ET_DM 
>> should not be converted to normal XIDeviceEvents but rather to their own 
>> type that can be expanded in the future to accommodate for additional 
>> features (e.g. the touch area).
> 
> I hacked some more today, and I'm not sure we need a separate XI2 event
> type. I'm going to send out another RFC "pull request" with the work
> soon, but as an overview I wanted to work on the "magicmouse" scenario.
> The result is the following:

Another plug for multilevel hierarchy instead of the two level master/slave.

I'm glad the xi2 pointers seem good enough now, that does seem simpler.  Just an
extra flag for "no_core".  Then again, can't we just mask out MT events from the
core, and then it won't matter?

> MT Touchscreen:
>   - Device is marked as a DID
>   - Starts out floating, can't be attached
>   - MT touch coordinates are the first two valuators, XIDeviceEvents are
>     in screen coordinates. However, an app can grab XIRawEvents for
>     device coordinates. *
>   - Since it's a DID, button events are also blocked for core, XI1

Slight problem with masking out buttons.  Henrik, didn't you add different
device types (ie, MT pen) to your protocol?  What do buttons mean on a mt pen?


> Magic Mouse:
>   - Device is *not* marked as a DID
>   - Starts out attached to virtual core master pointer
>   - First two axes are REL_X, REL_Y
>   - Device is marked as relative (for core and XI1 events)
>   - Abs MT axes are marked as absolute
>     * Can only be sent through XI2 since XI can't handle mixed mode
>   - Since Abs MT touches are not the first two axes, they are not
>     translated to screen coordinates
>   - Abs MT axes sent through XI2 are also sent through attached master
>   - Button events are not blocked for core, XI1
> 
> I am not aware of any use case that is not covered by the above scheme
> and requires a new XI2 event type. If you know of any, please let me
> know.
> 
> * Raw events are only delivered to the root windows and windows with a
> grab. This is a limitation currently, but I wonder why we can't just
> deliver them like normal events?

Still trying to understand what raw events mean in the scope of X clients.
Certainly I'm comfortable with using them outside, and I don't really see a
downside.  Do you also expose physical metadata (size of sensor)?

Rafi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwuxdYACgkQwuRiAT9o609qWQCfcWZvPrNExXSALRqYnCTzKGnZ
rG8AmgMW1srbpOaj6+tj0Vg11Ozf2tY7
=t5P0
-----END PGP SIGNATURE-----