New approach to multitouch using DIDs and bitmasked events

[Rafi Rubin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

I think its clear that valuators are an extremely limited solution that just
can't scale, certainly not without breaking up a device into multiple virtual
devices each with a reasonable number of contacts.  Just imagine something like
the 3m screens that support 20+ contacts, really, the numbers just start to get
a little silly, and I'm pretty sure we'll see much larger devices capable of
supporting many people working with many fingers simultaneously.

So moving each contact off into its device is definitely a step in the right
direction.

I think the points I was trying to get to in earlier threads was more about the
structure of input hierarchy, and even in those cases moving the contacts into
separate devices simplifies implementation.  Though I think that is really a
tangential discussion (which we may or may not get back too in time, no rush).


At the moment I'm using the evdev+mtdev driver that Chase composed out of Ben's
valuator implementation, and its working well.  I like what I've been seeing,
but it will still be a little while before I update to try out the DID version,
if you really want my post testing opinion, I will make it a higher priority.
- From what I've seen, I really don't foresee trouble.


Just one small point, if you do make these DIDs look more pointer nodes, and we
can come up with some way to associate them with a more classical device, I'd
like to see the magic mouse and others also abandon valuators and switch to
children of the normal mouse.  That just seems to me like a more consistent and
easier to support approach.

Cheers,
Rafi


On 07/02/10 05:36, Peter Hutterer wrote:
> Hi Chase,
> 
> I'm replying to this summary now just to get some of the higher-level
> issues clarified. I haven't looked at the code yet.
> 
> On 2/07/10 08:36 , Chase Douglas wrote:
>> Last week I posted some patches for xf86-input-evdev to enable
>> multitouch through array-indexed valuators. However, I wasn't quite
>> satisfied with the results. First, depending on how many touch
>> properties your device presents there may be only enough room in the
>> valuators for five or six simultaneous touches. There are products
>> readily available that support more than this amount. Second, it still
>> sent touches as valuators of a device tied to a pointer. This seems ok
>> for a device like the Magic Mouse, but for most multitouch devices we
>> need to separate touches from X pointers due to the X pointer
>> architecture. One other small issue was that we have a nice new
>> optimized slotted events protocol from the kernel that doesn't send data
>> for touches and properties that haven't changed. However, in the evdev
>> patches we essentially undid all of that and passed all the properties
>> for every touch in every event.
>>
>> To remedy the pointer architecture issues I turned to the concept of
>> direct input devices (DIDs). Peter Hutterer brought up the idea of DIDs
>> in an email here:
>>
>> http://lists.x.org/archives/xorg-devel/2010-June/009757.html
>>
>> I have implemented this DIDs approach by creating a new internal event
>> type: ET_DirectMotion. These events are mostly just like ET_Motion
>> events, except they cannot be turned into core or XI1 events.
>>
>> I heuristically determine if a device is a DID by checking whether it
>> has valuators for ABS_MT_POSITION_X or ABS_MT_POSITION_Y, but none of
>> ABS_X, ABS_Y, REL_X, or REL_Y. This check is performed at valuator class
>> device initialization. Any motion events that a DID posts are sent as
>> ET_DirectMotion events. The ET_DirectMotion events are transformed into
>> normal motion events before they are sent to an XI2 client, so XI2
>> clients will see touches as normal pointer motion events. Further, a DID
>> starts off as a floating device, unattached to any master pointer
>> device. Any attempt to attach it to a master will fail.
> 
> This is a good start. Eventually, we should have DIDs exposed as a new
> device class on the protocol though. Likewise, I don't think ET_DM
> should not be converted to normal XIDeviceEvents but rather to their own
> type that can be expanded in the future to accommodate for additional
> features (e.g. the touch area).
> 
>> To remedy the protocol issues I looked at the bitmask valuator event
>> support of XI2. XI2 events send axis values alongside a bitmask
>> describing what values are valid for the event. However, the current X
>> input module API provides only for valuator ranges for events. I
>> extended the API by adding xf86Post*EventsM() and Get*EventsM()
>> functions. These functions take a "uint8_t *mask" argument instead of
>> the "int first_valuator" argument. The mask always starts at the 0th
>> valuator, and num_valuators is used to provide the position of the last
>> valid valuator for the event.
> 
> It'd be interesting to see how much work it is to have this API
> _replace_ the current API. Gives us more exposure and better testing.
> Note that I have some more API changes planned (not coded) that simplify
> the init process, they should all go in in one go.
> Another change that goes with that is the ability to easily split up
> devices into multiple X devices. This would make it easier to handle
> devices that have both MT events and normal events - they would simply
> end up being two devices, one normal one, one DID.
> 
> Henrik, Rafi - do you think this would work for the MT devices we've
> seen so far?
> 
>> I updated the xf86-input-evdev module to use the new bitmask APIs and to
>> send each touch as its own event. I also added a small patch to xinput
>> for the test-xi2 command so it prints out the valuator index along with
>> all the valuator values (otherwise you have no clue what values
>> correspond to what properties).
> 
> Thanks, this patch is already applied.
> 
>> All of these changes are available in my git repos on
>> http://kernel.ubuntu.com/git. I will be sending pull requests for
>> xserver, xf86-input-evdev, and a patch for xinput so people can find and
>> comment on the code. Also, I have uploaded versions of these packages to
>> the Ubuntu Maverick xorg-edgers/multitouch PPA if anyone wants to test
>> them out:
> 
> Thanks, I'll try to look at the code asap.
> 
> Cheers,
>   Peter
> 

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkwuw1EACgkQwuRiAT9o6094GACgsFZj/sMbal15I2Iy4MmJeA6e
NpIAnjcRXMMs0j53Wwwm1NVx2k2g+trg
=ZKuu
-----END PGP SIGNATURE-----