New approach to multitouch using DIDs and bitmasked events

[Chase Douglas]

On Fri, 2010-07-02 at 19:36 +1000, Peter Hutterer wrote:
> Hi Chase,
> 
> I'm replying to this summary now just to get some of the higher-level 
> issues clarified. I haven't looked at the code yet.
> 
> On 2/07/10 08:36 , Chase Douglas wrote:
> > Last week I posted some patches for xf86-input-evdev to enable
> > multitouch through array-indexed valuators. However, I wasn't quite
> > satisfied with the results. First, depending on how many touch
> > properties your device presents there may be only enough room in the
> > valuators for five or six simultaneous touches. There are products
> > readily available that support more than this amount. Second, it still
> > sent touches as valuators of a device tied to a pointer. This seems ok
> > for a device like the Magic Mouse, but for most multitouch devices we
> > need to separate touches from X pointers due to the X pointer
> > architecture. One other small issue was that we have a nice new
> > optimized slotted events protocol from the kernel that doesn't send data
> > for touches and properties that haven't changed. However, in the evdev
> > patches we essentially undid all of that and passed all the properties
> > for every touch in every event.
> >
> > To remedy the pointer architecture issues I turned to the concept of
> > direct input devices (DIDs). Peter Hutterer brought up the idea of DIDs
> > in an email here:
> >
> > http://lists.x.org/archives/xorg-devel/2010-June/009757.html
> >
> > I have implemented this DIDs approach by creating a new internal event
> > type: ET_DirectMotion. These events are mostly just like ET_Motion
> > events, except they cannot be turned into core or XI1 events.
> >
> > I heuristically determine if a device is a DID by checking whether it
> > has valuators for ABS_MT_POSITION_X or ABS_MT_POSITION_Y, but none of
> > ABS_X, ABS_Y, REL_X, or REL_Y. This check is performed at valuator class
> > device initialization. Any motion events that a DID posts are sent as
> > ET_DirectMotion events. The ET_DirectMotion events are transformed into
> > normal motion events before they are sent to an XI2 client, so XI2
> > clients will see touches as normal pointer motion events. Further, a DID
> > starts off as a floating device, unattached to any master pointer
> > device. Any attempt to attach it to a master will fail.
> 
> This is a good start. Eventually, we should have DIDs exposed as a new 
> device class on the protocol though. Likewise, I don't think ET_DM 
> should not be converted to normal XIDeviceEvents but rather to their own 
> type that can be expanded in the future to accommodate for additional 
> features (e.g. the touch area).

I hacked some more today, and I'm not sure we need a separate XI2 event
type. I'm going to send out another RFC "pull request" with the work
soon, but as an overview I wanted to work on the "magicmouse" scenario.
The result is the following:

MT Touchscreen:
  - Device is marked as a DID
  - Starts out floating, can't be attached
  - MT touch coordinates are the first two valuators, XIDeviceEvents are
    in screen coordinates. However, an app can grab XIRawEvents for
    device coordinates. *
  - Since it's a DID, button events are also blocked for core, XI1

Magic Mouse:
  - Device is *not* marked as a DID
  - Starts out attached to virtual core master pointer
  - First two axes are REL_X, REL_Y
  - Device is marked as relative (for core and XI1 events)
  - Abs MT axes are marked as absolute
    * Can only be sent through XI2 since XI can't handle mixed mode
  - Since Abs MT touches are not the first two axes, they are not
    translated to screen coordinates
  - Abs MT axes sent through XI2 are also sent through attached master
  - Button events are not blocked for core, XI1

I am not aware of any use case that is not covered by the above scheme
and requires a new XI2 event type. If you know of any, please let me
know.

* Raw events are only delivered to the root windows and windows with a
grab. This is a limitation currently, but I wonder why we can't just
deliver them like normal events?

Thanks,

-- Chase