New approach to multitouch using DIDs and bitmasked events

[Peter Hutterer]

On Sun, Jul 11, 2010 at 11:31:13PM -0400, Chase Douglas wrote:
> On Mon, 2010-07-12 at 12:06 +1000, Peter Hutterer wrote:
> > On Thu, Jul 08, 2010 at 04:05:10PM -0400, Chase Douglas wrote:
> > > On Wed, 2010-07-07 at 15:21 -0400, Chase Douglas wrote:
> > > > On Tue, 2010-07-06 at 11:33 +1000, Peter Hutterer wrote:
> > > > I was keying all my work off of the concepts Peter pointed out here:
> > > > 
> > > >    http://lists.x.org/archives/xorg-devel/2010-June/009757.html
> > > > 
> > > >    The arguments boiled down mostly to: input devices can't be transient
> > > > due to grab issues. But our touch devices aren't transient anymore.
> > > 
> > > I thought some more about this again, and I think Peter may have been
> > > getting at the physical state of the touches as input devices, not the
> > > fact that the touch input devices literally go away (i.e.
> > > deallocated/destroyed).
> > > 
> > > This leaves us back at trying to figure out what to do about grabs on MT
> > > devices. It seems apparent that no matter what we do, nothing will be
> > > backwards compatible for MT touch event grabs. (We can still route one
> > > touch at a time to the master core pointer for client pointer backwards
> > > compatibility, but that's a different issue.)
> > 
> > My current thinking is that we may need to allow for two different grabs on
> > a device - one for touch events, one for traditional pointer/keyboard
> > events. There is still a use-case for grabs even with direct-touch input.
> 
> Yes, I agree to both points. I was wondering if maybe we should have
> device grabs as in a client can grab an entire physical hardware device,
> and thus grabbing all touch events it generates. A use case may be where
> gimp grabs an MT touchpad (like a Wacom Bamboo) and wants all touches to
> go to it instead of acting like a normal pointing device.

I think the passive and active grab definitions work equally well for touch
events. The thing that's giving me a headache right now is the pointer
emulation - if a device emulates pointer events on the first touch and that
pointer is being grabbed, how does this affect future touches?

> Secondly, it seems we also need a sane way of handling transient touch
> grabs. I'm not sure how this can be done other than to craft the
> protocol such that the grab will cease as soon as the touch leaves.
> Clients should key off the fact that the input device is
> transient/direct and thus be prepared to have the grab broken by the
> server.

I see two options - the passiv grab system already does this anyway
(releasing the grabs when the touch diesappears). The second one which may
be useful for touch events is a counted passive grab. i.e. a client
requesting a grab can say that the grab is only activated N times. This is
mostly to work around race conditions with the current system.

now, the tricky thing (and that's where the DID idea partially comes from)
is that although we have one device, there may be multiple active passive
grabs on the device, one per touch. so touch in client A triggers pgrab on
that touch, touch in client B triggers pgrab on that touch, though both
touches exist simultaneously.
and that could be interesting, though I suspect here the implementation is
tricker than the protocol definition.
 
> > > Generally, we would still have the concept of DIDs in the sense that the
> > > touch input devices themselves would be floating, and thus XI2 only. So,
> > > that leaves us with just the XI2 protocol to worry about.
> > 
> > True, though you still need to cater for XI 2.0 clients accessing the
> > devices for pointer/keyboard events. These two need to be synchronised.
> 
> I'm not sure I follow; can you explain what you mean by "These two need
> to be syncronized"?

assume a device that can send both ABS_MT_* and normal pointer/key events.
you'll have XI2 clients using the pointer/keyboard events from this device
and XI 2.1 clients using the touch events. 
- if an implicit passive grab activates in response to a button press event,
  what happens to the touch events?
- if an active grab is set by a 2.0 client on pointer events, what happens
  to touch events?
- if a touch emulates pointer events and any of the above two cases happens,
  what really happens?

things like that need to be clear, especially to us since we're supposed to
implement this without too much guesswork :)

> > Also, fwiw, it may not be necessary to float the devices after all, though
> > I haven't quite figured that out in detail yet.
> 
> Hrm? I'm eager to hear your thoughts on how else to handle DIDs.

give me a bit of time, I'm still rewriting too much of the current draft.

> > > To facilitate clients' need to know if the device is transient or not,
> > > and thus to grab in the proper way, I suggest adding a new "use" type
> > > called "TransientSlave" that is returned by XIQueryDevice.
> > > 
> > > I haven't put everything together yet, but I foresee using the direct
> > > devices (leaving aside the direct axes) approach in the commits I wrote,
> > > but renaming "direct" to "transient". Then, combine this with Carlos'
> > > evdev branch plus mtdev for proper transient input device pooling.
> > > 
> > > BTW, a lot of my misunderstandings come from not knowing exactly what
> > > device grabs are used for, and how they are used. Please let me know if
> > > there are other reasons for grabbing pointer devices that would break
> > > this approach.
> > 
> > grabs mean "don't send events to anyone but me". Events during grabs have a
> > reference to the grab window, hence you know if the event was within the
> > window or outside. Passive grabs simply do that in response to a button/key
> > + modifier combination instead of directly requested. The most common uses
> > these days are menus and window management.
> 
> Is there any documentation on how grabs are used for menus and window
> management? I pulled down the gtk source hoping that there might be some
> nuggets of implementation details I could figure out, but I haven't
> grepped through it all yet (and maybe some of the important stuff is in
> metacity?).

not sure, tbh. I learned most of it by looking at protocol events which
isn't the most straighforward way of doing so...
Carlos might be able to point you to it though.

Cheers,
  Peter