How to help on multitouch?

[Peter Hutterer]

On Wed, Jun 16, 2010 at 03:27:48PM -0400, Chase Douglas wrote:
> On Tue, 2010-06-15 at 16:25 +1000, Peter Hutterer wrote:
> > On Sun, Jun 13, 2010 at 10:28:04PM -0400, Chase Douglas wrote:
> > > Second, I was thinking about the DIDs approach, but I have some concerns
> > > about situations where legacy apps don't listen for them. My thought is
> > > that if a legacy app has not registered to listen for DIDs, the X server
> > > should do some work on its behalf. The server could create one master
> > > pointer for each legacy app and warp it to the location of the first
> > > touch in the app. 
> > 
> > The core protocol (i.e. legacy apps) does not permit devices to disappear.
> > This isn't much of an issue with the current devices but touchpoints appear
> > and disappear all the time. So for each touchpoint that you want to expose
> > as a core device, you'd have to keep the device around in case the client
> > requests a grab, etc. This is an issue in the MPX model already but much
> > less of a problem because one doesn't unplug mice and keyboards all the time.
> > 
> > Example case: touch comes in, sends button press), touch disappears, client
> > requests grab on this device. Now you can grab any other device to be
> > protocol-correct but that's not what the user will expect.
> 
> I read the following to get a better idea of how things work:
> 
> http://www.x.org/wiki/Development/Documentation/InputEventProcessing

I just updated this page, it was quite outdated (true only for up to 1.5,
partially 1.6).

> http://www.x.org/wiki/Development/Documentation/MPX
> 
> My thinking is that when a legacy client connects (i.e. a client that
> does not tell the server it is DID/MPX aware)

You don't know if a client is MPX aware until a random time during that
client's runtime. A client may register for core events and then decide to
switch to XI2 lateron.

> , the server creates a new
> master pointer for it. It then core-grabs the new master pointer on
> behalf of the client.

Once you core-grab the master pointer, no other device can interact with
this client anymore. Breaking this rule for DIDs to have in-server grabs be
treated differently is...tricky.

> Now, when the first DID button down event occurs
> inside a window of the client, we start sending core events for the
> client's master pointer.

what happens if the client has registered for events on a window of a
different client? Do you then send the DID event to both clients or just
this one?

> (This is starting to sound like reinventing slave devices, only this
> time we would be mucking with the heirarchy on the client side of the
> server instead of the driver side of the server.)

There is no slave device on the driver side of the server. It's all in
the DIX. Drivers don't get a say in that at all.

> In legacy apps, when a second master pointer (i.e. representing
> traditional mice/trackpads) enters the window, we can intercept their
> core events and feed them through to the app as though it came from the
> client's own master pointer.

wait, you want to sync up two master devices? So what happens if you send a
button event from that second master through the DID master and then the
second device gets grabbed by a different client? do you release the button
that you've just pressed through the sync? because if not, the other MD grab
may get stuck and you never see the button release.
Aside from passive grab issues if you suddenly sync button events from an MD
through another one.

when it enters what window btw? Please stop thinking that clients are
somewhat window bound - they're not. A client can register for events on
_any_ window, including their own.

> With this approach, all legacy apps see one and only one master pointer.
> This avoids some of the grab issues with MPX.

And completely removes the benefits of MPX.

> If a client reports that it is DID/MPX aware, then we don't do anything
> special with it. It receives core events from master pointers and DID
> events through XI2. 

If a client announces itself to be MPX aware and registers for core input
events I'd be tempted to just close the fd on it...

> > > Multiple touches beyond the first inside such an app
> > > do nothing. This would allow multitouch DIDs to control legacy apps
> > > while avoiding some of the master pointer dynamic allocation issues that
> > > required multitouchd to have a pool of pre-allocated pointers for.
> > 
> > How do you define "inside" an application? Clients can register for events
> > on windows that they do not own - does this count as "inside"?
> > Note that this is already solved with the Grab Ownership approach in MPX,
> > but again, touch is trickier than mice/keyboards and grab ownership doesn't
> > quite apply here unless you somehow tag new touchpoints with the touch group
> > they belong to.
> 
> Can't we determine if a DID event is in a client's window by the same
> techniques used in DoEnterLeaveEvent()? It already determines which
> client and window the event occur

It only tells you if an event is inside a window. and you know which client
created that window. Since clients may register and unregister for events at
any time this is only one part of the picture.

Seriously, I appreciate your eagerness but at the moment your proposals may
sound good on paper but the devil's in the detail. You really need to read
up on the core protocol spec, the XI and XI2 protocol specs and especially
the event processing and grab handling in the server. If after that you
haven't committed yourself to a mental institute, feel free to draft up some
more plans but when I mentioned this issue is tricky in earlier emails, I
wasn't joking. Right now, I don't think your proposals have the required
depth.

Cheers,
  Peter