How to help on multitouch?

[Chase Douglas]

On Tue, 2010-06-15 at 16:25 +1000, Peter Hutterer wrote:
> On Sun, Jun 13, 2010 at 10:28:04PM -0400, Chase Douglas wrote:
> > Second, I was thinking about the DIDs approach, but I have some concerns
> > about situations where legacy apps don't listen for them. My thought is
> > that if a legacy app has not registered to listen for DIDs, the X server
> > should do some work on its behalf. The server could create one master
> > pointer for each legacy app and warp it to the location of the first
> > touch in the app. 
> 
> The core protocol (i.e. legacy apps) does not permit devices to disappear.
> This isn't much of an issue with the current devices but touchpoints appear
> and disappear all the time. So for each touchpoint that you want to expose
> as a core device, you'd have to keep the device around in case the client
> requests a grab, etc. This is an issue in the MPX model already but much
> less of a problem because one doesn't unplug mice and keyboards all the time.
> 
> Example case: touch comes in, sends button press), touch disappears, client
> requests grab on this device. Now you can grab any other device to be
> protocol-correct but that's not what the user will expect.
> 
> > Multiple touches beyond the first inside such an app
> > do nothing. This would allow multitouch DIDs to control legacy apps
> > while avoiding some of the master pointer dynamic allocation issues that
> > required multitouchd to have a pool of pre-allocated pointers for.
> 
> How do you define "inside" an application? Clients can register for events
> on windows that they do not own - does this count as "inside"?
> Note that this is already solved with the Grab Ownership approach in MPX,
> but again, touch is trickier than mice/keyboards and grab ownership doesn't
> quite apply here unless you somehow tag new touchpoints with the touch group
> they belong to.

Hrm. I need to read up a bit more on X input handling. I'm in the middle
of a big RTFM and RTSL effort to do this :).

> > Lastly, I have concerns about keyboard focus with multitouch. I can't
> > think of a way to have the server manage focus with the help of DIDs due
> > to the fact that multiple apps may be used at the same time. However,
> > even when an app is used it may not need to become focused. For example,
> > if you are just making some motion to scroll a window that is partially
> > behind the focused window, you might not want to focus it. So, I think
> > the best way to handle focus would be to have the application receiving
> > DID events ask for focus if it feels it should gain it.
> 
> Keyboard is not handled by the X server, it's handled by the client. So
> you're fighting against the legacy clients here. If the WM issues a
> SetInputFocus request you need to know which keyboard to focus into the
> particular window. The ClientPointer principle tells you that with
> reasonable predictabilty but again, touch points are shortlived and if you
> pool them together with multiple keyboards that becomes a bit trickier.
> I don't have a good solution to this yet other than "fix the WM".
> 
> You can't easily deny a SetInputFocus request btw. Though the protocol
> allows for some errors (BadMatch, BadValue and BadWindow), sending them may
> cause the applications to behave weirdly. Because BadMatch is the only one
> you can really pick here anyway but if the application also tracks windows
> it's likely to become confused.
> 
> > I'd love to hear others' thoughts on these ideas. I'm really new to X
> > development, so I may have some concepts wrong in my head, so please let
> > me know if I've got anything wrong :).
> 
> I really think the only way out of this mess is pure DIDs, which allow us to
> ignore most or all of the core protocol range. Of course, that gets
> interesting again with XTest, RECORD and other extensions but ... well,
> urgh.

Yeah, I completely agree that pure DIDs is the best approach overall.
However, we still need to figure out what to do about the case where the
touchscreen is your only pointer interface (i.e. a tablet). If we don't
have some way to tie DIDs to pointers at all, then nothing will work.
What I'm suggesting is moving the linkage between touches and pointers
away from the incoming side of XInput (where we don't know what a touch
should really represent) and towards the outgoing side of XInput (where
we have more context for figuring out what a touch represents). Thus the
idea of creating core pointers for each app or window when they are
created, and warping them around to follow DIDs.

Thanks,

-- Chase