[RFC] Multitouch support, step one

[Benjamin Tissoires]

I do not want to break this discussion, but I think that the work you 
mentioned is a little bit aside of the original subject.

I rapidly had a look at the code and it seems to be a driver for 
trackpads with multitouch enabled. It's really a good job, but I thought 
the original proposal of Peter was to treat 'all' multitouch devices, 
including multitouch-screen.

In that case, I don't think this driver is appropriate as it only 
delivers one cursor move (thus reducing the bandwidth) which is really 
good in a trackpad case.

Personally, I would say that the work you mentioned is of great 
interest, and has to co-exist with the work we are making in this 
thread. But I think it is not enough broad as you treat only a sub-case 
of mt.

I will now comment some of your speach...

Le 16/03/2010 14:42, Henrik Rydberg a écrit :
> Peter Hutterer wrote:
>>[snip]
>
> 1. Consistent behavior for all devices
>
> The hardware stack supporting multitouch is diverse, and several different
> mechanisms and abstraction levels exists. The tracking ID is a good example. It
> may or may not be present in the driver output, and it may work poorly even if
> it exists. Thus, in order to support hardware consistently, there must be a
> middle layer outside of the kernel, parsing the driver data and patching it up
> to produce the same level of detail for all devices. This task can be quite
> complicated and uses some cpu, so having it in one place is imperative. Luckily,
> there exists such a solution in the multitouch X driver (see link above). This
> code can either be broken out as a standalone module or be placed in the X core.
> If there is a license issue, it can be resolved for the benefit of the X
> community. In the text below, this piece of code will be referred to as the
> contact driver.

really like the idea of offering the same interface for all devices. But 
it may just be a piece of code that can be reused by all the different 
drivers (I think it's what you already got, even if I did not found the 
tracking for N-Trig-like devices)

>
> 2. Bandwidth reduction should be made as early as possible
>
> The MT events from the kernel are non-filtered, bypassing the normal input
> filtering by necessity. Duplicating this behavior further into the food chain
> would be a mistake. After parsing the MT stream in the contact driver, the event
> stream can be filtered substantially, thereby restoring bandwidth usage to
> something more similar to non-mt devices.
>

It depends on the different uses. In case of a single user trackpad, ok. 
In case of a large multitouch display, not sure.

> 3. The contact driver produces the more digested contact events
>
> The contact driver takes the flora of driver MT events and produces a consistent
> stream of contact events. The contact event stream is less bandwidth-consuming,
> and follows the init-move-destroy concept we discussed last summer, if you
> recall. We are still talking about a low-level stream, there are no gesture or
> other high-level derivatives. Just a consistent stream of data.
>

You mean that the client can directly talk to the contact driver?

> 4. ABI, memory and cpu burden for nothing
>
> Although the currently hard limit of valuators most likely can be programmed
> away, it just feels wrong to burden all other applications with the additional
> memory and cpu usage implied by raising a comfortable limit to something much
> higher, only to satisfy the request of a completely different interface, which
> strains the existing concept to the limit of breakage.
>

~256 valuators should be enough as long as we don't have device that 
reports more than 20 touches. (I think that the current limit of the 
devices is around 10)
The point is that 32 axes is not enough for multitouch. We want it to be 
raised at 256 (the hard limit if I read correctly), but we don't know 
the side-effects.

> 5. Use appropriate data structures to solve the problem
>
> By defining a handful of contact api functions, operating on simple structures,
> the whole problem of forward compatibility with multi-user multitouch can be
> solved in one go, without changing a single bit in the existing interfaces. Yes,
> it means a new interface, but the functionality is new, so this is the way it
> should be.
>
> ---
>
> X Multitouch Support Proposal
> -----------------------------
>
> Introduction
> ------------
>
> Back in summer 2009, when this was discussed informally between some in the
> present party, the general structure that emerged was a split into a low level
> protocol and a gesture library, here citing two of the formulations:
>
>> Henrik Rydberg:
>> X multipointer via init-move-destroy to X gesture driver
>> X gesture driver via enhanced X events to X application
>>
>> Peter Hutterer:
>> X server ->  protocol ->  application
>>                            |->   x gesture library
>
> The change I am proposing today simply means inserting the contact driver
> discussed above before or in conjunction with the X server in Peter's chain:
>
> kernel ->  [contact driver | X server] ->  protocol ->  application
>                                                        |->   x gesture library
>
> The details of the protocol in this chain depends on the output of the contact
> driver, which is only slightly higher level than the kernel events.
>
> The Contact Driver
> ------------------
>
> The general structure of the MT events is that of contacts appearing, changing
> and disappearing. Because of the diversity of capabilities of the drivers, this
> structure is quite relaxed in the kernel stream, to the point that it requires
> work to fully impose this structure further down the stream. That is the job of
> the Contact Driver. It translates the relaxed kernel MT events into a steady
> stream of contact events, containing the same level of information for all
> drivers. The contact events follow the same logic as the MT events, but because
> all data is present, the init-move-destroy mechanism can be employed fully. Here
> is an example of what a two-finger scroll would look like:
>
> init id = 588, x = -234, y = 42
> init id = 123, x = 933,	y = 3
> sync
> move id = 588, x = -211, y = 529
> move id = 123, x = 863,	y = 732
> sync
> destroy	id = 588
> destroy	id = 123
> sync
>

currently the structure is the following:

- the device send a consistent packet of N touches.
- the kernel serialized it for some reasons (I did not follow the 
discuss on the kernel list)
- the X driver has to pack them all in order to being able to track the 
touches in the case the device does not send the trackingID

You want then to re-serialize the data? To which end?

Cheers,
Benjamin

>
> The X Protocol
> --------------
>
> The details here are well beyond my expertise, but I am suggesting a contact
> interface implementation based on the contact driver event structure. I cannot
> imagine this is harder than using the XI event structure, but should be a lot
> less of a headache for everyone.
>
> Cheers,
> Henrik