[RFC] Multitouch support, step one

[Henrik Rydberg]

Daniel Stone wrote:
> Hi,
> 
> On Mon, Mar 15, 2010 at 03:41:24PM +0100, Henrik Rydberg wrote:
>>> Multi-touch as defined in this proposal is limited to single input-point
>>> multi-touch. This is suitable for indirect touch devices (e.g. touchpads)
>>> and partially suited for direct touch devices provided a touch is equivalent
>>> to a single-gesture single-application input.
>> User-space applications need tools to *use* MT devices, not route raw data from
>> the devices to the application. The latter is not much more complicated than
>> opening a file, and everyone can do that already. Thus, unless there is a model
>> for how MT devices work and interact with other MT devices, I see little point
>> in having an X protocol at all.
> 
> This is true, but we actually want to punt responsibility to the clients
> here; some kind of common gesture-recognition library would be nice to
> avoid duplicate work.

Yep. I envision splitting the current multitouch driver into two parts, one
contact driver part that goes into evdev, and one gesture part that forms the
basis for that library.


>> The number of dimensions also increases dramatically, as pointed out by Mr.
>> Poole. It makes much more sense to define contacts as multiple instances of the
>> same thing, than to define each new contact as potentially something completely
>> different.
> 
> Sure, but we don't have the knowledge to classify it as such.  So it
> means that we'd have to send the event to the client, then have the
> client make probably multiple round trips to update the grouping.  And
> since the server doesn't have anything meaningful to do with the
> grouping, the only benefit would be in the state the client sees.  This
> would be pretty disastrous for latency.

I believe this all depends on the type of application we are talking about.
Updating a pointer is one thing, detecting a four-finger gesture something else.
I can see the need to support both. Anyway, with no foreseeable constraints on
the number of valuators in XI2, I am (gruntily) okay with the valuator approach,
rendering this question somewhat academic.


>> The idea of adding and removing contacts dynamically I believe is a good idea. A
>> contact has a set of attributes (x, y, etc). Why not provide a clean interface
>> for the contacts as a concept, rather than mapping the not-so-independent x and
>> y values into separate dynamic entities? As an example of the smallest
>> meaningful dynamic entity:
>>
>> struct Contact {
>> 	int tracking_id;
>> 	float x, y;
>> 	etc etc...
>> };
> 
> The x, y, etc are the valuators and you have a tracking ID, so Peter's
> proposed API seems to fit this (it'd be nice if the server only
> _actually_ recycled tracking numbers when all buttons/fingers were up,
> or somesuch).

Using the valuator mask rather than dynamic allocation of valuators makes the
two formulations more or less equivalent, yes, so no problem here.


>>> Core requires us to always send x/y, hence for core emulation we should
>>> always include _some_ coordinates that are easily translated. While the
>>> server does caching of absolute values, I think it would be worthwile to
>>> always have an x/y coordinate _independent of the touchpoints_ in the event.
>>> The driver can decide which x/y coordinates are chosen if the first
>>> touchpoint becomes invalid.
>> Seconded, but the single-touch x/y coordinates are properties of a contact
>> group, not of a single contact. Example:
>>
>> struct ContactGroup {
>> 	int group_id;
>> 	float x, y;
>> 	ContactList list;
>> 	etc etc...
>> };
> 
> Yep, and if the toolkit is doing the grouping internally, then the
> toolkit can handle this.  As I said, this could be a common library.

The comment here was that the "main" x/y coordinates really belong to a group,
but the x/y coordinates are part of the generated contact stream, and can thus
not be handled by the toolkit. As long as the x/y pair sent in the contact
stream is well defined, it does not matter that much -- the client can always
define additional x/y coordinates with other properties if needed.


>>> A device exposing several axes _and_ multitouch axes will need to be
>>> appropriately managed by the driver. In this case, the "right" thing to do
>>> is likely to expose non-MT axes first and tack the MT axes onto the back.
>>> Some mapping may need to be added.
>>>
>>> The future addition of real multitouch will likely require protocol changes.
>>> These changes will need to include a way of differentiating a device that
>>> does true multitouch from one that does single-point multi-touch.
>>>
>>> That's it, pretty much (well, not much actually). Feel free to poke holes
>>> into this proposal.
>> Ok, in conclusion, my two cents are: Do not add MT functionality as evaluators
>> in X, but implement a proper Contact interface from the start.
> 
> Valuators are the lowest common denominator interface we can implement.
> If we develop something better on top that we can move into the protocol
> without completely blowing input response time out of the water, then we
> can look at that and deprecate the use of valuators for MT.

Sounds good.

Cheers,
Henrik