[RFC] Multitouch support, step one

[Rafi Rubin]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 03/17/10 20:02, Peter Hutterer wrote:
> On Wed, Mar 17, 2010 at 07:58:08PM +0100, Benjamin Tissoires wrote:
>> Le 16/03/2010 02:58, Peter Hutterer a écrit :
>>>> An other point for keeping the valuator trackingID. Some device
>>>> (stantum and magicmouse) send a trackingID different than touch
>>>> point. i.e. the trackingID is between 1 and 255 on the stantum, and
>>>> between 1 and 16 on the magic mouse. I don't know if the user app
>>>> should use this, or a higher level id could do the job.
>>>
>>> the tracking ID can be abstracted away in the driver. In fact, if we chose
>>> the valuators to be representative the tracking ID can fall away
>>> immediately. i.e. valuator n is _always_ touchpoint m.
>>>
>>> Of course, if we instead choose to have touchpoints dynamically assigned,
>>> then we need to send the tracking ID down the wire as well. I'm not a big
>>> fan of this at this point but I can be convinced otherwise.
>>
>> I've got an interesting argument to keep the tacking ID. The Magic
>> Mouse has a very 'intelligent' algorithm to assign the tracking ID:
>> it seems to increment as usual the different tracking IDs, but if
>> you release your touch, take a coffee or whatever, and even touch
>> other zones of the device; then, when you re-touch the same initial
>> zone, you will get the same tracking ID.
>>
>> I don't know if I'm enough clear, but this particular device keeps
>> the names of the different tracking IDs.
>>
>> I don't know either if it will be useful for client applications,
>> but it is something to take into account.

Just to stir things up a little more, I'd argue that tracking can be seen as
just one of several spatial reporting strategies.  As Henrik pointed out, it
mostly just serves to reduce the computational burden of the tracking code.
There are several other reporting methods that if known could also be used to
improve the efficiency of the tracking code.

For example my screen reports contacts from bottom to top.  So we know that to
track a contact we don't have to do a full N^2 Euclidean distance computation to
determine if two contacts have swapped order in the list.

If we can identify a set of strategies or scan patterns, it might make more
sense to just report the contacts in the order they come in and let the tracker
use that knowledge.  In that scheme dense reporting of tracked contacts with
tracking id makes sense if the common case is a sparsity of over 50%.  We emit
no more events if we report holes with a single event instead of id on active
contacts.  The magic mouse example (crazy or not), clearly demonstrates a device
where the common case is considerably more than 50% sparsity and therefore
justifies including the hardware tracking ids (what use is it to remember which
finger was in that spot, 15 minutes after you walk away from the computer).


Of course this doesn't make as much sense if we follow Henrik's argument about
suppressing motion until the current values have changed enough.  But there
again, do we emit just the one contact or the whole group?  If the whole group,
then my argument stands.  As for reporting only subsets of the contacts as
needed, we would either have to move tracking to the kernel, or be pretty
careful about the conditions for "enough".

Also from a pure efficiency argument, it would seem to make most sense to reduce
bandwidth as early as possible, which also suggests moving tracking to the
kernel.  Though I definitely do not think each driver should be responsible for
the code, more of putting the contact manager in the kernel, if efficiency is
the priority.


In another dependent argument, any approach that holds while motion is bellow a
threshold needs some way to keep the unmoving contact alive to the higher
levels.  The solution is likely dependent on the approach.  If all contacts are
reported whenever a change occurs, a simple "all contacts gone" event should
suffice.  In the case where all contacts are constantly reporting, we can
continue to get away without anything, but it still might be nice to have that
"all contacts gone" event.  In the more sparse protocols, heart beats would
work, but so would a "contact changed" event.  Much of that would simply be
covered by the addition of a "contact count" event (though clearly in some cases
that value needs double checking from what the hardware reports).


One final thought about the emit only after enough has changed.  If we consider
three values: last emitted by the kernel, previous hardware position, and
current hardware position, we can use the previous and current to determine if a
contact is consistent.  With that, the kernel can save the tracker some work
even on the sloppiest hardware.  We just have to keep a map of currently used
id's and assign one that has been cleared if the kernel has any doubts.

>>> [snip]
>>>
>>>>>>> Work needed:
>>>>>>> - drivers: updated to parse ABS_MT_FOO and forward it on.
>>
>> already started to work on that, to provide a proof of concept (code
>> still not published as I need to reorganize parts of it).
>>
>>>>>>> - X server: the input API still uses the principle of first + num_valuators
>>>>>>>   instead of the bitmask that the XI2 protocol uses. These calls need to be
>>>>>>>   added and then used by the drivers.
>>
>> as soon as it will be available, I can put it in the piece of
>> patches I'm working on. But currently, the workaround I told before
>> (packing all mt values at the beginning of the mt segment in
>> valuators, and sending tracking ID) is working quite well.
> 
> I actually expect this to be the largest change since a fair bunch of code
> utilises the first + num approach. All that has to be switched to
> masked-based approaches now. This is just a heads-up to not get too
> disappointed :)
> 
>>> oh, I'm not suggesting modifying the data from the kernel. I'm suggesting
>>> to add a label for ABS_MT_PRESSURE in case the kernel will get this in the
>>> next revision and the devices will start sending this information. If a
>>> device doesn't have it, then we'll never see it anywhere anyway.
>>
>> In commit a34812b09000db2ff2a1dc6182602839123edd4e "Add labels for
>> multitouch valuators", you made me add ABS_MT_PRESSURE. So it's at
>> the same level than any other ABS_MT_*.
> 
> I did? 
> wow. sometimes I impress myself.
>  
> Cheers,
>   Peter

Anyway, just some of my thoughts in response to some of the issues that have
come up in this thread.  And since Peter's impressing himself, just thought we
should join in.

Rafi
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkui9/YACgkQwuRiAT9o60+mugCeIniEqj3gu+FVV6F9AVwpFaDm
798AoM8sxFkvok55OILBwxD4KSFqQnZc
=y1oN
-----END PGP SIGNATURE-----