multitouch

[Simon Thum]

Bradley T. Hughes wrote:
> On 01/18/2010 11:54 PM, ext Carsten Haitzler (The Rasterman) wrote:
>> hey guys (sorry for starting a new thread - i only just subscribed - lurking on
>> xorg as opposed to xorg-devel).
>>
>> interesting that this topic comes up now... multitouch. i'm here @ samsung and
>> got multi-touch capable hardware - supports up to 10 touchpoints, so need
>> support.
>>
>> now... i read the thread. i'm curious. brad - why do u think a single event (vs
>> multiple) means less context switches (and thus less power consumption, cpu
>> used etc.)?
> 
> Even though the events may be buffered (like you mention), there's no 
> guarantee that they will fit nicely into the buffer. I'm not say that this 
> will always be the case, but I can foresee the need to write code that scans 
> the existing event queue, possibly flushes and rereads, scans again, etc. to 
> ensure that the client did actually get all of the events that it was 
> interested in.
They other guys at my workplace do a touchtable, so I'm not particularly
qualified. But there's 10 points in carsten's HW already, and from what
I know it's not hard to imagine pressure or what not to become
important. That's 30 axes and a limit of 36 axes - if that's not easy to
lift I'd be wary of such an approach.

> There's also the fact that the current approach that Benjamin suggested 
> requires an extra client to manage the slave devices.

OTOH, if you're getting serious, there needs to be an instance
translating events into gestures/metaphors anyway. So I don't see the
point of avoiding an instance you're likely to need further on.


[...]
> be the case, but I can foresee the need to write code that scans the 
> existing event queue, possibly flushes and rereads, scans again, etc. to 
> ensure that the client did actually get all of the events that it was 
> interested in.
Maybe the best thing would be to allow for drivers and the protocol to
group events, such that the client knows when he has the whole group. A
group wouldn't be limited to a driver/device, and maybe get an own atom
to identify. Semantics of groups would be up to the client, with help of
props or whatever.

That's 2 additional Xi events, and could work quite good with all the
approaches outlined below. Most boilerplate code might go into libXi. Of
course, that's the classic fast vs. good tradeoff.

Just my 5c,

Simon


> 
>  > but as such you should be
>> seeing all these events alongside other events (core mousepress/release/motion
>> etc. etc. etc.). so i think the argument for all of it in 1 event from a
>> power/cpu side i think is a bit specious.
> 
> The power savings are probably going to minimal, that I agree to. My 
> argument has mostly been on the convenience side of having a single blob 
> (based on experiments we did early last year).
> 
>> but... do you have actual data to
>> show that such events actually dont get buffered in x protocol as they should
>> be and dont end up getting read all at once? (i know that my main loop will
>> very often read several events from a single select wakeup before going back to
>> sleep, as long as the events come in faster than they can be acted on as they
>> also get processed and batched into yet another queue before any rendering
>> happens at the end of that queue processing).
> 
> I don't have any data related to X11 on this, no.
> 
>> but - i do see that if osx and windows deliver events as a single blob for
>> multiple touches, then if we do something different, we are just creating work
>> for developers to adapt to something different. i also see the arguument for
>> wanting multiple valuators deliver the coords of multiple fingers for things
>> like pinch, zoom, etc. etc. BUT this doesnt work for other uses - eg virtual
>> keyboard where i am typing with 2 thumbs - my presses are actually independent
>> presses like 2 core pointers in mpx.
>  >
>> so... i think the multiple valuators vs multiple devices for mt events is moot
>> as you can argue it both ways and i dont think either side has specifically a
>> stronger case... except doing multiple events from multiple devices works
>> better with mpx-aware apps/toolkits, and it works better for the more complex
>> touch devices that deliver not just x,y but x, y, width, height, angle,
>> pressure, etc. etc. per point (so each point may have a dozen or more valuators
>> attached to it), and thus delivering a compact set of points in a single event
>> makes life harder for getting all the extra data for the separate touch events.
> 
> Indeed. There are cases where one is more convenient over the other and vice 
> versa. This is what we struggled with for a while when doing the Qt API for 
> multi-touch. In the end, we went with the single blob approach and tag each 
> point in the blob with pressed/moved/released state (so that it's possible 
> to cover both use cases).
> 
> The only thing that concerns me with the idea of sending each touch point as 
> a separate device is that it
> 
>> so i'd vote for how tissoires did it as it allows for more information per
>> touch point to be sanely delivered. as such thats how we have it working right
>> now. yes - the hw can deliver all points at once but we produce n events. but
>> what i'm wondering is.. should we....
>>
>> 1. have 1, 2, 3, 4 or more (10) core devices, each one is a touch point.
>> 2. have 1 core with 9 slave devices (core is first touch and core pointer)
>> 3. have 1 core for first touch and 9 floating devices for the other touches.
>>
>> they have their respective issues. right now we do #3, but #2 seems very
>> logical. #1 seems a bit extreme.
> 
> I agree, #1 sounds a bit extreme. An approach like 2 or 3 is also doable.
> 
>> remember - need to keep compatibility with single touch (mouse only) events and
>> apps as well as expand to be able to get the multi-touch events if wanted.
> 
> Exactly. Do #2 and #3 keep that compatibility? My understanding is that if 
> we did #2, then the master pointer would still deliver events for all slaces 
> (with DeviceChanged events mixed in between). Couldn't this confuse 
> non-mulit-touch and/or non-mpx aware clients?
>