multitouch

[Daniel Stone]

Hi,

On Mon, Mar 01, 2010 at 02:56:57PM +0100, Bradley T. Hughes wrote:
> On 03/01/2010 01:55 PM, ext Daniel Stone wrote:
>> I don't really see the conceptual difference between multiple devices
>> and multiple axes on a single device beyond the ability to potentially
>> deliver events to multiple windows.  If you need the flexibility that
>> multiple devices offer you, then just use multiple devices and make your
>> internal representation look like a single device with multiple axes.
>
> This is where the context confusion comes in. How do we know what the  
> user(s) is/are trying to do solely based on a set of x/y/z/w/h 
> coordinates? In some cases, a single device with multiple axes is enough, 
> but in other cases it is not.

Sure.  But in this case you don't get any extra information from having
multiple separate devices vs. a single device.  The only difference --
aside from being able to direct events to multiple windows -- is the
representation.

> On a side note, I have a feeling this is why things like the iPhone/iPad 
> are full-screen only, and Windows 7 is single-window multi-touch only.

Rather.

I'm tempted to punt responsibility here by saying:
  * implement the multi-level device tree as described before
  * have every touchpoint as a separate device
  * if you want to grab fingers separately, then shift those devices to
    a new MD

But that's assuming hardware exists which can do reliable finger
detection.  Failing that, it's all a bit pointless, so ...

>> Given that no-one's been able to articulate in much detail what any
>> other proposed solution should look like or how it will actually work
>> in the real world, I'm fairly terrified of it.
>>
>> Can you guys (Bradley, Peter, Matthew) think of any specific problems
>> with the multi-layered model? Usecases as above would be great, bonus
>> points for diagrams. :)
>
> I'm concerned about the event routing and implicit grabbing behaviour,  
> specifically. I don't know enough about the internals to really put my  
> concerns into words or link to code in the server.
>
> Use-cases? Collaboration is the main use-case. Class rooms, meeting 
> rooms, conferences are ones that I often think about. Think about the 
> GIMP having multi-user and multi-touch support so that art students could 
> work together on a multi-touch table top. I think the MS Surface 
> marketing videos are a good indication of what could be done as well.
>
> One thing that we definitely want is for normal button and motion events 
> for one of the active touch-points over a client window. As Peter pointed 
> out, we shouldn't have to rewrite the desktop to support multi-touch. In 
> addition to specialized applications like I described above, we 
> definitely want "normal" applications to remain usable in such an 
> environment (I can definitely see someone bringing up a terminal and/or 
> code editor just for themselves to try out an idea that they get while in 
> a meeting).
>
> (Sorry for the lack of diagrams, my ascii-art kung-fu is non-existent. 
> How about a video? http://vimeo.com/4990545)

If the hardware is intelligent enough to be able to pick out different
fingers, then cool, we can split it all out into separate focii and it's
quite easy.

Failing that, how are we supposed to do it? Say two people have a
logical button press active (mouse button, finger down, pen down,
whatever) at once.  Now a third button press comes along ... what do we
do? Is it a gesture related to one of the two down? If so, which one
(and which order do we ask them in, etc).  A couple of years ago we
still could've guessed, but as Qt and GTK are now doing client-side
windows, it's really hard to even make a _guess_ in the server.

None of this even really relates to legacy clients or not, it's just
deciding how to deliver the events to anyone at all in the first place.
How do you do it -- especially without causing unacceptable latency?

I'm fine with the concepts and whathaveyou, but have you any thoughts on
how the event delivery flow should look like?

Cheers,
Daniel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg-devel/attachments/20100301/5547813e/attachment.pgp>