New approach to multitouch using DIDs and bitmasked events
Daniel Stone
daniel at fooishbar.org
Mon Aug 23 06:32:17 PDT 2010
Hi,
Sorry for the delay on this one - holidays, moving continent, work, etc.
I've been approaching this from a slightly different standpoint. Let me
explain ...
These were my goals/invariants:
* be conservative in event delivery - better to drop an event and
require a second press than to accidentally deliver an event to
two clients
* touches do not change targets during their lifetime, akin to the
implicit grab on button press
* touch tracking must be done on the server side, if not already
done by the driver
* have the least painful API possible for clients and drivers alike
* the simplest solution is probably the best
* keep latency (i.e. event generation -> client reception) as low as
possible
I feel that DIDs are ruled out by #4 and #5. Event delivery is already
fairly nightmarishly complex, and I don't think we want to make it any
more complicated here.
Server-side gesture recognition is ruled out by #6, as well as others:
gestures aren't uniformly meaningful, and interpreting them correctly
requires knowledge that can only be found in the client. Similarly, I'm
not entirely convinced that it will be possible to develop a usefully
generic gesture engine for quite a few reasons, including but not
limited to licensing.
This also rules out anything that requires a round trip before event
delivery can actually occur, e.g. an XEvIE-like approach where input
events are first sent to a client which mangles the event and/or tells
the server where to direct it, before the server later sends it out for
real.
The approach I took was to introduce a separate TouchClass, which reuses
existing valuators rather than adding a new TouchAxisClass. We already
have valuators which (IMO) already have all the semantics we need.
Here's the lifecycle of a typical touch event:
* TouchNotify event sent to selecting client with detail TouchBegin,
which includes a bitmask of valuators used for the touch.
* All motion events as part of this touch are sent to that client
only.
* When the finger is lifted, TouchNotify event sent with TouchEnd.
'Selecting client' has the same meaning as it does for MotionNotify and
co: walk back up the tree from the window immediately under the finger
until we find a client selecting for TouchNotifies. I don't believe
that grabbing in response to a touch is practical: at best, you miss the
first events that started the touch, and at worst, you miss the touch
completely.
Eagle-eyed readers might note that this only covers absolute devices
rather than relative. I can't see any way we can cleanly support
multiple focii with touchpads, nor any usecase for doing so.
Relative-mode devices thus always use the current focus of the device as
the basis for delivery.
As touch events don't generate core events[0], a core grab has no effect
on the further delivery of touches. I haven't covered XI2 grabs in my
first draft, but I think Peter's proposal for those looks reasonable,
with the exception of the num_touches field.
I've attached the protocol diff, and the driver API looks like this:
int touch_id = xf86TouchBegin(dev, tool);
if (touch_id < 0)
FatalError("mein leben\n");
xf86PostTouchEvent(dev, touch_id, x, y, touch_maj, touch_min,
width_maj, width_min, orientation);
void xf86TouchEnd(dev, touch_id);
I think this is the cleanest approach so far, which requires minimal
work on the toolkit side of things while still doing everything we need
to. Am I missing something?
Cheers,
Daniel
[0]: That being said, I have explicitly designed the driver-side API to
allow for core event emulation in the future, so this only refers
to touches in the XI2 protocol sense of the word, rather than the
xf86 input API.
diff --git a/XI2proto.txt b/XI2proto.txt
index 706f50a..082ac2c 100644
--- a/XI2proto.txt
+++ b/XI2proto.txt
@@ -1,16 +1,20 @@
The X Input Extension
- Version 2.0
+ Version 2.1
Peter Hutterer
peter.hutterer at redhat.com
Red Hat, Inc.
+ Daniel Stone
+ daniel at fooishbar.org
+ Collabora Ltd.
+
1. Introduction
-The X Input Extension version 2.0 (XI2) is the second major release of the X
+The X Input Extension version 2.1 (XI2) is the second major release of the X
Input Extension.
XI2 provides a number of enhancements over version 1.5, including:
@@ -31,6 +35,9 @@ used on applications employing the core protocol. XI2 addresses both of these
issues by enabling devices to be both extended and core devices and providing
device information in each event (with the exception of core events).
+XI 2.1 introduced the notion of 'touches' in order to better support
+multitouch devices.
+
❧❧❧❧❧❧❧❧❧❧❧
2. Notations used in this document
@@ -61,7 +68,7 @@ COMPLEXFIELDTYPE: { name of subfield: type of subfield,
❧❧❧❧❧❧❧❧❧❧❧
-3. Interoperability between version 1.x and 2.0
+3. Interoperability between version 1.x and 2.x
There is little interaction between 1.x and 2.x versions of the X Input
Extension. Clients are requested to avoid mixing XI1.x and XI2 code as much as
@@ -295,6 +302,15 @@ are required to be 0.
max: FP3232
value: FP3232
resolution: CARD32 }
+ TOUCHCLASS* { type: TouchClass
+ length: CARD16
+ sourceid: CARD16
+ touchid: CARD32
+ tool: CARD32
+ valuators_len: CARD16
+ valuators_mask: SETofVALUATORMASK }
+
+ *: since XI 2.1, must not be sent to 2.0 clients
XIQueryDevices details information about the requested input devices.
@@ -397,6 +413,23 @@ are required to be 0.
An axis in Relative mode may specify min and max as a hint to the
client. If no min and max information is available, both must be 0.
+ TouchClass:
+ type
+ Always TouchClass.
+ length
+ Length in 4 byte units.
+ sourceid
+ The device this class originates from.
+ touchid
+ An identifier describing this touchpoint.
+ tool
+ The physical tool used to generate this touch.
+ valuators_mask
+ The valuators currently used by this touchpoint.
+
+ For more information, please see the TouchNotify section.
+
+
┌───
XISelectEvents
window: Window
@@ -789,6 +822,7 @@ are required to be 0.
masks: SETofEVENTMASK
▶
status: Success, AlreadyGrabbed, Frozen, InvalidTime, NotViewable
+
└───
This request actively grabs control of the specified input device. Further
@@ -892,7 +926,10 @@ are required to be 0.
time: TIMESTAMP or CurrentTime
event_mode: { AsyncDevice, SyncDevice,
AsyncPairedDevice, SyncPairedDevice,
- ReplayDevice, AsyncPair, SyncPair }
+ ReplayDevice, AsyncPair, SyncPair,
+ ReplayTouch* }
+ detail*: CARD32
+ * since XI 2.1
└───
The XIAllowEvents request releases some queued events if the client
@@ -905,6 +942,9 @@ are required to be 0.
event_mode
Specifies whether a device is to be thawed and events are to be
replayed.
+ detail
+ If applicable, provides an additional detail to match; interpretation
+ depends on the event_mode.
The request has no effect if the specified time is earlier than the
last-grab time of the most recent active grab for the client, or if the
@@ -978,6 +1018,14 @@ are required to be 0.
thaws for both. AsyncPair has no effect unless both the device and the
paired master device frozen by the client.
AsyncPair has no effect if deviceid specifies a slave device.
+ ReplayTouch
+ If the client has an active touch grab on the touchid specified in
+ detail, the touch stream for that touchid is thawed, the grab is
+ released and that event is completely reprocessed. This time, however,
+ the request ignores any passive grabs at or above (towards the root) the
+ grab-window of the grab just released.
+ The request has no effect if the specified device is not grabbed by
+ the client or if it is not frozen as the result of an event.
┌───
XIPassiveGrabDevice
@@ -999,12 +1047,12 @@ are required to be 0.
└───
GRABTYPE { GrabtypeButton, GrabtypeKeycode, GrabtypeEnter,
- GrabTypeFocusIn}
+ GrabTypeFocusIn, GrabTypeTouchBegin }
GRABMODIFIERINFO { status: Access
modifiers: CARD32 }
- Establish an explicit passive grab for a button or keycode
+ Establish an explicit passive grab for a button, keycode, or touch event
on the specified input device.
cursor
@@ -1015,7 +1063,8 @@ are required to be 0.
AllMasterDevices.
detail
The button number, or key symbol to grab for.
- Must be 0 for GrabtypeEnter and GrabtypeFocusIn.
+ Must be 0 for GrabtypeEnter and GrabtypeFocusIn. For
+ GrabTypeTouchBegin, this is taken to match the tool ID if non-zero.
grab_type
The type of grab to establish.
grab_window
@@ -1087,6 +1136,19 @@ are required to be 0.
- a passive grab of the same grab_type + modifier combination does not
does not exist on an ancestor of grab_window.
+ Or if grab_type is GrabTypeTouchBegin, a stream of touch events
+ (TouchBegin, all subsequent motion events, and TouchEnd) is grabbed
+ for exclusive delivery to that client if:
+ - the specified modifier keys are down, and
+ - if detail is non-zero, tool matches the detail, and
+ - the touch is generated from an absolute device and the co-ordinates
+ of the first event are within grab_window or a descendant of
+ grab_window; or if the touch is generated from a relative device and
+ the device's focus is currently within grab_window or a descendant
+ of grab_window, and
+ - a passive grab of the same grab_type + detail + modifier combination
+ does not exist on an ancestor of grab_window.
+
A modifier of GrabAnyModifier is equivalent to issuing the request for
all possible modifier combinations (including no modifiers). A client
may request a grab for GrabAnyModifier and explicit modifier
@@ -1097,6 +1159,8 @@ are required to be 0.
A GrabtypeEnter or GrabtypeFocusIn grab is released when the
pointer or focus leaves the window and all of its descendants,
independent of the state of modifier keys.
+ A GrabtypeTouchBegin grab is released when the touch is ended, as
+ signified by a TouchEnd event.
Note that the logical state of a device (as seen by means of the
protocol) may lag the physical state if device event processing is
frozen.
@@ -1109,7 +1173,7 @@ are required to be 0.
with the same button or keycode and modifier combination, the
failed modifier combinations is returned in modifiers_return. If some
other client already has issued an XIPassiveGrabDevice request of
- grab_type XIGrabtypeEnter or XIGrabtypeFocusIn with the same
+ grab_type XIGrabtypeEnter or XIGrabtypeFocusIn with the same
grab_window and the same modifier combination, the failed modifier
combinations are returned in modifiers_return. If num_modifiers_return
is zero, all passive grabs have been successful.
@@ -1148,7 +1212,7 @@ are required to be 0.
deviceid
The device to establish the passive grab on.
detail
- The button number or key symbol to ungrab.
+ The button number, key symbol, or tool ID to ungrab.
Must be 0 for GrabtypeEnter and GrabtypeFocusIn.
grab_type
The type of grab to establish.
@@ -1338,6 +1402,8 @@ Version 2.0:
FocusIn
FocusOut
PropertyEvent
+Version 2.1:
+ TouchNotify
All events have a set of common fields specified as EVENTHEADER.
@@ -1673,5 +1739,47 @@ EVENTHEADER { type: BYTE
what
Specifies what has been changed.
+ ┌───
+ TouchNotify
+ EVENTHEADER
+ sourceid: DEVICEID
+ detail: { TouchBegin, TouchEnd }
+ mode: { Absolute, Relative }
+ tool: CARD32
+ touchid: CARD32
+ valuators_len: CARD16
+ valuators_mask: SETofVALUATORMASK
+ └───
+
+ TouchNotifies are sent whenever a tool or finger comes into or out of
+ physical proximity from a touch device. This event can be used to track
+ the state of a multitouch device by associating touch identifiers with a
+ set of valuators.
+
+ Once a TouchNotify with detail TouchBegin has been sent to a client, all
+ events generated by that touch will be sent to that client, similar to
+ the implicit passive grab provoked by a ButtonPress. If a TouchBegin
+ event is generated but is not delivered to any client, the subsequent
+ motion events will be discarded.
+
+ sourceid
+ The physical device that provoked this event.
+ mode
+ If the mode is Absolute, then the touch is focussed at its original
+ position, scaled to screen co-ordinates. If Relative, then the current
+ device focus is used.
+ touchid
+ A unique identifier describing this touch; may be reused after TouchEnd
+ has been sent. Has no meaning in and of itself; see 'tool'.
+ tool
+ If the device is capable of differentiating touches from multiple tools
+ or fingers, this field may contain an implementation-defined identifier.
+ This can be used to, e.g., differentiate pen from eraser events on a
+ tablet, or differentiate fingers on a touchscreen.
+ valuators_len
+ The length of valuators in 4 byte units.
+ valuators_mask
+ The valuators affected by this event.
+
❧❧❧❧❧❧❧❧❧❧❧
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://lists.x.org/archives/xorg-devel/attachments/20100823/faa5e9c1/attachment.pgp>
More information about the xorg-devel
mailing list