[RFC PATCH] New XCB socket handoff mechanism for Xlib/XCB and other libraries

[Jamey Sharp]

Libraries like Xlib/XCB, some XCB language bindings, and potentially
others have a common problem: they want to share the X connection with
XCB. This requires coordination of request sequence numbers. XCB
currently has an Xlib-specific lock, and allows Xlib to block XCB from
making requests. The attached patches to XCB replace that lock with a
handoff mechanism, xcb_take_socket, allowing external code to ask XCB
for permission to take over the write side of the socket and send raw
data with xcb_writev. The caller of xcb_take_socket must supply a
callback which XCB can call when it wants the write side of the socket
back to make a request. This callback synchronizes with the external
socket owner, flushes any output queues if appropriate, and then returns
the sequence number of the last request sent over the socket.

The attached patch to Xlib/XCB converts it to use the new handoff
mechanism. Previously, Xlib/XCB used XCB's Xlib lock to prevent XCB from
sending requests between calls to Xlib's LockDisplay and UnlockDisplay
macros. Xlib/XCB then sent all of its requests using XCB's
xcb_send_request, and had to flush its requests when unlocking the
display.

Using the new handoff mechanism, we replaced much of the existing
Xlib/XCB implementation with the use of xcb_take_socket to take
ownership of the write side of the X connection socket, and a
return_socket callback which writes any outstanding requests with
xcb_writev. This approach allows Xlib/XCB to use the same buffering as
traditional Xlib did. In particular, programs which use Xlib/XCB and
never make XCB calls will never need to hand the socket back to XCB, and
vice versa.

This allows us to discard large quantities of synchronization code from
Xlib/XCB, together with the synchronization bugs present in that code.
Several test cases which previously failed now work perfectly, including
multi-threaded ico. In addition, the infamous locking correctness
assertions, triggered when double-locking or when unlocking without a
previous lock, no longer exist, because Xlib/XCB no longer has any
reason to care more about application locking than traditional Xlib
does.

Let's repeat that for emphasis: the locking assertions *no longer
exist*. Existing Xlib applications with buggy locking will now work just
as well, or as badly, as they did with traditional Xlib.

Furthermore, the handoff approach provides great improvements to
performance. Results from x11perf's XNoOp test, which represented the
worst case for the lock-based Xlib/XCB:

Traditional Xlib:       average 19100000/sec
Lock-based Xlib/XCB:    average  3350000/sec
Handoff-based Xlib/XCB: average 17400000/sec

Thus, for no-ops, the handoff mechanism provides more than a 4x speedup
to Xlib/XCB, bringing Xlib/XCB within 9% of traditional Xlib no-op
performance. Of course, real-world workloads do not use no-op, so your
mileage may vary. In particular, since no-ops represent the worst case,
we expect real workloads to more closely match the performance of
traditional Xlib.

Finally, we anticipate that this new handoff mechanism will prove useful
for more than just Xlib/XCB. In particular, several authors or potential
authors of language bindings based on the XML-XCB protocol descriptions
have expressed an interest in implementing the protocol code in their
native language, rather than simply using a foreign function interface
(FFI) to call XCB's protocol layer. In the past, we have recommended
against this, because then other X client code in the same process (such
as other libraries for which the language uses an FFI) could not share
the same X connection. Those language bindings could now use XCB's new
handoff mechanism when sending requests from their own protocol layer,
and rely only on XCB's transport layer.

We blame Keith Packard for the fundamental idea of the handoff
mechanism, and for most of its advantages.

Please review and provide comments. We hope to commit these patches and
release new versions of XCB and Xlib/XCB soon.

- Josh Triplett and Jamey Sharp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xcb-0001-Remove-libxcb-xlib-and-xcbxlib.h.patch
Type: text/x-diff
Size: 10389 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xcb-0002-Inline-_xcb_lock_io-_xcb_unlock_io-and-_xcb_wait_i.patch
Type: text/x-diff
Size: 6507 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment-0001.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xcb-0003-Use-sequence-number-ranges-in-pending-replies.patch
Type: text/x-diff
Size: 2074 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment-0002.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xcb-0004-Support-handing-off-socket-write-permission-to-exter.patch
Type: text/x-diff
Size: 10139 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment-0003.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xcb-0005-Bump-version-number-to-1.2.patch
Type: text/x-diff
Size: 593 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment-0004.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: xlib-0001-Use-XCB-s-new-socket-handoff-mechanism-rather-than-t.patch
Type: text/x-diff
Size: 21537 bytes
Desc: not available
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment-0005.patch>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.x.org/archives/xorg/attachments/20080315/75acd23a/attachment.pgp>