RFC: Use libxkbcommon in Xorg

[Kristian Høgsberg]

Hello,

At XDS I volunteered to make X use libxkbcommon and I've spent a few
days this week trying to make that happen.  libxkbcommon is a new
library started by Dan Nicholson which pulls in bits and pieces from
various XKB components, and primarily it provides a library interface
to the xkb compiler.  The idea is then to use libxkbcommon in Xorg to
compile the XKB keymaps instead of writing out a temp file and then
forking to run the compiler, all which is very messy.

We put the libxkbcommon library up here:

  http://cgit.freedesktop.org/xorg/lib/libxkbcommon

Since Dans and Daniels initial work on libxkbcommon I've added a few
changes myself towards making the library more self contained and the
public API as small as possible.  To that end I've copied over structs
from kbproto so that libxkbcommon becomes the new owner of those
structs and removed dependencies on libxkbfile.  At this point
libxkbcommon only depends on xproto (for keycodes and a few other
defines) and kbproto.  I added a flex generated lexer instead of the
handcoded one in xkbcomp - sadly this didn't give much of a
performance boost.  It did make it easy to make the compiler read from
a string, which avoids messing with pipes/temp files.

The Xorg side of the work is over here:

  http://cgit.freedesktop.org/~krh/xserver/log/?h=xkbcommon

and the diffstat for the changes look pretty good:

  [krh at hinata xserver]$ git diff --shortstat origin/master HEAD
   66 files changed, 1554 insertions(+), 5845 deletions(-)

especially considering that among the deleted code is the fork of
xkbcomp and code to create and read tmp files and the
/usr/share/X11/xkb cached XKM file.  As for caching the compiled
keymap, I'm not really sure that we need to do that.  I tried timing
500 compilations (including forking the small test binary) of a map
and it comes in at around 10ms.  If anybody wants to try that, use the
rulescomp binary in the test/ directory of libxkbcommon but don't run
it with the libtool wrapper, that's going to account for 50% of the
time... even better write a dedicated test program that calls the
compiler entry point in a loop and profile that.

Now, it's all looking pretty promising, but there are a few open
issues I'd like to throw to the list.  First of all, there's the issue
of how this intersects/dovetails/conflicts with xkb2.  Part of the
original plan for libxkbcommon as part of xkb2 was to also increase
the size of certain datatypes in xkb: bump keycodes and virtual mods
to 32 bits.  We can't do that with the existing xkb protocol and will
require xkb2 protocol with new requests to serialize the xkb
descriptions and we need to rethink some of the ways we represent
certain state (per keycode repeat info makes a pretty big bit vector
for 32 bit keycodes).  I don't think that's realistic for a 1.10
xserver release, and I've changed the libxkbcommon structs back to be
compatible with the current xkb protocol and implementation.  We can
extend libxkbcommon to support bigger keycodes and vmods in a later
release.  It's not going to be as pretty as if we did it all in the
first release, but I think there's too much potential here to block it
on xkb2.

The remaining issues are smaller and easier to sort out;

1) the libxkbcommon API still needs review cleaning up from an
namespace point of view.

2) I'm considering whether to include a copy of the Xorg keysym under
a XKB_KEY_* namespace to break the dependency on xproto?

3) Right now libxkbcommon is essentially just an xkb keymap parser
library, should we include the xkb state machine logic too (basically
the action processing from xkbAction.c and the keysym lookup logic
from libX11)? Any user of libxkbcommon (xcb, wayland, clutter with
evdev input on kms etc) will have to implement this logic...

4) One of the big chunks of code left in the xkb modules deals with
writing out an xkb keymap based on a struct xkb_desc and a few changes
only to then parse it back into a new struct xkb_desc
(XkbWriteXKBKeymapForNames followed by xkb_compile_keymap_from_string
in XkbDDXLoadKeymapByNames).  If we moved this to libxkbcommon, we
could avoid the dump+parse step and just create the struct xkb_desc
directly... not sure if it's worth it.

and I'm sure there's more.  Like, exposing an API that's 90% hairy
structs is recipe for disaster, but it's kinda late to change that.
Anyway, looking forward to some feedback.

Kristian