Xkb extension breaks KeyPress handling in Xt with multiple keyboard layouts (groups)

[jumbophut]

I have struck a problem with Xt and Xkb when using multiple keyboard
layouts.  I am hoping someone on this list may have a neat solution or
be able to lend some weight to tweak libXt.

I use "None <Key> a : func() \\n" in an XtTranslations table, which
should call a routine when 'a' is pressed on the keyboard with no
modifier keys pressed.  However, it does not get called.

The reason for this is that Xt wants to see an event->state field
equal to nil to match the 'None' modifier, before it passes on the
KeyPress.  Unfortunately, I have two keyboard layouts available and
XKB sets a modifier bit (0x2000) in the event->state field when I am
using the second (this is the one I usually use).  When this happens,
event->state will obviously never be nil.  Xt does not know about this
bit -- it only knows about bits 1 through 12 and 15.

There are workarounds but they are not ideal:

1. If I use "~Ctrl ~Shift ~Lock ~Mod1 ~Mod2 ~Mod3 ~Mod4 ~Mod5 <Key> a"
I get what I want but my translation table becomes hideous and
possibly inefficient.

2. Setting XkbIgnoreExtension(True) also fixes the problem, but it
means dropping XKB support entirely, with consequences unknown.

3. I can go back to the other keyboard layout (bog-basic US layout)
but then I can't get characters I need.

I might have found the relevant parts of the source:

ParseModifiers in TMParse.c sets event.modifierMask equal to ~0 and
event.modifiers equal to 0 when the modifier is None.

I think this means that the check in _XtRegularMatch in TMstate.c will
probably not work, since it will be seeing an event for which
event.modifiers is never nil (will always be at least 0x2000).

A solution would be to set the bits in event.modifierMask to 1 only if
Xt knows about them.  Other bits (13 and 14, used for Xkb) would be
set to 0 in TMParse.c (that is "don't care" about these bits).  The
modifier "None" then might become slightly misleading, but will do
exactly what was intended when Xt was written and make "None" useful
again.

This may not be the end of the story if there are similar problems
with XGrabKey in libX11.  However, I have not investigated this
further, having got stuck at GrKey.c in the source.

Maybe this is academic, given that (1) many people will not be using
Xt any more and (2) people will not use the None modifier anyway
because NumLock and CapsLock can mess with it.

However, I have spent a long time working out what the difficulty is
and it seems useful to at least document the problem in a place where
people might find it after me.

Also, it's a problem in at least one widely deployed application.  You
can see the problem in action using xcalc, which can only receive
number input from the numeric keypad (top row on the keyboard does not
work) when switched to a second keyboard layout.

-- 
Tony (echo 'spend!,pocket awide' | sed 'y/acdeikospntw!, /l at omcgtjuba.phi/')