[PATCH RFC] On console keyboard handling

[Arthur Taylor]

I have mentioned this on this list before, so apologies if it seems like
spamming.

The development 2.6.39 linux kernel has a new console keyboard mode OFF
which disables all kernel key buffering and special key handling in the
console. If used on new kernels it simplifies X's job and removes to
need to handle redundant key events (reducing possible race conditions)
and is just plain more logical.

Looking at the log for xserver/hw/xfree86/os-support/linux/lnx_init.c
there have been a few bugs related to the drain_console() console
handler. By switching the console to OFF rather than RAW, such a handler
is not needed. Further, switching the console to OFF mode works with
input-keyboard, as console init is called before input init, and
input-keyboard will switch the console to RAW mode on it's own. As there
is no handler installed, there is no chance of a race between
drain_console() and input-keyboard or the vt getting a vhangup. Further,
as kbd init always sets the console to RAW mode, lnx_init can always set
the console to OFF mode without knowing what is going on with the input
configuration.

OFF mode is implemented in the kernel to ignore special keys, as like in
RAW mode, and disable key event buffering. As this is a per vt mode, no
work is needed on X's part when doing a vt switch, the kernel maintains
it's own keyboard state throughout.

The problem with OFF mode is that it requires new kernel headers at
compile time and a new kernel to be running. As of this message's
writing kernel 2.6.39 has not been released. However, OFF mode's
definition can be checked for at compile time with a simple #ifdef
K_OFF. If X is compiled with K_OFF but the kernel doesn't support it,
the vt ioctl KDSKBMODE will return EINVAL, which can caught and the
current method of using K_RAW and drain_console() can be used instead.

I've attached a prospective patch (awaiting 2.6.39's release of course)
in the interest of generating feedback.

-- 
Arthur Taylor
art at ified.ca
theycallhimart at gmail.com


---
 hw/xfree86/os-support/linux/lnx_init.c |   22 ++++++++++++++++------
 1 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/hw/xfree86/os-support/linux/lnx_init.c b/hw/xfree86/os-support/linux/lnx_init.c
index 9c71a42..8e0ad56 100644
--- a/hw/xfree86/os-support/linux/lnx_init.c
+++ b/hw/xfree86/os-support/linux/lnx_init.c
@@ -207,9 +207,22 @@ xf86OpenConsole(void)
             tcgetattr(xf86Info.consoleFd, &tty_attr);
             ioctl(xf86Info.consoleFd, KDGKBMODE, &tty_mode);
 
-            if (ioctl(xf86Info.consoleFd, KDSKBMODE, K_RAW) < 0)
-                FatalError("xf86OpenConsole: KDSKBMODE K_RAW failed %s\n",
-                        strerror(errno));
+#ifdef K_OFF
+            /* Disable the kernel virtual console keyboard handling. Available
+             * since kernel 2.6.39 */
+            if (ioctl(xf86Info.consoleFd, KDSKBMODE, K_OFF) < 0)
+#endif
+            {
+                /* Otherwise, set the console to raw mode to disable kernel
+                 * special key handling... */
+                if (ioctl(xf86Info.consoleFd, KDSKBMODE, K_RAW) < 0) {
+                    FatalError("xf86OpenConsole: KDSKBMODE K_RAW failed %s\n",
+                            strerror(errno));
+                } else {
+                    /* ...and keep the console keyboard buffer empty. */
+                    xf86SetConsoleHandler(drain_console, NULL);
+                }
+            }
 
             nTty = tty_attr;
             nTty.c_iflag = (IGNPAR | IGNBRK) & (~PARMRK) & (~ISTRIP);
@@ -222,9 +235,6 @@ xf86OpenConsole(void)
             cfsetospeed(&nTty, 9600);
             tcsetattr(xf86Info.consoleFd, TCSANOW, &nTty);
 
-            /* need to keep the buffer clean, else the kernel gets angry */
-	    xf86SetConsoleHandler(drain_console, NULL);
-
 	    /* we really should have a InitOSInputDevices() function instead
 	     * of Init?$#*&Device(). So I just place it here */
         }
-- 
1.7.5.rc1