MPX status update

[Peter Hutterer]

It has been a while so I thought I'd give a short update on MPX since  
the last email (14. December), for those that didn't go to XDC and  
don't read commit logs or IRC.

Contents of this email:
- What does it do?
- What's the current state?
- Internal device keeping
- Events
- ClientPointer
- Device pairing
- Grab issues

==========================
What does it do?

Essentially it gives each mouse its own cursor and each keyboard its  
own focus. So rather than having multiple devices moving the single  
cursor, each device is independent. Hotplugging for mice and  
keyboards works, and cursors appear as devices are hotplugged.  
Removing cursors when devices are removed isn't quite sorted out yet.

The whole infrastructure of X is built around a single cursor and a  
single keyboard focus and there's quite a number of interesting  
issues that come up. Especially with standard X applications. Of  
course any application has to work, regardless of how many devices  
are connected. A lot of the definitions on what the server does is  
specified for the single-mouse single-keyboard case only, so I have  
to make up a lot of things as I go along.

==========================
What's the current state?

MPX is part of XInput, but on its own branch (mpx). As it looks like  
now, it should be part of XInput 2.0, as a few semantics will change  
to the current XI version.
It's reasonably stable from what I can tell, one of my colleagues  
uses snapshots as his main desktop (under GNOME). Its nowhere near  
being finished, although I think a lot of the big changes are done by  
now. But little things keep coming up, and some turn out to be more  
problematic than others.

A short demo video is here http://www.youtube.com/watch?v=0MUOn_nJmRA



The following is lower level internals:

==========================
Internal device keeping

Daniel's input-hotplug added two devices to the server, the "Virtual  
Core Pointer" (VCP) and the "Virtual Core Keyboard" (VCK). Both allow  
you to start an X server without any physical devices connected. In  
MPX, both VCP and VCK have been removed from the internal device list  
and vegetate as inputInfo.pointer and inputInfo.keyboard. They are  
only used if no real device has been connected.
e.g., if xeyes is started up without any physical devices, it will  
look at the VCP. As soon as a mouse is plugged in, it will start  
looking at the new mouse.
The role of the VCK is similar, although it is not quite as defunct  
as the VCP. But it will get there, eventually.

==========================
Events and requests


Core events and a lot of core requests are deprecated. Of course they  
still work, but a new application should not use them anymore, as  
they don't provide an interface to identify the device that caused  
the event.
Each time a device generated an event, the XI event was processed as  
coming from the device and the core event was processed as coming  
from the VCP or VCK. The big internal change here is that MPX lets  
all events originate from the real device. This changes some of the  
semantics, as we can have multiple devices originating independent  
events and the core devices can send device events as well, something  
that wasn't possible before.

Each core event has (or will have) a device-specific counterpart in  
XInput, use them if you want to write an app. Keep in mind that  
devices can just appear and disappear on the fly. I'm adding requests  
to XI such as QueryDevicePointer, so each ambiguous request has a  
device-specific counterpart in XI. Use them in the future.

Internally,

==========================
ClientPointer

A lot of apps use the core events and the core protocol. Things like  
WarpPointer, QueryPointer, GetKeyboardMapping don't actually say  
which cursor to warp, query or which keyboard to get. So the server  
has to guess.

That's where the ClientPointer comes in: The window manager  
(preferably) can set a specific pointer to be a client's  
ClientPointer. Each time the client issues a request that doesn't  
have a defined state, the server will pick the ClientPointer device.  
If no ClientPointer is set, the server will pick the first physical  
mouse device it can find. If there are none, the server picks the VCP.
The ClientPointer does not affect event delivery. You can still  
interact with applications with any device.

==========================
Device pairing

With X, you have a mouse and a keyboard and they are tightly coupled.  
Some pointer events contain data from the keyboard and vice versa.
With multiple mice and keyboards you need to pair devices to create  
those tightly coupled devices. Once two devices are paired, they act  
like a traditional mouse/keyboard combination.

Each time the pointer sends an event, it will take the modifiers of  
the paired device to fill up the state. For the ClientPointer, the  
keyboard that is paired with the ClientPointer device will be used  
for undefined requests.
Device pairing can be changed by the window manager at any time.  
Mapping is one-to-many, so multiple keyboards can be paired with one  
mouse, but not the other way round.

==========================
Grab issues

Turns out the grab support was like a Jenga tower after a fair bit of  
gametime. Each piece relied on something else, and moving the bottom  
piece did not work out too well.
We could have only one pointer grab and one keyboard grab (tgrabs for  
core events), but each extension device could have a grab for  
extension events. The core devices however couldn't have extension  
grabs. Now this is reasonably stupid once you have multiple devices,  
and so I had to change parts of the grab system and the grab  
semantics too.

As a result, each device can have a core grab and a device grab, each  
grab affecting only the matching events. Core grabs are deprecated,  
don't use them. If device grabs don't do what you need, shout at me  
and I'll implement it.

The core grabs work in the following fashion: If a client has a grab,  
the device will only be sending events to the grabbing client. Device  
is picked with the ClientPointer. No other device will be sending  
core events TO THIS CLIENT, but other clients are still accessible  
with other devices. The most exciting feature you get out of that is  
simultaneous scrolling in two apps or two popups open at the same  
time. I know, very exciting. My pulse races right now too.

The disadvantage of all that is that normal window managers don't  
really cut it any more. Menus and popups will automatically belong to  
one cursor only, so if the window manager doesn't set the  
ClientPointer accordingly, the other devices are severely limited. I  
was thinking of allow events from other devices for the grab window  
only, but it isn't implemented yet.

==========================
TODO:
- Passive grabs: Don't work properly yet. High priority.
- Cursor rendering: API break to route a CursorInit and CursorRemove  
into the mi part. High priority.
- Move Generic Event Extension into mpx. High priority.
- Add missing requests to XI to match core protocol requests. High  
priority.
- sort out grab ownership issues. Medium priority.
- Enter/Leave events:   One enter on first mouse enter, one leave on  
last mouse exit for core events. Otherwise apps get confused.  Medium  
priority.
- Cursor rendering: Use screen-wide buffer to resolve flickering. Low  
priority.
- Write man pages for new Xlib calls. Very low priority.

Those are the things I definitely need to do, but there are others  
lurking in the shadows and my time to work on MPX will cut severely  
over the next months.

==========================
Summary:

I'm definitely getting somewhere, but it'll still take a while until  
all the core protocol issues (I haven't event started looking at  
other extensions yet) are sorted out and all the bugfixing is done.  
But as said, it is useable already, albeit you have to deal with some  
behavioural inconsistencies.


Comments, suggestions, beer donations etc. welcome.

Cheers,
   Peter

--
Multi-Pointer X Server
http://wearables.unisa.edu.au/mpx