Some ideas

[Alan Cox]

> This already exists, it's just not bundled with X.Org due to the different
> license - TigerVNC uses current Xorg sources to build both Xvnc and a
> vnc.so loadable extension module that are compatible with current servers
> and extensions.   http://www.tigervnc.com/

(Providing you don't have any non GPL compatible drivers loaded - eg some
vendors binary only ones...)

You don't actually need so urgently with the damage/compositing
extensions because you've got a pretty good idea what needs adjusting.
X11vnc is quite passable without being part of the server.

> > 2) Low priority: Dynamic runtime loadable and unloadable drivers.
With
> > the possibility of hot pluggable devices, has this been considered? 
> 
> Already done for input devices.   Graphics is harder, but do you really
> have systems where you're often hotplugging PCI-E cards while the system
> is running?   (Yes, I know about USB-connected DisplayLink devices, but
> most work around hotplugging those has been configuring them as separate
> X seats/sessions, not incorporating them into a running server.)

Unfortunately - although to use them properly is tricky anyway as the
expectation is that the main video card composites the image in off
screen ram and you then fire at at the USB, possibly using nutty 3D card
hacks on the way to work out which chunks changed. That means you are
trying to allocate resources on one device for rendering for another.
Doubly fun if you hotplug the device doing the rendering work.

Some of this also needs kernel work - being able to give up a device and
claim a device reliably when it is being passed between users requires a
couple of bits the Linux kernel currently lacks except for tty devices.
This is much of the same stuff you need to run X as the user not as its
own user or root.

Alan