Ideas for X improvement.

[Glynn Clements]

David Jackson wrote:

> A display driver that contains a VNC server. The problem with x11vnc is that
> it is slow, very slow. XVnc server, which is a X server that contains a VNC
> server but has no hardware drivers, is much faster since the VNC server is
> built directly into the X server,

What sample size does your analysis use? How many different hardware
configurations, and how many different applications?

x11vnc has the drawback that it's reading a framebuffer which is
typically in video memory, so the data has to be read over the bus. 
The speed of this operation may vary significantly depending upon the
hardware being used. It may also vary depending upon the amount of
activity (i.e. if it has to wait for the outstanding rendering
operations to complete before it's allowed to read the framebuffer).

Xvnc has the advantage that the framebuffer is in system memory. But
this is also a drawback, as it means that all rendering is performed
in software. Try running an application which uses OpenGL to render
detailed scenes; you might want to reconsider your assertion about
Xvnc is fast.

IOW, it's a case of "choose your poison". x11vnc has fast rendering
but slow export, Xvnc has slow rendering but fast export. A similar
tradeoff exists for X11 verus VNC for remote display.

> however this does not allow one to export
> their main X display which is also displayed directly to video hardware. The
> solution here is to include a driver in the X.org main server distribution
> for a VNC server that can be loaded into the X server. The VNC server driver
> should be able to be dynamically loaded while the server is running and the
> output of the server displayed simultaneously to VNC clients and to the
> local video hardware. This can be controlled from provided command line and
> GUI utilities.

Does the VNC driver read the framebuffer on the video card (which
suffers from the same performance issues as x11vnc), or does it
attempt to duplicate the framebuffer by emulating whatever video
hardware is installed? If it's the latter, the application will be
slowed to the speed of the VNC driver's software renderer (which will
be extremely complex, as it will have to mimic every feature which is
available in at least one hardware driver).

> One of the very severe problems I have been having is that Xvnc does not
> support Render extensions, and many applications no longer work without the
> Render extension. VNC driver with X.org therefore must support the Render
> extension and other ones.

The main "other one" being OpenGL, for which a software implementation
will be much, much slower than a modern GPU.

> Dynamic runtime enabling and disabling, configuration and setup and removal
> of display output and input drivers while the server runs without server
> restart. this allows for instance, the user to have the X server display to
> a new target while the server runs, or display to many different display
> outputs at the same time This includes the VNC Server driver above, this
> allows a person to easily swtich the VNC on and off from displaying to
> certain outputs, such as they could turn off display to the local monitor
> and then turn it back on again, or turn on and off VNC display.
> 
> Another feature that increases flexibility to the user would be to allow the
> user to direct display of a certain window or the entire root window and
> display over an X client connection to another server, or any number of
> other servers. This would also forward the windows children who would also
> be displayed on the remote server inside the parent window.

To do this at the protocol level requires a completely new protocol
and significant support from the toolkit. The X protocol exposes a
significant amount of implementation detail to the client. Much of
that information is required to remain constant for the lifetime of
the client.

E.g. if the client queries the list of OpenGL extensions, and starts
using some of them, there's no mechanism by which to inform the client
that an extension is suddenly unavailable, which would be required if
you were to "redirect" the window to a different server with different
hardware.

Even if such a mechanism existed, it's debatable how many applications
would support it. Reconstructing the current server-side state from
scratch is a lot of work, and toolkits can't always help (e.g. they
won't help reconstruct the server-side OpenGL state, as the toolkit
doesn't get involved in the rendering process).

> Many users, including myself, want to have many X servers running at the
> same time and then at run time be able to change to where these servers are
> being displayed, and as well when an app is started, to which server it is
> displayed with the -display option.

AFAICT, there are only two feasible approaches to window "mobility":

1. VNC-like framebuffer sharing. The application connects to a
specific X server which performs all rendering. You have the option to
forward rendered images to other systems for physical display.

2. Use GUI toolkits which offer an abstract, high-level interface to
the client. The toolkit has the ability reconstruct and clone windows
at will.

-- 
Glynn Clements <glynn at gclements.plus.com>