RandR 1.4 restart

[James Jones]

On Tuesday 01 March 2011 11:12:14 Soeren Sandmann wrote:
> Ville Syrjälä <ville.syrjala at nokia.com> writes:
> > On Mon, Feb 28, 2011 at 05:29:56PM -0800, ext Keith Packard wrote:
> >> Ok, let's try to get a bit of protocol review, then look at the library
> >> API, then the server internal APIs and finally the server
> >> implementation. Most of the code already exists, so this shouldn't take
> >> a terribly long time.
> >> 
> >> I've cleaned up the protocol a bit, reducing the changes so that we've
> >> got only one 'set' and one 'get' request, each of which do 'everything'
> >> except deal with output properties. The separate requests to set and
> >> query the sprite transforms are gone.
> >> 
> >> For the library changes, I'd like to know is whether I should emulate
> >> the new SetCrtcConfigs interface using old requests on old servers. That
> >> would involve duplicating a bit of structure from the insides of the X
> >> server, but would mean that applications could switch to the new
> >> interface and not worry about preserving compatibility with the old.
> >> 
> >> Here's a diff from 1.3.2 of just the protocol specification. Please see
> >> if this looks reasonable from a protocol perspective.
> > 
> > So what about also allowing windows to use the scanout pixmaps as their
> > backing pixmap? I'm mainly interested in something that would allow
> > unredirected 32bpp windows on a 16bpp screen.
> 
> Presumably the main use case for this is fullscreen video. IMO, we need
> to move to a model where
> 
>   - The backing store for a window is managed by the compositing
>     manager, and not by the X server.
> 
>   - Applications send update requests directly to the compositing
>     manager (through ClientMessages or the like) in the form of Picture
>     IDs plus coordinates for a position where those pictures are to be
>     copied.
> 
>   - Applications either tell the compositing manager what format the
>     window pixmap should be, or they let the compositing make that
>     decision itself based on the root format and/or the formats of the
>     updates.
> 
> In this model, toplevel windows would be InputOnly and not InputOutput
> as they are now.

Interesting.  I've been talking up almost this exact idea for a year and a 
half now and never gotten much feedback on it, so I'm surprised to hear you've 
got very similar ideas.

More comments below, and for reference, yes I'm referring to the ideas Aaron 
presented at XDC 2009 again, available here:

  http://people.freedesktop.org/~aplattner/x-presentation-and-
synchronization.pdf

> Advantages include:
>
>   - Completely tear-free updates because the compositing manager can
>     delay updating the window pixmaps until the application signals that
>     it has finished drawing a frame. (We may need client-visible
>     refcounts on pictures for this to work).
> 
>   - With the RandR 1.4 SetCRTCPixmap, almost all the current copying
>     goes away. For reference, here are the copies that may currently
>     take place:
> 
>       - Application draws into offscreen pixmap
> 
>       - Application draws offscreen pixmap into redirected window
> 
>       - Redirected window is copied onto redirected window frame (if the
>         frame has a different depth than the window)
> 
>       - Redirected window frame is copied onto offscreen pixmap by
>         compositing manager
> 
>       - Offscreen pixmap is copied into shadow framebuffer by X server
> 
>       - Shadow framebuffer is copied into real framebuffer
> 
>     In the new model, the application draws into offscreen pixmaps,
>     which are copied into backing pixmaps, which are copied directly
>     into the (eventual) framebuffer. Enterprising compositing managers
>     could even try to reuse the framebuffer as backing store for the
>     parts of windows that are visible.

Our proposal shares all these advantages, plus has the following:

-It defines a more robust presentation framework for X.

-Doesn't require a composite manager.  It allows falling back to server-side 
rendering without losing many of the presentation benefits (such as tear-free 
updates).

-Puts full control of backing store allocation and management in application's 
hands, rather than X or the composite manager.  Legacy applications would 
behave as they do today, but applications that chose to take advantage of the 
new extensions could specify how many offscreen backing pixmaps they wanted, 
whether they want tear-free updates, timed updates (for smooth animation 
control), etc.
 
-Integrates well with GLX/EGL.

> For full-screen video in particular, if Render were to gain support for
> YUV formats, the advantages are significant:
> 
>   - The video player could submit YUV pictures directly to the
>     compositing manager, which could then decide how to best scale it
>     and convert to RGB. An obvious option would be scanning them out
>     directly if the window is fullscreen and the hardware supports it.
> 
>   - The backing pixmap could be updated with just the blocks that
>     actually changed since the last frame (as provided by the video
>     codec).

That second optimization is an interesting idea.  I'd be curious how much work 
it would be to implement, and I'm wondering whether we can get that sort of 
information out of our hardware decoders...

> The net result would be a huge reduction in memory bandwidth for video
> playback.

And the first reasonable non-tearing video presentation solution X has had 
since composite came around.

> With a scheme like this, the Composite extension itself would become
> simply a way to support legacy applications since window redirection
> doesn't really mean anything for InputOnly windows.

I would love to see the composite extension extended rather than replaced.  I 
think the API changes needed to implement the basics of this would be pretty 
minimal.

FWIW, NVIDIA also proposed some modifications that would make this 
rendering/presentation model fit in better with randr 1.4 when Keith first sent 
out the proposed randr updates for comment a year ago or so.  He shot them 
down as trying to drag multi-buffering into randr.  I never felt that was the 
case, but didn't have time to pursue it further then.  I can dig up the old 
thread if anyone's interested.

Thanks,
-James

> 
> Soren
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: http://lists.x.org/mailman/listinfo/xorg-devel