[RFC] Visual Class for On-Screen HDR Drawables

[Alex Goins]

Hello all,

For those who aren't aware, Andy Ritger gave a talk at XDC 2016 about what the
advent of HDR displays means for the Linux ecosystem:
https://www.x.org/wiki/Events/XDC2016/Program/ritger_hdr/. To make a long story
short, displays capable of displaying HDR content mean that we need to push HDR
content through the whole pipeline, rather than tonemapping to SDR before it
makes it on-screen.

One of the issues that Andy brought up in his talk pertains to how we would go
about compositing HDR content with other SDR content and the rest of the
desktop. For this, we intend to use the scRGB color space, a linear FP16
encoding where the range [0.0, 1.0] corresponds to the traditional sRGB colors;
the sRGB range is contained within [0.0, 1.0], but doesn't match exact values
without conversion due to being non-linear. Values above or below [0.0, 1.0]
extend the range of colors. These properties make it good both for representing
HDR content, and for compositing HDR content with SDR content.

However, in order for scRGB visuals to coexist with the X desktop and to create
HDR capable windows, we need a way to represent them with X visuals so they can
be used to create GLX/X windows and pixmaps, rather than relying on pbuffers and
FBOs. Existing X visual classes can only represent up to 32-bits per pixel. They
cannot represent the 48-bits per pixel (FP16 per component) necessary for scRGB.

To allow for scRGB visuals, as well as others with arbitrarily large color
components, we'd like to propose a new visual class that is opaque to X
rendering, but allows for drawables with arbitrary attributes to be used for
on-screen rendering and compositing via GLX. I'll call it VoidColor, for now.

VoidColor is a visual class defined to have a static colormap of 0 elements.  It
is incapable of receiving X rendering, but is compatible with any GLX drawable
of the correct depth regardless of other attributes. The idea is that you could
pass any GLX fbconfig to glXCreateWindow as long as the corresponding Window has
a VoidColor visual with a matching depth.

Such a design would likely prevent X Render compositing, but should allow for
OpenGL compositors to handle HDR drawables as long as they are designed
accordingly. One concern we have is the texture_from_pixmap extension. The
current pattern compositors such as Mutter and Compton seem to use is to look up
an X drawable, create an intermediate GLX drawable, and use that for
BindTexImage. If we don't store distinguishing attributes in the Visual,
compositors would have trouble creating a correct, matching GLX drawable.
Perhaps the compositor could wait for the application to create a GLX drawable
before querying it, but there would have to be some mechanism to know when it's
ready. The alternative would be to store distinguishing attributes in the
Visual, but that would mean limiting it to fbconfigs that can be differentiated
by those attributes, and would necessitate the creation of many visuals to match
the fbconfigs, which we are trying to avoid.

Adding a new visual class would have a lot of implications, particularly for
window managers and compositors that may have to handle windows and pixmaps with
the new visual class despite not having created them themselves. I've prototyped
creation of X drawables with VoidColor visuals locally, and with some changes to
Xorg/libX11/NVIDIA X and GLX, a handful of common window managers seem to handle
it without choking. Compositors, of course, will require more work to become HDR
aware and successfully composite HDR drawables, but they at least seem to handle
it without crashing. However, even so, we would likely want to lock the creation
of VoidColor visuals behind a flag while the rest of the ecosystem catches up.

There is much work to be done, but in the interest of discussion, we'd be
interested in hearing ideas and criticism from the community. What do you think?

Thanks,
Alex Goins
NVIDIA Linux Driver Team