[RFC] Visual Class for On-Screen HDR Drawables

[Alex Goins]

Thanks Adam. Inline -

On Fri, 10 Feb 2017, Adam Jackson wrote:

> On Thu, 2017-02-09 at 19:06 -0800, Alex Goins wrote:
> > Any input?
> 
> Inline below...
> 
> > > Would it be a major problem for automatic redirection to destroy HDR?
> > > Compositors could disable automatic redirection for DeepColor windows, and
> > > redirect themselves with preserved HDR.
> 
> It's not a problem in the sense that the actual HDR window content
> would still exist somewhere; we just need to be able to do _something_
> when an HDR window ends up in Automatic redirection. We could skip
> painting it to the parent, we could clamp to some range...

Right, as long as it's acceptable for it to be done on a best-effort basis I
think that makes sense, and if we can get away with skipping painting it
altogether in the worst case that gives us some flexibility to add new formats
later without breaking compatibility with older versions.

> > > One of the reasons why we initially went with the idea of a visual opaque to
> > > core/Render operations is that there are other HDR encodings that have different
> > > transfer functions from the HDR format to sRGB. It sounds like the current train
> > > of thought with clamping to the sRGB range assumes that we are operating only on
> > > the linear FP16 encoding of scRGB, which is highly limiting. scRGB is useful if
> > > we are trying to composite HDR and SDR content together, but there are other
> > > formats as well. Even the scRGB color space has support for a 12-bit nonlinear
> > > encoding in addition to the 16-bit linear one. Android and Windows both have
> > > support for multiple HDR formats.
> 
> Sure. Picking sRGB here is just a plausible default. What we're
> trying to cover by exposing HDR content through existing visual classes
> and imaging ops is just "do something reasonable for compatibility with
> non-HDR-aware apps". If there's some other transformation to unorm8
> that would produce better visual results along those paths then great,
> we just need to figure out how to expose that to a) application control
> b) compositor visibility. There are options there, it could vary by
> visual (although I think that's not how GLX visuals want to work), it
> could be some new bit of per-window state that we add to Fixes or XC-
> MISC or a new extension, it could be a new NETWM property...

I don't have much of an opinion on whether we should define a whole new
extension or extend an existing one, although from reading the description of
the XFIXES extension it doesn't seem like the right scope, in that we aren't
trying to eliminate a problem caused by a workaround. I hadn't considered NETWM,
that could make sense to put pressure on window managers to composite HDR
content correctly rather than relying on automatic redirection, even if most of
the info is queryable through an extension.

More on this topic below.

> > > One concern that came up internally is handling pixmaps, which specify a depth
> > > but not a Visual. I looked a bit into Mutter and Compton, and they seem to deal
> > > with this by querying visual properties from the root window, and assuming there
> > > aren't any pixmap-specific parameters that would affect the texture target.
> > > However, since the Pixmap is queried from the Window, they should be able to get
> > > the parameters from the Visual of the Window backing the Pixmap.
> 
> I feel comfortable saying these compositors are wrong and should be
> using glXQueryDrawable to figure out the fbconfig for the window-
> pixmap; though, having said that, I'm not sure Xorg's glx would return
> something sensible.
> 
> But again, I think the colorspace is more properly a property of the
> drawable than of the fbconfig. The EGL HDR specs seem to be written in
> terms of per-drawable state rather than new configs. So we might need
> to add a new attribute for QueryDrawable; that's fine, the set of
> people involved in new GLX extension specs are all already on this
> list, if not this thread.

FYI - it's only the texture_from_pixmap texture target attributes that they base
off of the root window, e.g. GLX_TEXTURE_2D_EXT, GLX_TEXTURE_RECTANGLE_EXT,
GLX_TEXTURE_FORMAT_RBGA_EXT. They get the fbconfig from the X pixmap's depth,
along with a few other static attributes, which is still a problem for us since
with HDR we'll need more info than just depth. The texture target derivation is
still wrong behavior, however, as it should be done using FBConfig attributes.
If they want to avoid looking up FBconfig attributes every time a window is
mapped, a better implementation suggested internally would be to cache FBconfig
capabilities in a hash map or something based on the visual ID, due to the
variability of texture format attributes between different visuals/fbconfigs.

Thanks, after going back over the EGL HDR specs I see what you're getting at.
With GLX we don't really address colorspace with visuals or fbconfigs -- they
just address the storage. EGL, being the first to address colorspace explicitly,
sets the precedent that colorspace is a drawable property. That way, we don't
need a different fbconfig for every interpretation of FP16 component storage. I
agree with that direction, especially considering EGL's precedent.

The glXQueryDrawable() method of finding the fbconfig could be a problem with
our driver, and possibly others per the spec. glXCreateWindow() doesn't
guarantee that the GLX window and X window are the same, despite being
associated, e.g.:

    glXWin = glXCreateWindow(xWin);
    assert(glXWin == xWin);
    glXQueryDrawable(xWin);

will work in Mesa, but with our driver the assert will fail, and
glXQueryDrawable() should too; it certainly can per the spec.

If glXWin != xWin, the composite manager can't rely on glXQueryDrawable(xWin)
when it discovers a redirected window xWin. So, if we're going to use
glXQueryDrawable(), we'd want a more reliable way to ascertain the GLX window
XID from the native X window.

> > > The only problematic case would be an app that has a Pixmap of unknown
> > > origin/format and wants to texture from it. For GLX, we might have difficulties
> > > finding a correct fbconfig to create a GLXPixmap to use with glXBindTexImage.
> 
> If this is drawable state not config state, then this difficulty
> disappears, right? You look it up from the window whose pixmap you've
> bound. Though you do need to get notified if it changes, so we may also
> need a new event.

Beyond the above caveat, yeah it should solve the problem for GLX. However, in
EGL, you can texture directly from a native pixmap without creating an EGL
surface for it, so we'd need a way to find the pixmap format data at EGLImage
creation time. There's been discussion about passing it via eglCreateImage()
attributes.

If we make format data a property of the native X window, compositors should
have no issues querying it from the redirected window, and we could piggyback
off of it to do the transformation to unorm8 based on the specified format if we
have a known transformation for it.

If it's also a property of native X pixmaps such that composite window-pixmaps
share the attributes of their corresponding redirected windows, it would be
ideal if drivers had a way of intercepting that data before any application can
attempt to bind those pixmaps as textures.  That way, EGL binding a native X
pixmap would work like binding any EGL image target, in that format information
is implied by the target.

> - ajax
> 

It makes sense for colorspace info to be stored as a property of the drawable,
i.e. scRGB. EXT_gl_colorspace_bt2020 also sets precedent for storing encoding
along with the colorspace, e.g. BT2020 linear, BT2020 pq.

Encoding can impact storage format. With scRGB, there are two encodings: 16 bpc
linear, and 12 bpc nonlinear. To confirm, we're not planning on exposing any of
that information via X visuals, only using X visuals to interpret the resulting
pixel data after converting to unorm8, correct?

Automatic redirection would have to imply the properties of the underlying
storage from the colorspace/encoding, which may not be ideal. Maybe we could
expose extended X visual properties as well, for reading the raw HDR pixel data
before transforming it based on the colorspace/encoding information garnered
from the drawable properties.

Thanks,
Alex