[RFC] DeepColor Visual Class Extension
Alex Goins
agoins at nvidia.com
Tue Aug 15 02:17:11 UTC 2017
We brainstormed all of the suggestions amongst ourselves, and have a revised
version of the spec. It also includes some changes of our own.
Unfortunately, we weren't able to bottom out on everything (most notably the
masquerading DeepColor/TrueColor visuals), but the suggestions we haven't yet
implemented in the spec are included in the issues section, along with our
outstanding concerns. We will likely need more discussion over email or at XDC
to make sure that these suggestions are fully fleshed out.
Thanks,
Alex
--
DeepColor Extension
Version X.X
2017-XX-XX
Alex Goins
agoins at nvidia.com
NVIDIA Corporation
1. Introduction
The DeepColor Extension provides a new visual class, DeepColor, designed for
handling visuals that are incompatible with the existing core X visual classes:
StaticGray, StaticColor, TrueColor, GrayScale, PseudoColor, or DirectColor.
These visual classes provided by the core X11 protocol are insufficient for
visuals that require a greater than 32 bit depth, or non-integer formats. As
such, they are not suitable for many HDR formats.
In order to support a variety of HDR formats or any other formats that are
incompatible with the core X11 visual classes, DeepColor doesn't rely on a
colormap to define the relationship between pixel values and colors. Instead,
windows with DeepColor visuals will rely on a window property to define a
colorspace/encoding, chosen from an enumerated set of colorspaces/encodings
associated with a dedicated child of the root window. Compositors may detect
changes in the colorspace/encoding of redirected windows by listening for
PropertyNotify events from those windows.
In addition to supporting a variety of colorspaces/encodings, DeepColor visuals
must also support a variety of pixel formats, e.g. FP16 RGBA. The information
provided by the existing XVisualInfo structure is insufficient for
differentiating such formats, so a mechanism must be provided for clients to
determine the pixel formats of DeepColor visuals.
1.1. Acknowledgements
Zach Angold, NVIDIA Corporation <zangold at nvidia.com>
Adam Jackson, Red Hat, Inc. <ajax at nwnk.net>
James Jones, NVIDIA Corporation <jajones at nvidia.com>
Robert Morell, NVIDIA Corporation <rmorell at nvidia.com>
Keith Packard, Hewlett-Packard Company <keithp at keithp.com>
Aaron Plattner, NVIDIA Corporation <aplattner at nvidia.com>
Andy Ritger, NVIDIA Corporation <aritger at nvidia.com>
2. DeepColor Visual Class
The DeepColor extension defines a new visual class, DeepColor.
DeepColor visuals do not make use of the red_mask, green_mask, blue_mask, or
colormap_size fields of the XVisualInfo structure. Colormap size is defined to
be 0.
Information pertaining to the pixel format and colorspace/encoding of DeepColor
visuals and associated windows, respectively, must be queried using the
mechanisms defined below.
3. Colorspace/Encoding Window Properties
Windows with DeepColor visuals must rely on window properties, as opposed to
colormaps, to determine the relationship between pixel values and colors. These
properties must specify constants that correspond to colorspace/encoding pairs.
Possible colorspace/encoding constants are defined to be:
// Undefined:
// Undefined colorspace and encoding
#define _DEEPCOLOR_UNDEFINED 0
// scRGB Linear:
// scRGB colorspace with linear OETF
#define _DEEPCOLOR_SCRGB_LINEAR 1
// BT.2020 Linear:
// BT.2020 colorspace with linear OETF
#define _DEEPCOLOR_BT2020_LINEAR 2
// BT.2020 PQ:
// BT.2020 colorspace with SMPTE ST.2084 Perceptual Quantizer (PQ) OETF,
// also known as HDR10
#define _DEEPCOLOR_BT2020_PQ 3
// BT.2020 HLG:
// BT.2020 colorspace with ARIB STD-B67 Hybrid Log-Gamma (HLG) OETF, also
// known as HLG10
#define _DEEPCOLOR_BT2020_HLG 4
// DCI-P3 Linear:
// DCI-P3 colorspace with linear OETF
#define _DEEPCOLOR_DCI_P3_LINEAR 5
// DCI-P3 Nonlinear:
// DCI-P3 colorspace with Gamma 2.6 OETF
#define _DEEPCOLOR_DCI_P3_NONLINEAR 6
// Display-P3 Nonlinear:
// DCI-P3 colorspace with sRGB-like OETF
#define _DEEPCOLOR_DISPLAY_P3_NONLINEAR 7
// ACES2065-1:
// ACES colorspace with AP0 primaries and linear OETF
#define _DEEPCOLOR_ACES_2065_1 8
// ACEScg:
// ACES colorspace with AP1 primaries and linear OETF
#define _DEEPCOLOR_ACES_CG 9
// ACEScc:
// ACES colorspace with AP1 primaries and logarithmic OETF
#define _DEEPCOLOR_ACES_CC 10
// ACEScct:
// ACES colorspace with AP1 primaries and modified logarithmic OETF
#define _DEEPCOLOR_ACES_CCT 11
3.1. Server Window Properties
3.1.1. Root Window Properties
Servers that support the DeepColor extension MUST, when initialized, create a
child of the root window named "DEEPCOLOR_PROPERTIES".
The DeepColor extension defines a root window property that applications can use
to find the "DEEPCOLOR_PROPERTIES" window:
_DEEPCOLOR_PROPERTIES, WINDOW/32
This is the window ID of the "DEEPCOLOR_PROPERTIES" window. The server MUST set
this to the window ID of the "DEEPCOLOR_PROPERTIES" window after it has been
created, and MUST NOT change it thereafter.
3.1.2. "DEEPCOLOR_PROPERTIES" Window Properties
The DeepColor extension defines a window property associated with
"DEEPCOLOR_PROPERTIES" for determining the set of colorspaces/encodings
supported by the compositor:
_DEEPCOLOR_COMPOSITOR_COLORSPACES, colorspace, score, CARDINAL[][2]/32
This is a list of 2-tuples indicating the colorspaces/encodings supported by the
compositor (whether internal or external to the server), and their level of
preference as indicated by their score, where higher is better. The server MUST
initialize the property with a list of colorspaces/encodings supported by its
internal compositor, which may be an empty list if no DeepColor capabilities are
supported. As discussed in section 3.2, applications MUST choose a
colorspace/encoding from this set, if non-empty.
If an external application becomes responsible for compositing and supports the
DeepColor extension, it MUST override this property with its own supported
colorspaces/encodings prior to calling XCompositeRedirectSubwindows on the root
window. The server MUST delay sending the PropertyNotify event for the change in
_DEEPCOLOR_COMPOSITOR_COLORSPACES until after the root window hierarchy has been
redirected.
If XCompositeRedirectSubwindows is requested for the root window without being
preceded by a corresponding change to _DEEPCOLOR_COMPOSITOR_COLORSPACES, the
server MUST assume that the external compositor making the request is unaware of
the DeepColor extension, clearing the property to indicate the loss of
compositor DeepColor support.
Whether or not the external compositor is aware of the DeepColor extension, if
the root window hierarchy is unredirected, either explicitly via an
XCompositeUnredirectSubwindows request or implicitly when the external
compositor disconnects, the X server MUST restore
_DEEPCOLOR_COMPOSITOR_COLORSPACES to a list of colorspaces/encodings supported
by its internal compositor, if applicable. Just as when the root window
hierarchy was redirected, the server MUST delay sending the PropertyNotify event
for the change in _DEEPCOLOR_COMPOSITOR_COLORSPACES until after the root window
hierarchy has been unredirected.
The colorspace/encoding of the compositor's target surface MUST be a member of
the set supported by the server for display. In order for an external compositor
to be aware of the set of colorspaces/encodings supported by the server for
display, the DeepColor extension defines another window property associated with
"DEEPCOLOR_PROPERTIES":
_DEEPCOLOR_DISPLAY_COLORSPACES, colorspace, score, CARDINAL[][2]/32
This is a list of 2-tuples indicating the set of colorspaces/encodings supported
by the server for display, and their level of preference as indicated by their
score, where higher is better. The server MUST initialize the property with a
list of colorspaces/encodings supported for display, which may be an empty list
if no DeepColor capabilities are supported.
The set of supported colorspaces MUST NOT change after being first initialized,
and MUST be used by an external compositor to determine limits on the
colorspace/encoding of its target surface. Compositors MAY prioritize the
selection of certain colorspaces/encodings by referencing their associated
scores in _DEEPCOLOR_DISPLAY_COLORSPACES. The scores associated with the
supported colorspaces MAY change after first initialized, i.e. when the display
configuration changes.
3.2. Application Window Properties
The DeepColor extension defines a window property for determining the
colorspace/encoding being used by a window with a DeepColor visual, if
applicable:
_DEEPCOLOR_COLORSPACE, CARDINAL/32
This is the colorspace/encoding being used for rendering into a window with a
DeepColor visual. The application MUST set this to a colorspace/encoding
constant from _DEEPCOLOR_COMPOSITOR_COLORSPACES, if non-empty, and must
otherwise be undefined. Applications MAY prioritize the selection of certain
colorspaces/encodings by referencing their associated scores in
_DEEPCOLOR_COMPOSITOR_COLORSPACES, where higher is better.
The application MAY change this window property to any colorspace/encoding from
the _DEEPCOLOR_COMPOSITOR_COLORSPACES at any time. Compositors MUST listen for
PropertyNotify events to determine if and when an application changes
_DEEPCOLOR_COLORSPACE, and adjust their composition accordingly.
Applications MUST listen for PropertyNotify events originating from
"DEEPCOLOR_PROPERTIES" to determine if _DEEPCOLOR_COMPOSITOR_COLORSPACES
changes, and adjust their rendering and window properties accordingly. This
could occur due to an external compositor being launched or killed, or due to a
modeset.
If at any point the colorspace/encoding reflected by _DEEPCOLOR_COLORSPACE
associated with a window is not a member of _DEEPCOLOR_COMPOSITOR_COLORSPACES,
the behavior of the compositor in attempting to handle the composition of this
window is undefined.
4. Requests
Please refer to the X11 Protocol Encoding document as this section uses
syntactic conventions and data types established there.
DPCGetVersion
client_major_version: CARD8
client_minor_version: CARD8
=>
server_major_version: CARD8
server_minor_version: CARD8
If supplied, client_major_version and client_minor_version numbers indicate what
version of the protocol the client wants the server to implement. The
server_major_version and the server_minor_version numbers returned indicate the
protocol this extension actually supports. This might not equal the version sent
by the client. An implementation can (but need not) support more than one
version simultaneously. The server_major_version and server_minor_version allow
the creation of future revisions of the DeepColor extension protocol that may be
necessary. In general, the major version would increment for incompatible
changes, and the minor version would increment for small, upward-compatible
changes. Servers that support the protocol defined in this document will return
a server_major_version of one (1), and a server_minor_version of zero (0).
DPCGetVisualInfo
visual_list: LISTofVISUALID
=>
per_visual_info: LISTofVISUALINFO
where:
VISUALINFO: [ core_visual_id: VISUALID
pixel_format: CARD32 ]
- pixel_format is an enumeration of pixel formats:
FP_R16G16B16A16 = 0,
UINT_R16G16B16A16,
UINT_A2R10B10G10,
UINT_A2B10G10R10,
5. Events and Errors
No new events or errors are defined by this extension. Events for
colorspace/encoding changes are handled by the existing PropertyNotify event,
and pixel format cannot change without destroying and re-creating the drawable
with a new visual.
6. Encoding
The name of this extension is "DEEP-COLOR".
This document uses the same conventions as those used for the core X11 Protocol
Encoding.
DPCGetVersion
1 CARD8 opcode
1 0 DPC opcode
2 2 request length
2 CARD16 client_major_version
2 CARD16 client_minor_version
=>
1 1 reply
1 unused
2 CARD16 sequence number
4 0 length
2 CARD16 server_major_version
2 CARD16 server_minor_version
20 unused
DPCGetVisualInfo
1 CARD8 opcode
1 1 DPC opcode
2 unused
4 n number of VISUALIDs in list
4n CARD32 VISUALIDs
=>
1 1 reply
1 unused
2 CARD16 sequence number
4 m number of VISUALINFOs in list
24 unused
8m LISTofVISUALINFO items
VISUALINFO
4 VisualID core_visual_id
4 CARD32 pixel_format
7. Issues
This spec does not address how HDR content should be downsampled to SDR content
for e.g. automatic redirection, XGetImage(), or core X11 / RENDER rendering.
Furthermore, clients that are not aware of the new visual class may trip over it
and crash.
* The current standing suggestion for addressing this problem involves having
DeepColor visuals exposed in the core protocol as TrueColor visuals, with
anything pertaining to DeepColor queryable only through DPCGetVisualInfo,
including the "true" visual class. Normal TrueColor visuals would reflect
TrueColor in the extended visual info, but DeepColor visuals would reflect
DeepColor, along with pixel format. Core protocol requests using drawables
with this visual would treat them as TrueColor, with the server using its
knowledge of the true format to convert where necessary, with operations
restricted to GXCopy, planemask ~0.
One standing issue with this idea is that from the core protocol's
perspective, these masquerading DeepColor/TrueColor visuals would appear to
be identical to the "true" TrueColor visuals, as well as other masquerading
DeepColor/TrueColor visuals that have different pixel formats. While many
clients may traverse the list of visuals linearly, picking the first match
from the set of visuals obtained through the connection block, this is not a
given. If a naive client wrongly picks a masquerading visual, we incur undue
overhead doing unnecessary conversions, and maybe even corruption if the
conversion if imperfect or lossy.
It was suggested that if a compositor is started that does not support a
colorspace/encoding that is currently being used by a drawable, and the
application fails to adapt to a new colorspace/encoding, the drawable can be
treated as if it was TrueColor like with core/RENDER, and the server can
implicitly downsample to SDR on a best-effort basis using its more
comprehensive knowledge of colorspaces/encodings. However, if a compositor
chose to disable automatic shadow updates in order to do presentation in a
more unique fashion, then handed that responsibility to the server, the
server would have no context about where it should present those contents
within the parent window.
An alternative method to handling clients tripping over the new visual class
would be to provide an opt-in mechanism to cause the server to hide these
visuals from the connection block for older clients, via e.g. a bump to the
X11 core protocol minor version. This would prevent clients from tripping
over DeepColor visuals when creating a new window, but would not prevent
them from getting confused when they see other windows using VisualIDs that
do not correspond to visuals in its connection block. Perhaps these visuals
could masquerade as TrueColor to unsupported clients, but they still
wouldn't correspond to a visual in the connection block unless they were
shown with VisualIDs that correspond to those of the actual TrueColor
visuals.
This spec does not address the suggestion that _DEEPCOLOR_COLORSPACE should
reflect the colorspace being used by next frame. It is difficult to determine
what the "next frame" is without the Present extension, and a concrete solution
has yet to be found.
* Perhaps this functionality could be the domain of an interaction between
the Present extension and DeepColor-aware clients, where clients hand off
the responsibility for changing _DEEPCOLOR_COLORSPACE to the Present
extension, which would atomically update it with the presentation of the
next frame.
There is a suggestion to use PropertyNotify event filters instead of a
server-created child of the root window to allow for clients to narrow the scope
of PropertyNotify events to listen for, but this functionality has yet to be
concretely defined. There is existing precedent for window managers to create
children of the root window to store properties for this purpose, and this isn't
much different, so perhaps the server-created child of the root window is an
acceptable solution.
Due to a suggestion to move to constants instead of atoms for defining supported
colorspaces, and the resulting requirement that additional colorspaces are added
through revisions to the DeepColor spec, the set of colorspaces defined in the
first version of the spec has been expanded to cover every colorspace that is
suspected to be in demand in the immediate future. The BT.2020 HLG and ACES
colorspaces are currently not supported by the requisite EGL/Vulkan extensions;
that support will have to be added before drivers/the server can support them.
More information about the xorg-devel
mailing list