Beginner Questions About XCB - xcb_alloc_color is too slow
Carsten Haitzler
raster at rasterman.com
Tue Feb 23 10:46:32 UTC 2021
On Mon, 22 Feb 2021 23:30:27 -0600 Chris Sorenson <csoren at isd.net> said:
> Around 25 years ago, before the X Render Extension, Eric Foster-Johnson
> wrote a great article on the topic of using color in X, for the "Cross
> Thoughts" column in Unix Review Magazine. I've kept a copy ever since:
>
> When the X Window system became popular, particularly on UNIX
> workstations, you were lucky if you had a display that supported 256
> colors. At the time, most X programmers used the simplest color
> functions because most workstations didn't provide more than the
> simplest color displays.
>
> Cross Thoughts
>
> In recent years that's all changed and we now expect even low-end
> hardware to display thousands of colors. Even so, most X programs
> still stick to old habits and don't take advantage of the greater
> color capabilities that X provides.
Ahhh. This dates the article. This is when PCs started to become common to run
Linux and other x86 unix ports and PCs had then gotten video cards capable of
15/16bit RGB (555/565). It was very common then to find and took your color
quality leaps and bounds above the previous top of 256.
> This month we'll go over color programming, starting from setting up
> colors in a colormap to using more-complicated X visuals.
>
> X builds its color model on RGB components, the red, green, and blue
> values used to define color on a typical monitor. Using the Xcms
> routines, short for X color management system, you can define colors
> in terms of CIE (a color model defined in 1931 that represents all
> possible colors in a three-dimensional color space)1 or HSV (hue
> saturation value) definitions, but the Xcms routines translate either
> definition to the RGB values the X server requires.
>
> In X, the RGB values go from 0 (all off) to 65535 (all on). Many
> color definitions that you'll find, though, go from 0 to 255. It's
> not that hard to convert with a macro such as the following:
>
> #define Conv256To64K(v) \
> ((65535 * (long) v)/256)
^^^ bug. Classic mistake getting colors wrong with "off-by-1's" where pure
white can become just a bit less than pure white for example. Even the name is
wrong as 0->255 colors do NOT go up to 256. You want:
#define To65k(v) (((long)v << 8) | v)
Of course this only applies to allocating colors or looking them up. Once you
deal with "truecolor displays" (15/16/24 etc. bit) you should change how you do
color entirely and now you are going to want to think/work in RGB and then
scale them to the bit masks for RGB for your visual. e.g. if yuou start with
RGB being 0->255. a 16bit RGB565 pixel will be:
unsigned short pixel =
(((unsigned short)(r & 0xf8)) << 8) |
(((unsigned short)(g & 0xfc)) << 2) |
(((unsigned short)(b )) >> 3);
You don't need to allocate color cells or deal with 0->65535 RGB values handed
to X -> you just need to use pixel values that map to the RGB masks of the
visual you are using (and in almost all cases these days that's just the
default visual... the only case you need a special visual these days is the
Xrender extended RGBA visual for a window that indicates the window has an
alpha channel and compositors will treat it as such, or with opengl where you
have to create windows with specific visuals to have opengl render).
So having skipped ahead down below, the rest is all sane, but it's a world that
no longer exists. Reality is all we have now are truecolor displays. Almost all
do at least 24bit color (8 bits per RGB). In the early smartphone days we
still had 16/15bit displays but that ended in about 2010-2012 or so as even
the humble phone moved to 32bpp. Many displays can now do 30 bit (10 bits per
RGB) but this is not the default. Most uses don't need this (and this tends to
kill off your alpha channel if you don't want to start talking about 64bit
pixels...).
So you basically want your app to think in "truecolor" in some way and then map
this to the RGB masks that the visual tells you (just use the default visual
and default colormap). In rare cases you may have to deal with 16bit RGB565
truecolor. Maybe 15bit RGB555. Mostly xRGB888 (x is unused bits) or ARGB8888
(Xrender visual with alpha added). I'd really only bother spending effort on
these kinds of displays.
On a very rare day you may still hit a pseudocolor display and thus have to
alloc colors. Thus why I suggest doing a 666 colorcube (216 colors) in this
case. If that alloc of the cube fails you could drop down to 222 (2 bits per
RGB) (64 colors), 111 (8 colors) or variants of this with more bits for some of
R, G and B, but I've found the results where number of bits per RGB being
different at these low number of bits to be... not so good. Finally you get down
to 1 bit monochrome. These nicely devolve from the above truecolor displays and
as they are vanishingly rare, can have less effort put in just to the point of
"it works".
You may encounter staticcolor displays still (EGA, CGA for example or a
"commodore 64", and I shall throw staticgray type displays into this bucket
too), but chances of this are exceedingly low to the point of nonexistent. You
could just treat these as 1 bit mono displays and be done with it once you know
what is black and what is white.
You COULD do an old trick which is to examine the colors and find those that
are visually brighter or darker (remember that green appears brighter to us
humans than red or blue) so you could sort the static color set as if it were
grayescale with RGB weighted like grey = (0.3 * r) + (0.58 * g) + (0.11 * b)
and then convert your RGB values to a greyscale value with the above formula
aand then choose the staticcolor entry that is closest to the greyscale
brightness of it given the above formula. A lookup table will do this quickly
and efficiently. You may end up with something like:
https://www.nightfallcrew.com/wp-content/uploads/2018/07/Gubbdata_2018.jpg?47b77a
Throw in dithering for all the color modes that are lower than 24/32bit (yes -
even dither 565 and 555 - when dithered it's incredibly close to 24/32bit RGB
and dithering works also for all the lower color modes too).
Now we talk of dithering... we begin to talk about rendering not using x
primitives. You can't sensibly dither using these (yes you can have fill
patterns ... a whole topic on its own).
In the end you devolve for sanity to an ARGB8888 (32bit) buffer in client
memory where alpha may or may not be used. You use the CPU to simply fill it in
pixel value by pixel value with whatever rendering mechanisms you like
client-side. Then you have a "convert to screen color" conversion client-side.
This may scale down to RGB565/555 or 8bit or mono 1 bit values based on
colormaps with dithering. You then XPutImage/XShmPutImage (Shm preferred if
available) and be done with it. Note that you can have a fast path for 99% of
cases where you use a 32bit RGB buffer that is the same pixel layout as the
screen needs and then directly upload this buffer with no conversion process.
the conversion is only needed if you happen to run on a "lesser display" which
is very rare.
Yes - you can use Xrender to render. This is not going to function on displays
that are not truecolor. At least last I remember in the early Xrender days when
pseudocolor etc. still was a thing - a niche thing, but a thing, I think the
entire extension was missing in this case? I'm pretty sure it didn't work or it
worked so poorly you wanted to do your own conversion/dithering. A well written
converter/dithering set of routines can run rings in speed around anything
server-side and can have massively better quality.
In a modern world really you probably want to either do the client-side
software rendering in a buffer, then upload as above, or actually move over to
OpenGL of Vulkan to get hardware accelerated rendering. These latter rendering
methods can be used for 2D drawing, not just 3D "games".
> Whatever method you use, if you start with RGB values, you need to
> follow the X scale of 0 to 65535.
>
> You can allocate color cells from a colormap using RGB values. Or,
> you can look up colors by name from a color database. X provides a
> database of more than 700 color names (most of them shades of gray,
> though). Each color in the database is defined with RGB values
> (interestingly, using a range of 0 to 255). You can look up common
> colors by name with the XLookupColor function:
>
>
> Status XLookupColor(
> Display* display,
> Colormap colormap,
> char* colorName,
> XColor* rgbColor, /* RETURN */
> XColor* hardwareColor) /* RETURN */
>
> On success, XLookupColor returns a nonzero value and fills in the two
> XColor structures. XLookupColor fills up the rgbColor structure with
> the red, green, and blue components as read in from the color
> database. XLookupColor also fills the hardwareColor structure with
> the closest match to the color definition that is supported on your
> hardware. Because of this, I tend to use the hardwareColor structure
> rather than the rgbColor structure.
>
> In both cases, the XColor structure holds the following values:
>
> typedef struct {
> unsigned long pixel;
> unsigned short red, green, blue;
> char flags;
> char pad;
> } XColor;
>
> XLookupColor fills in the red, green, and blue fields, along with the
> flags, which indicate which of the fields were used by bitmasks:
> DoRed, DoGreen, and DoBlue, respectively.
>
> You can then allocate cells in a colormap from the color definitions
> held in the XColor structures filled in by XLookupColor. X calls
> these color cells pixels, which aren't dots on the screen, but values
> that represent a position in a colormap that holds a given color. In
> simple colormaps, the pixel is merely an index into an array. This is
> not always the case though, as we'll discuss later.
>
> To allocate a color cell, call XAllocColor:
>
> Status XAllocColor(
> Display* display,
> Colormap colormap,
> XColor* hardwareColor) /* in/out */
>
> The XColor pointer is both an input and output parameter. On input,
> XAllocColor reads the RGB values and flags fields in the structure.
> On completion, XAllocColor fills in the pixel field with the pixel
> value for that color in the passed-in colormap.
>
> Before allocating a new color cell, XAllocColor checks if a color
> cell already holds the given RGB values in the colormap you pass in.
> This lets applications share common color cells, say, for red or
> gray. If you pass in the default colormap, chances are most common
> colors are already allocated.
>
> On success, you can use the pixel field for drawing. With the X
> library, you can call XSetForeground to store the color into a
> graphic context as the drawing color:
>
> int XSetForeground(
> Display * display,
> GC gc,
> unsigned long pixel)
>
> With a Motif application, you can use the pixel value to set the
> foreground or background resources for a widget. For example:
>
> XtVaSetValues(widget,
> XmNforeground,
> hardwareColor.pixel,
> NULL);
>
> You can also combine the tasks of looking up a color from a name and
> allocating a cell with XAllocNamedColor.
>
> Both XAllocColor and XAllocNamedColor allocate read-only color cells.
> That is, once set, you cannot change the definition of the color in
> that cell. Because of this, multiple applications, using the same
> colormap, can share the same colors.
>
> It's slightly more difficult to allocate cells that you can change
> later, called read-write color cells. To do this, call
> XAllocColorCells or XAllocColorPlanes. Allocating color planes lets
> you do fun things like use color planes for double-buffering. A later
> column will take that on. For now, XAllocColorCells takes the
> following parameters:
>
> Status XAllocColorCells(
> Display* display,
> Colormap colormap,
> Bool contiguous,
> unsigned long* planeMasks,
> unsigned int numberPlanes,
> unsigned long* pixels,
> unsigned int numberPixels)
>
> With XAllocColorCells, you can allocate both color planes and
> individual cells. If you set the contiguous flag to True, you can
> insist that the color planes allocated must be contiguous. This is
> not always possible if a number of colors have already been allocated
> from the colormap.
>
> If you're just interested in read-write color cells, pass False for
> the contiguous value, pass NULL for the planeMasks and zero for the
> numberPlanes. Pass the number of pixels-color cells-you want to
> allocate in the numberPixels parameter, and an array of unsigned long
> values long enough to hold the returned pixel IDs in the pixels
> parameter. For example:
>
> unsigned long pixels[12];
>
> status = XAllocColorCells( display, colormap, False, (unsigned long*)
> NULL, 0, pixels, 12);
>
> if (status != 0) { /* We're OK to use the colors in the pixels
> array...*/ }
>
> This call defines a 12-element array and asks for 12 color cells.
>
> Allocating read-write color cells is just the first step. X gives you
> these color cells without defining any colors in the cells. The next
> step is to do this by calling XStoreColor:
>
> XStoreColor(
> Display* display,
> Colormap colormap,
> XColor* colorDefinition)
>
> With XStoreColor, you must not only fill in the RGB values in the
> XColor structure, but the pixel field as well. Set the pixel field to
> one of the pixels you got back from the call to XAllocColorCells.
> Also remember to set the flags field, typically to the
> DoRed | DoGreen | DoBlue, the OR of these bitmasks.
>
> Once you store in colors, you can then use these color cells, the
> pixel values, with normal X drawing or widget color values. You can
> call XStoreColor, or its brethren XStoreColors or XStoreNamedColor,
> to then change the definition of a color cell on the fly. This is
> useful for animated effects or letting a user modify the interface.
>
> Colormaps
>
> So far, all of these calls require a colormap. You allocate colors in
> a colormap. You can create multiple colormaps for different tasks and
> your system hardware may be able to display one or more colormaps at
> the same time.
>
> If you use more colormaps than your hardware can support
> simultaneously, you'll see color flashing as you select windows.
> Parts of your display may also go black as other color maps are
> swapped in.
>
> To avoid color flashing, and to conserve what used to be precious
> color cells, most X applications use the default colormap. Using the
> default colormap makes your code simpler (you often don't have to do
> special coding to use this colormap) and helps enforce a unity of
> colors on your display. You can get the default colormap by calling
> the DefaultColormap macro:
>
> Colormap DefaultColormap(
> Display* display,
> int screenNumber)
>
> You can create your own colormaps, but here things start to become
> more complex because you have to deal with visuals.
>
> Visuals and Colormaps
>
> X abstracts the differences between color hardware through the
> concept of a visual. X provides six visual abstractions:
>
> PseudoColor visuals provide read-write color arrays. The pixel
> value provides an index into the array and the contents of the
> array hold the actual color value (the RGB data). You can change
> cells in a colormap created under a PseudoColor visual.
>
> GrayScale visuals are a lot like PseudoColor visuals, except you
> can only have shades of gray. With such a visual, you must set
> color cells to hold equal red, green, and blue values.
>
> StaticColor visuals are a lot like PseudoColor visuals, but the
> RGB values in the colormaps are predefined. These colormaps are
> read-only.
>
> StaticGray visuals are like StaticColor visuals, but allow for
> only predefined shades of gray. A black and white display has a
> very simple StaticGray visual.
>
> DirectColor visuals decompose the pixel values into separate
> fields for red, green, and blue. Each separate field then
> provides an index into a separate red, green, or blue colormap.
>
> TrueColor visuals also decompose the pixel values like
> DirectColor visuals. TrueColor visuals, though, have read-only
> RGB values. Typically, these RGB values provide a near linear
> ramp of colors.
>
> Most of the differences of the visual abstractions relate to the type
> of colormaps you can create for a given class of visual, such as
> PseudoColor or DirectColor.
>
> To create a colormap, call XCreateColormap:
>
> Colormap XCreateColormap(
> Display* display,
> Window window,
> Visual* visual,
> int allocateFlag)
>
> When creating a colormap, you can pre-allocate all the cells in the
> colormap by passing the flag AllocAll as the allocateFlag parameter.
> Pass AllocNone to request that no cells be allocated for your
> application up front. Note that for visual types that only provide
> read-only colormaps, you must pass AllocNone. These visuals are the
> StaticColor, StaticGray, and TrueColor visual types. It's up to you
> to remember which visuals lead to which types of colormaps.
>
> Working with Multiple Visuals
>
> Like the default colormap, there's also a default visual. Typically,
> this is a PseudoColor visual, since most X applications are coded to
> assume a PseudoColor visual. To get the default visual, call the
> DefaultVisual macro:
>
> Visual* DefaultVisual(
> Display* display,
> int screenNumber)
>
> To find the visuals available on your display, you can call
> XMatchVisualInfo to find a visual that matches a specific type of
> visual, such as DirectColor, and a preferred depth (number of color
> planes) that you pass in. If you are not sure which type of visual
> you want to use or its depth (perhaps you want to find the visual
> with the greatest depth, for example), then you can call
> XGetVisualInfo to get a list of visuals to choose from. Both of these
> functions provide XVisualInfo structures, from which you can get the
> Visual pointer you need for the X routines.
>
> The tricky part is that when you create a window, it must be
> associated with a visual. You can then associate any number of
> colormaps with a window, but each of these colormaps must have been
> created for the visual-the same visual the window is associated with.
> If you don't follow these rules, the X server will generate BadMatch
> errors.
>
> Because of this, it's generally a good idea to use the same visual
> across all the windows your application creates. Now, this becomes
> more complicated when you work with X toolkits, such as Motif. Motif
> tries to hide the creation of actual windows. In most cases, each
> Motif widget creates a window on the display (windows in X are
> relatively cheap resources). Motif (actually the underlying X Toolkit
> Intrinsics library) creates these windows when you call
> XtRealizeWidget, which is often the very last call made before
> jumping into the event loop.
>
> Furthermore, all Motif widgets have foreground, background, and
> borderColor resources, all of which refer to color cells. These color
> cells must be valid cells for the colormap associated with the
> window, which in turn must be created under the visual associated
> with the window.
>
> In practical terms, you should set the following resources for every
> top-level shell widget (widgets beneath the top-level shell can
> inherit the proper values):
>
> visual, which holds the visual you want to use for the window
>
> colormap, which must be a colormap created under the visual
>
> depth, which should match the depth (number of color planes) of
> the colormap, and be a valid depth supported by the visual
>
> background or backgroundPixmap, which must be a valid color cell
> in the colormap associated with the window or a valid pixmap that
> has a depth that matches the depth of the colormap
>
> borderColor or borderPixmap, which follow the same rules as
> listed previously
>
> foreground, which must be a valid color cell as described
> previously
>
> You need to set these resources on each top-level shell widget,
> before calling XtRealizeWidget. Remember that menus and dialogs are
> also top-level shell widgets. I've found these guidelines helpful:
>
> Try to use the same visual throughout your application. This
> stopss a whole series of BadMatch errors where the visuals,
> colormaps, and color cells don't match up.
>
> Where possible, use the same colormap throughout the application.
> High-end imaging applications won't be able to do this, of
> course. If you cannot use the same colormap, then try to use
> different colormaps only for special image-processing or other
> high-color-usage windows. For the rest of the application, use
> the same colormap. This often works well with a Motif drawing
> area widget for displaying an image or other color-intensive
> rendering.
>
> Always make sure all color cells used in a window come from a
> colormap associated with that window.
>
> With the rules associated with visuals, colormaps, and color cells,
> making use of X color capabilities can seem a daunting task, but if
> you start small and gradually build up your application, it will turn
> out fine.
>
> References
>
> 1. From TechWeb's TechEncyclopedia
> (http://www.techweb.com/encyclopedia/),
> a service of CMPnet and The Computer Language Co. Inc.
>
> Eric Foster-Johnson is the co-author of Power Programming Motif (with
> Kevin Reichard), and author of UNIX Programming Tools
> (MIS: Press/M&T Books).
>
> https://twitter.com/ericfosterjohns
>
> _______________________________________________
> xorg at lists.x.org: X.Org support
> Archives: http://lists.freedesktop.org/archives/xorg
> Info: https://lists.x.org/mailman/listinfo/xorg
> Your subscription address: %(user_address)s
>
--
------------- Codito, ergo sum - "I code, therefore I am" --------------
Carsten Haitzler - raster at rasterman.com
More information about the xorg
mailing list