How to convert ZPixMap to BGRA reliably?

[Ilya Anfimov]

On Tue, Aug 22, 2017 at 11:39:26AM +0530, Sai Prasanna wrote:
>    I thought Zpixmap mapped directly to BGRA. But my assumption didn't turn
>    out to be correct in some older X versions.
> 
>    I am using xcb_get_image and xcb_shm_get_image to grab screen pixels. I
>    used Zpixmap option for image format. In new-ish Xorg versions it works
>    perfectly.
> 
>    But in some machines with older Xorg (1.15 and below) , I get discolored,
>    overlapped data, and it is less than 4 * width * height bytes. If I try to
>    read with bytes per pixel as 4 , overflow occurs.
> 
>    Could  this be some environment issue , or is there any other "proper" way
>    to convert Zpixmap to bgra/rgb formats?

 1) The best source of knowledge about core X11 functionality
is the "X Window System Protocol" book, available e.g. here:
  
  https://www.x.org/docs/XProtocol/proto.pdf

 2) The general pixel storage of the Z pixmap format is described
there in section 8 (Connection Setup), "Server  information"  and
pixel contents -- in "Visual information".

 It  does not need to be 4 bytes per pixel, or even integer bytes
per pixel. Number of bits per pixel is specified in FORMAT: field
of  server  connection  setup  response  (and  may  be  taken via
xcb_get_setup in xcb library).
 Many X servers round pixel size up to integer number  of  bytes,
as  it  allows  some shortcuts and speedups in server implementa-
tion, but not all do that.

 Pixels does not need to be in {pad,r,g,b} format. The masks  for
red,  green and blue are specified in VISUALTYPE: field of server
connection setup response. Fortunately, that masks  are  contigu-
ous,  but  they  don't  need to lay on a byte boundary.  However,
modern processors don't care much about bit shift  size,  and  it
does not matter -- shift by 8 bits or 11.


 Also,  pixels  will  be in different byte orders as specified by
image-byte-order: field in server response. You should prepare to
access  individual  bytes, not integers (or find some other solu-
tion).

 It  would be good to build you program for something like Debian
mips architecture (not mips-el!) and check  all  combinations  of
your program and X server in something like qemu-system-mips.


 PS  And  yes,  probably  the most recommended ways to read image
from X11 is to use some image manipulation library,  like  libgdk
or imagemagick.
 The  required  transformations are fairly simple, though, and if
you are messing with xcb, that it  should  be  really  simple  to
write a conversion code in like 30-50 lines.