Looking for low-latency method to read pixels from X framebuffer

Andreas Schmid andreas.schmid at ur.de
Tue Nov 23 16:42:07 UTC 2021


Dear Xorg developers,

I am trying to build an application that can measure the 
latency/processing time of graphics frameworks on Linux.
My idea is, for a first simple use case, to implement test programs 
gradually changing the screen's color while logging timestamps using 
different graphics and UI frameworks (e.g. SDL, OpenGL, Qt, ...). More 
complex test programs rendering user interfaces or 3D scenes will then 
be implemented in the future.
A separate measuring program should detect a pixel being updated in some 
kind of framebuffer (as close to the application as possible) and log 
another timestamp. This way, the actual rendering time of the test 
application should be measured.
The measuring program should be independent of vblank events, as I'm 
interested in the time the rendering is done, not the time users might 
be able to see something.

My problem is getting the framebuffer content with the second program.
As those measurements are part of a research project, I want to make 
sure to measure as exactly as possible.

Things I already tried:

1. Getting framebuffer content with fbdev or libdrm.
The problem here is that both require the programs to be run in a 
terminal without an active XServer (which I would prefer due to external 
validity). Additionally, fbdev is deprecated and accessing a libdrm 
framebuffer with mmap seems to be supported by embedded graphics chips only.

2. Using MIT-XShm (reading a pixel with XShmGetImage)
XShm looks promising but I have not found any information regarding its 
performance. It also looks like XShm gets blocked for a while when 
another application (in my case the test application) is drawing 
something. I assume this is due to a "LockDisplay()" call in XShmGetImage.
Additionally, XShm behaves very strangely (reading out old color values) 
when the test application is rendering as fast as it can. Adding a delay 
of at least 10 milliseconds between every update of the test application 
seems to fix this issue. This behavior also occurs, when a compositor 
(in my case compton) is running.

3. Using the Xdamage extension
Xdamage also looks very promising as its notifications arrive quite soon 
(about one millisecond) after the test application finishes rendering. 
However, this way it is obviously not possible to read a pixel's color, 
so I can't tell if the rendering is already finished when the 
XDamageNotify event arrives.

For better understanding, a visualization of my current measuring setup 
can be found here: https://hci.ur.de/framework_latency_problem

My questions are:
a) What is, in your opinion, the best time to stop the measurement timer?
b) Are there other ways besides XShm and Xdamage to detect a pixel 
changing its color?
c) Where does the strange delay of XShm come from?
d) Is the XDamageNotify event triggered before or after pixels are updated?

Kind regards,
Andreas



More information about the xorg-devel mailing list