[PATCH xserver] xfree86: fix gamma compute when palette_size > 256

[Michel Dänzer]

On 31/10/17 10:55 AM, Yu, Qiang wrote:
> 
>>> If so, does this what you want:
>>> 1. for palette_size <= 256, keep the original way
>>> 2. for palette_size > 256, enlarge the crtc gamma_size and copy into it
>>
>> Not sure what you mean by "copy into it". The driver should simply make
>> sure the CRTC gamma CLUT is large enough, in which case the code changed
>> by your patch works as is.
>>
> So you mean in DDX init, we should check if crtc gamma size is big enough,
> and enlarge when needed

Right.

> instead of doing the check and enlarge in xserver like the attached patch?

This patch doesn't make sense I'm afraid, because crtc->gamma_size and
crtc->randr_crtc->gammaSize don't have to be equal. The former should be
equal to (or larger than) the number of gamma CLUT entries used by the
hardware, which is a function of the screen depth. The latter is equal
to the maximum number of different R/G/B values that can be encoded in a
colormap entry, which is 1 << (maximum number of R/G/B bits in a single
gamma CLUT entry). Our drivers currently hardcode 1 << 10 for the
latter, because the hardware gamma CLUT we've been using has 10 R/G/B
bits per entry.


If we're going to change xserver code, I'd make xf86CrtcCreate choose
gamma_size based on the screen depth in the first place.


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer