[PATCH xserver] xfree86: fix gamma compute when palette_size > 256

[Michel Dänzer]

On 31/10/17 01:02 AM, Yu, Qiang wrote:
> 
>>>>> But I see driver like amdgpu with old KMS driver only has 256 entries and the
>>>>> gamma set syscall won't work correctly for size>256.
>>>>
>>>> That needs to be fixed then, and the userspace driver should probably
>>>> only allow setting depth 30 when it knows the kernel driver can handle it.
>>> That's the DDX work. For amdgpu, seems we can support it in DC KMS.
>>
>> Right, let's handle this in the driver, and drop this patch.
> Do you mean do the 30bit support check in DDX,

Right.

> but the gamma calculation still need be fixed here (because this
> patch is all about this)?

I don't see the point of this patch, because I don't see how running at
depth 30 with a 256-entry gamma CLUT can work in any useful way.


> If so, does this what you want:
> 1. for palette_size <= 256, keep the original way
> 2. for palette_size > 256, enlarge the crtc gamma_size and copy into it

Not sure what you mean by "copy into it". The driver should simply make
sure the CRTC gamma CLUT is large enough, in which case the code changed
by your patch works as is.

Another possibility would be making xf86CrtcCreate choose the initial
gamma CLUT size depending on the screen depth. But the driver would
still need to handle it for currently released versions of xserver.


> 3. after enlarge crtc gamma_size, if palette size go back to <= 256,
> shrink it to 256

AFAIK the screen depth and by extension palette size can never change
during the server's lifetime.


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer