[PATCH xserver] xfree86: fix gamma compute when palette_size > 256

[Yu, Qiang]

>>>> But I see driver like amdgpu with old KMS driver only has 256 entries and the
>>>> gamma set syscall won't work correctly for size>256.
>>>
>>> That needs to be fixed then, and the userspace driver should probably
>>> only allow setting depth 30 when it knows the kernel driver can handle it.
>> That's the DDX work. For amdgpu, seems we can support it in DC KMS.
>
> Right, let's handle this in the driver, and drop this patch.
Do you mean do the 30bit support check in DDX, but the gamma calculation
still need be fixed here (because this patch is all about this)?
If so, does this what you want:
1. for palette_size <= 256, keep the original way
2. for palette_size > 256, enlarge the crtc gamma_size and copy into it
3. after enlarge crtc gamma_size, if palette size go back to <= 256, shrink it to 256

For 3, not sure if it's possible because we config 30 in the xorg.conf.

Regards,
Qiang