[RFC PATCH xserver 0/5] Check glamor-xv usability

[Michel Dänzer]

On 21/04/17 12:35 AM, Olivier Fourdan wrote:
> It appears that on some hardware/diver combo such as nv30/nouveau, using
> GL_ALPHA as format for 8-bit depth will cause an incomplete attachment 
> error (GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT) when trying to bind the
> texture.
> 
> As a result, the FBO is NULL and glamor segfaults when trying to access
> the FBO width/height in pixmap_priv_get_scale().
> 
> This happens with glamor-xv which uses a 8bit pixmap, meaning that on such
> hardware/driver, trying to play a video using Xv will lead to a crash of
> the Xserver. This affects Xwayland, Xephyr and modesetting driver with
> glamor accel:
> 
>   https://bugs.freedesktop.org/show_bug.cgi?id=100710
>   https://bugzilla.redhat.com/1412814
> 
> So this series of RFC patches consist of 3 things:
> 
>  1. Check if using a 8bit pixmap causes a incomplete attachment error as
>     soon as glamor_xv_core_init() and return a boolean value indicating
>     the caller whether or not glamor-xv is usable.
>  2. Modify each callers, namely ephyr, modesetting and xwayland to take
>     advantage of this boolean value and do not enable xv if not usable.
>     One difficulty here is that to check FBO usability, one has to have
>     the screen resources created, which means that the changes also
>     include moving glamor_xv_core_init() to CreateScreenResources()
>     instead of screen init.

This means that external drivers calling glamor_xv_init would also have
to be changed accordingly, which is a bit unfortunate. Also, I'm not
sure deferring xf86XVScreenInit to CreateScreenResources is a good idea.

So, I hope one of Eric's suggestions will work out.


-- 
Earthling Michel Dänzer               |               http://www.amd.com
Libre software enthusiast             |             Mesa and X developer