[PATCH 0/5] Improve Present support in Xwayland with per window flips

[Roman Gilg]

Originating from the bug report

https://bugs.freedesktop.org/show_bug.cgi?id=99702

and my own observations with Xwayland misbehaving when outscanning on overlay planes, this patch series aims at improving Present support in Xwayland.

For that it introduces an internal flip mode API to Present, with that it's possible to try other Pixmap flips than just for a whole screen like before. For Xwayland we add a flip mode per window, but for example in the future we could also try to add a mode for flips per CRTC. Anyway the idea is to clearly separate different flip modes with their own code paths.

In Xwayland we flip per window, and also with the last patch in the series use sub-surfaces for that in order to flip on child windows. In my tests this was still somewhat fragile.

I tested these patches with several applications and environments. All patches should work on their own without their successors.

Test systems:
* RX470 with AMDGPU
* Intel graphics

Test environments:
* KDE Neon Dev Unstable (KWin) with Padoka PPA
* Fedora 26 (Mutter)

Test applications:
* Neverball
* VLC
* Google Chrome (with forced hardware acceleration)
* Steam
* Tomb Raider, Middle-Earth: Shadow of Mordor

Problems without sub-surfaces:
Steam login window is not showing buffer content. Might be a wrong implementation of direct rendering in the application.

Problems with sub-surfaces:
* The above problem with the Steam login window
* Resizing windows shows artifacts in the not yet updated sub-surface areas at the expanded edges
* KWin on AMDGPU: Won't quit completely when a sub-surface was used for Pixmap flipping at some point on run time. The error might be KWin internal, since it works with Mutter.
* KWin on AMDGPU: Surface of Chrome sometimes hangs on resizing operation. Might be again error in KWin, since it works with Mutter
* Mutter: VLC sub-surface misaligned (probably Mutter needs fix, since I needed to do something similar for KWin)
* Mutter: Sub-surface not correctly updating in Chrome (for example on tab switch, or when switching a Youtube video to full screen)
* Mutter: Neverball framerate capped to 120 (probably Mutter doesn't release buffers faster)

You can pull the patches also from this branch of my GitHub repo: https://github.com/subdiff/xserver/tree/gsoc