[PATCH xserver] xwayland: avoid a crash with empty window pixmaps

[Olivier Fourdan]

This is a rare occurrence of a crash in Xwayland for which I don't have
the reproducing steps, just a core file.

The backtrace looks as follow:

  #0  raise () from /usr/lib64/libc.so.6
  #1  abort () from /usr/lib64/libc.so.6
  #2  OsAbort () at utils.c:1361
  #3  AbortServer () at log.c:877
  #4  FatalError () at log.c:1015
  #5  OsSigHandler () at osinit.c:154
  #6  <signal handler called>
  #7  xwl_glamor_pixmap_get_wl_buffer () at xwayland-glamor.c:162
  #8  xwl_screen_post_damage () at xwayland.c:514
  #9  block_handler () at xwayland.c:665
  #10 BlockHandler () at dixutils.c:388
  #11 WaitForSomething () at WaitFor.c:219
  #12 Dispatch () at dispatch.c:422
  #13 dix_main () at main.c:287

The crash is caused by dereferencing “xwl_pixmap->buffer” in
xwl_glamor_pixmap_get_wl_buffer() because “xwl_pixmap” is NULL.

Reason for this is because the corresponding pixmap has a size of 0×0
and no xwl_pixmap is created for pixmaps of size 0×0.

Avoid the NULL pointer dereference by checking the actual “xwl_pixmap”
value in both glamor and shm implementations of pixmap_get_wl_buffer()
and return a NULL buffer if there is no “xwl_pixmap”.

Signed-off-by: Olivier Fourdan <ofourdan at redhat.com>
---
 hw/xwayland/xwayland-glamor.c | 3 +++
 hw/xwayland/xwayland-shm.c    | 7 ++++++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/hw/xwayland/xwayland-glamor.c b/hw/xwayland/xwayland-glamor.c
index 8ffb40d6f..be2d0e8f5 100644
--- a/hw/xwayland/xwayland-glamor.c
+++ b/hw/xwayland/xwayland-glamor.c
@@ -159,6 +159,9 @@ xwl_glamor_pixmap_get_wl_buffer(PixmapPtr pixmap)
     struct xwl_pixmap *xwl_pixmap = xwl_pixmap_get(pixmap);
     int prime_fd;
 
+    if (!xwl_pixmap)
+        return NULL;
+
     if (xwl_pixmap->buffer)
         return xwl_pixmap->buffer;
 
diff --git a/hw/xwayland/xwayland-shm.c b/hw/xwayland/xwayland-shm.c
index 29732eaca..8aeb7c205 100644
--- a/hw/xwayland/xwayland-shm.c
+++ b/hw/xwayland/xwayland-shm.c
@@ -274,7 +274,12 @@ xwl_shm_destroy_pixmap(PixmapPtr pixmap)
 struct wl_buffer *
 xwl_shm_pixmap_get_wl_buffer(PixmapPtr pixmap)
 {
-    return xwl_pixmap_get(pixmap)->buffer;
+    struct xwl_pixmap *xwl_pixmap = xwl_pixmap_get(pixmap);
+
+    if (!xwl_pixmap)
+        return NULL;
+
+    return xwl_pixmap->buffer;
 }
 
 Bool
-- 
2.14.3