[PATCH v3] xnest: Ignore GetImage() error in xnestGetImage()

[Egbert Eich]

From: Radek Doulik <rodo at novell.com>

When an Xnest instance is not viewable it will crash when a client in
that instance calls GetImage. This is because the Xnest server will
itself receives a BadMatch error.
This patch ignores the error. The application which has requested the
image will receive garbage - this however is fully legal according
to the specs as obscured areas will always contain garbage if there
isn't some sort of backing store as discussed in
https://bugs.freedesktop.org/show_bug.cgi?id=9488
The applied patch is a version from Dadek Doulik.

v2: Call XSync() before changing error handlers as suggested by
    Daniel Stone <daniel at fooishbar.org>.
v3: Don't call Xsync before restoring error handler as any errors
    generated by XGetImage() should be processed when this call
    returns as suggested by Jamey Sharp <jamey at minilop.net>

Signed-off-by: Egbert Eich <eich at freedesktop.org>
---
 hw/xnest/GCOps.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/hw/xnest/GCOps.c b/hw/xnest/GCOps.c
index e26a136..d00511d 100644
--- a/hw/xnest/GCOps.c
+++ b/hw/xnest/GCOps.c
@@ -94,15 +94,28 @@ xnestPutImage(DrawablePtr pDrawable, GCPtr pGC, int depth, int x, int y,
     }
 }
 
+static int
+xnestIgnoreErrorHandler (Display     *display,
+                         XErrorEvent *event)
+{
+    return False; /* return value is ignored */
+}
+
 void
 xnestGetImage(DrawablePtr pDrawable, int x, int y, int w, int h,
               unsigned int format, unsigned long planeMask, char *pImage)
 {
     XImage *ximage;
     int length;
+    int (*old_handler)(Display*, XErrorEvent*);
+
+    /* we may get BadMatch error when xnest window is minimized */
+    XSync(xnestDisplay, False);
+    old_handler = XSetErrorHandler (xnestIgnoreErrorHandler);
 
     ximage = XGetImage(xnestDisplay, xnestDrawable(pDrawable),
                        x, y, w, h, planeMask, format);
+    XSetErrorHandler(old_handler);
 
     if (ximage) {
         length = ximage->bytes_per_line * ximage->height;
-- 
1.8.1.4