[PATCH] shape: Report empty region correctly

Pauli ext-pauli.nieminen at nokia.com
Thu Nov 18 04:42:20 PST 2010


From: Pauli Nieminen <ext-pauli.nieminen at nokia.com>

If client calls XShapeGetRectangles when region is empty return values
is NULL instead of empty rectangle.

Steps to reproduce problem:
XShapeCombineRectangles(ShapeSet with rectangle that has zero width or height)
XShapeGetRectangles() // returns NULL

Documentation claims:
"If the extension is not supported, XShapeGetRectangles returns NULL.
Otherwise, it returns a list of rectangles that describe the region
specified by kind." [1]

[1] http://www.x.org/releases/X11R7.6-RC1/doc/libXext/shapelib.html

To fix the problem XShapeGetRectangles has to check if there is no
rectangles in region and return sane values instead of no rectangles.

Signed-off-by: Pauli Nieminen <ext-pauli.nieminen at nokia.com>
---
 Xext/shape.c |    4 ++++
 1 files changed, 4 insertions(+), 0 deletions(-)

diff --git a/Xext/shape.c b/Xext/shape.c
index ac95328..4228d9f 100644
--- a/Xext/shape.c
+++ b/Xext/shape.c
@@ -1044,6 +1044,10 @@ ProcShapeGetRectangles (ClientPtr client)
 	BoxPtr box;
 	nrects = RegionNumRects(region);
 	box = RegionRects(region);
+	if (nrects == 0) {
+		nrects = 1;
+		box = &RegionEmptyBox;
+	}
 	rects = malloc(nrects * sizeof (xRectangle));
 	if (!rects && nrects)
 	    return BadAlloc;
-- 
1.7.0.4



More information about the xorg-devel mailing list