[PATCH] render: Avoid infinite loops in alpha map handling (#23581)

[Adam Jackson]

On Wed, 2010-05-12 at 10:19 -0700, Keith Packard wrote:

> Is there somewhere in the server that is recursing through the alphaMap?
> fb doesn't, and I don't know of any acceleration code that deals with alphaMap.

Using the testcase in the bug on an Xvfb:

(gdb) bt
#0  _int_malloc (av=0x3bef17ae80, bytes=<value optimized out>) at malloc.c:4727
#1  0x0000003beee79b0d in __libc_malloc (bytes=304) at malloc.c:3660
#2  0x0000003bfda1797b in _pixman_image_allocate () at pixman-image.c:98
#3  0x0000003bfda3a50a in pixman_image_create_bits (format=PIXMAN_x8r8g8b8, 
    width=10, height=<value optimized out>, bits=<value optimized out>, 
    rowstride_bytes=40) at pixman-bits-image.c:1004
#4  0x000000000041c0ed in create_bits_picture (pict=0x94d5a0, has_clip=0, 
    xoff=0x7fffff3ff18c, yoff=0x7fffff3ff188) at fbpict.c:291
#5  image_from_pict (pict=0x94d5a0, has_clip=0, xoff=0x7fffff3ff18c, 
    yoff=0x7fffff3ff188) at fbpict.c:434
#6  0x000000000041c220 in set_image_properties (pict=0x94d5a0, has_clip=0, 
    xoff=0x7fffff3ff1fc, yoff=0x7fffff3ff1f8) at fbpict.c:392
#7  image_from_pict (pict=0x94d5a0, has_clip=0, xoff=0x7fffff3ff1fc, 
    yoff=0x7fffff3ff1f8) at fbpict.c:459
#8  0x000000000041c220 in set_image_properties (pict=0x94d5a0, has_clip=0, 
    xoff=0x7fffff3ff26c, yoff=0x7fffff3ff268) at fbpict.c:392
#9  image_from_pict (pict=0x94d5a0, has_clip=0, xoff=0x7fffff3ff26c, 
    yoff=0x7fffff3ff268) at fbpict.c:459
/* ... */
#224620 0x000000000041c220 in set_image_properties (pict=0x94d5a0, has_clip=0, 
    xoff=0x7fffffffe08c, yoff=0x7fffffffe088) at fbpict.c:392
#224621 image_from_pict (pict=0x94d5a0, has_clip=0, xoff=0x7fffffffe08c, 
    yoff=0x7fffffffe088) at fbpict.c:459
#224622 0x000000000041c56a in fbComposite (op=1 '\001', pSrc=0x94d5a0, 
    pMask=0x0, pDst=0x94d5a0, xSrc=0, ySrc=0, xMask=0, yMask=0, xDst=0, 
    yDst=0, width=10, height=10) at fbpict.c:169
#224623 0x00000000004aa440 in damageComposite (op=1 '\001', pSrc=0x94d5a0, 
    pMask=0x0, pDst=0x94d5a0, xSrc=<value optimized out>, 
    ySrc=<value optimized out>, xMask=0, yMask=0, xDst=0, yDst=0, width=10, 
    height=10) at damage.c:643
#224624 0x00000000004a27ce in ProcRenderComposite (
    client=<value optimized out>) at render.c:723
#224625 0x000000000050392c in Dispatch () at dispatch.c:439
#224626 0x00000000004f424a in main (argc=<value optimized out>, 
    argv=0x7fffffffe3d8, envp=<value optimized out>) at main.c:286

So, no, nowhere in the server as such.

I'm actually a little impressed that gdb got to the bottom of that.

- ajax
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: This is a digitally signed message part
URL: <http://lists.x.org/archives/xorg-devel/attachments/20100512/67965241/attachment.pgp>