[PATCH] avoid a potential endless loop.

[Keith Packard]

On Tue, 2009-01-06 at 10:03 +0800, Xiang, Haihao wrote:
> Previously it is possible that creating rotation data, then cleaning
> up and creating again so that pScreen->BlockHandler and
> xf86_config->BlockHandler all point to xf86RotateBlockHandler.

Yes, this looks correct to me. So, the race condition I see here is:

xf86CrtcRotate (with transform)
	allocates rotation_damage
	wraps block handler
	pScreen->BlockHandler == xf86RotateBlockHandler
	xf86_config->BlockHandler = (other block handler)
xf86CrtcRotate (without transform)
	frees rotation_damage
	pScreen->BlockHandler == xf86RotateBlockHandler
	xf86_config->BlockHandler = (other block handler)

(note lack of xf86RotateBlockHandler call here)

xf86CrtcRotate (with transform)
	allocates rotation_damage
	wraps block handler
	xf86_config->BlockHandler = pScreen->BlockHandler

Note that this makes xf86->BlockHandler == xf86RotateBlockHandler, 
causing a loop as the block handler attempts to unhook itself from
the block handler chain.

xf86RotateBlockHandler
	pScreen->BlockHandler = xf86RotateBlockHandler
	(and recurse, causing a stack overflow)

-- 
keith.packard at intel.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.x.org/archives/xorg/attachments/20090109/6942f284/attachment.pgp>