[PATCH 1/5] Xdmx: integer overflow in GetGLXVisualConfigs()

[Alan Coopersmith]

On 06/ 1/13 02:47 AM, Julien Cristau wrote:
> On Thu, May 23, 2013 at 09:27:26 -0700, Alan Coopersmith wrote:
>
>> @@ -92,15 +96,16 @@ GetGLXVisualConfigs(Display * dpy, int screen, int *nconfigs)
>>       }
>>
>>       /* Allocate memory for our config structure */
>> -    config = (__GLXvisualConfig *)
>> -        Xmalloc(nvisuals * sizeof(__GLXvisualConfig));
>> +    if (nvisuals < (INT_MAX / sizeof(__GLXvisualConfig)))
>> +        config = Xcalloc(nvisuals, sizeof(__GLXvisualConfig));
>
> Isn't the switch to calloc enough to fix the overflow?

On calloc implementations which check for overflow themselves, yes,
which we hope all common ones do, but I sadly know of some uncommon
ones which do not.

http://src.illumos.org/source/xref/illumos-gate/usr/src/lib/libmtmalloc/common/mtmalloc.c#329

-- 
	-Alan Coopersmith-              alan.coopersmith at oracle.com
	 Oracle Solaris Engineering - http://blogs.oracle.com/alanc