[PATCH] libXvMC: avoid overflowing by making nameLen and busIDLen addition overflow

[Dave Airlie]

On Tue, May 28, 2013 at 1:26 AM, Julien Cristau <jcristau at debian.org> wrote:
> On Mon, May 27, 2013 at 09:45:06 +1000, Dave Airlie wrote:
>
>> From: Dave Airlie <airlied at redhat.com>
>>
>> Al Viro pointed this out on lwn, and I think this patch should fix it,
>> but making sure we don't do the addition as 32-bit on 64-bit machines.
>>
> How about this instead:
>
> diff --git a/src/XvMC.c b/src/XvMC.c
> index 74c8b85..a78bb0d 100644
> --- a/src/XvMC.c
> +++ b/src/XvMC.c
> @@ -573,7 +573,8 @@ Status XvMCGetDRInfo(Display *dpy, XvPortID port,
>         unsigned long realSize = 0;
>         char *tmpBuf = NULL;
>
> -       if (rep.length < (INT_MAX >> 2)) {
> +       if ((rep.length < (INT_MAX >> 2)) &&
> +           (rep.nameLen + rep.busIDLen >= rep.nameLen)) {
>             realSize = rep.length << 2;
>             if (realSize >= (rep.nameLen + rep.busIDLen)) {
>                 tmpBuf = Xmalloc(realSize);
>
> nameLen and busIDLen are unsigned int, so it should be enough to ensure
> the addition doesn't wrap around?

I prefer yours if you add a comment for the unwary.

Dave.