[PATCH 2/3] drm: Be more paranoid with integer overflows

[Daniel Vetter]

On Thu, May 24, 2012 at 08:53:59PM +0300, ville.syrjala at linux.intel.com wrote:
> From: Ville Syrjälä <ville.syrjala at linux.intel.com>
> 
> Make sure 'width * cpp' and 'height * pitch + offset' don't exceed
> UINT_MAX.
> 
> Signed-off-by: Ville Syrjälä <ville.syrjala at linux.intel.com>
> ---
>  drivers/gpu/drm/drm_crtc.c |   10 +++++++++-
>  1 files changed, 9 insertions(+), 1 deletions(-)
> 
> diff --git a/drivers/gpu/drm/drm_crtc.c b/drivers/gpu/drm/drm_crtc.c
> index 80a34e7..e1b53fb 100644
> --- a/drivers/gpu/drm/drm_crtc.c
> +++ b/drivers/gpu/drm/drm_crtc.c
> @@ -2211,13 +2211,21 @@ static int framebuffer_check(struct drm_mode_fb_cmd2 *r)
>  
>  	for (i = 0; i < num_planes; i++) {
>  		unsigned int width = r->width / (i != 0 ? hsub : 1);
> +		unsigned int height = r->height / (i != 0 ? vsub : 1);
> +		unsigned int cpp = drm_format_plane_cpp(r->pixel_format, i);
>  
>  		if (!r->handles[i]) {
>  			DRM_DEBUG_KMS("no buffer object handle for plane %d\n", i);
>  			return -EINVAL;
>  		}
>  
> -		if (r->pitches[i] < drm_format_plane_cpp(r->pixel_format, i) * width) {
> +		if ((uint64_t) width * cpp > UINT_MAX)
> +			return -ERANGE;
> +

iirc that blows up on 32bit because gcc likes to use a compiler built-in.
And the usual pattern I've seen is if (UINT_MAX / a < b) return -ERANGE;
-Daniel

> +		if ((uint64_t) height * r->pitches[i] + r->offsets[i] > UINT_MAX)
> +			return -ERANGE;
> +
> +		if (r->pitches[i] < width * cpp) {
>  			DRM_DEBUG_KMS("bad pitch %u for plane %d\n", r->pitches[i], i);
>  			return -EINVAL;
>  		}
> -- 
> 1.7.3.4
> 
> _______________________________________________
> dri-devel mailing list
> dri-devel at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/dri-devel

-- 
Daniel Vetter
Mail: daniel at ffwll.ch
Mobile: +41 (0)79 365 57 48