[Mesa-dev] [PATCH 3/3] glsl: rewrite glsl_type::record_key_hash() to avoid buffer overflow

[Ian Romanick]

Reviewed-by: Ian Romanick <ian.d.romanick at intel.com>

On 04/17/2015 06:52 AM, Brian Paul wrote:
> This should be more efficient than the previous snprintf() solution.
> But more importantly, it avoids a buffer overflow bug that could result
> in crashes or unpredictable results when processing very large interface
> blocks.
> 
> For the app in question, key->length = 103 for some interfaces.  The check
> if size >= sizeof(hash_key) was insufficient to prevent overflows of the
> hash_key[128] array because it didn't account for the terminating zero.
> In this case, this caused the call to hash_table_string_hash() to return
> different results for identical inputs, and then shader linking failed.
> 
> This new solution also takes all structure fields into account instead
> of just the first 15 when sizeof(pointer)==8.
> 
> Cc: mesa-stable at lists.freedesktop.org
> ---
>  src/glsl/glsl_types.cpp | 23 +++++++++++++----------
>  1 file changed, 13 insertions(+), 10 deletions(-)
> 
> diff --git a/src/glsl/glsl_types.cpp b/src/glsl/glsl_types.cpp
> index 4aa36a7..9c9b7ef 100644
> --- a/src/glsl/glsl_types.cpp
> +++ b/src/glsl/glsl_types.cpp
> @@ -738,24 +738,27 @@ glsl_type::record_key_compare(const void *a, const void *b)
>  }
>  
>  
> +/**
> + * Generate an integer hash value for a glsl_type structure type.
> + */
>  unsigned
>  glsl_type::record_key_hash(const void *a)
>  {
>     const glsl_type *const key = (glsl_type *) a;
> -   char hash_key[128];
> -   unsigned size = 0;
> -
> -   size = snprintf(hash_key, sizeof(hash_key), "%08x", key->length);
> +   uintptr_t hash = key->length;
> +   unsigned retval;
>  
>     for (unsigned i = 0; i < key->length; i++) {
> -      if (size >= sizeof(hash_key))
> -	 break;
> -
> -      size += snprintf(& hash_key[size], sizeof(hash_key) - size,
> -		       "%p", (void *) key->fields.structure[i].type);
> +      /* casting pointer to uintptr_t */
> +      hash = (hash * 13 ) + (uintptr_t) key->fields.structure[i].type;
>     }
>  
> -   return hash_table_string_hash(& hash_key);
> +   if (sizeof(hash) == 8)
> +      retval = (hash & 0xffffffff) ^ ((uint64_t) hash >> 32);
> +   else
> +      retval = hash;
> +
> +   return retval;
>  }
>  
>  
>