[PATCH:libX11 4/4] Fix out-of-range comparison.

[Alan Coopersmith]

On 06/25/13 09:34 AM, Thomas Klausner wrote:
> clang complained (correctly):
> warning: comparison of constant 768614336404564650 with expression
> of type 'CARD32' (aka 'unsigned int') is always true
> [-Wtautological-constant-out-of-range-compare]
> ---
>   src/Font.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/Font.c b/src/Font.c
> index a32f740..ac18024 100644
> --- a/src/Font.c
> +++ b/src/Font.c
> @@ -588,7 +588,7 @@ _XF86BigfontQueryFont (
>   	    if (!(extcodes->serverCapabilities & CAP_VerifiedLocal)) {
>   		struct shmid_ds buf;
>   		if (!(shmctl(reply.shmid, IPC_STAT, &buf) >= 0
> -		      && reply.nCharInfos < (LONG_MAX / sizeof(XCharStruct))
> +		      && reply.nCharInfos < (INT_MAX / sizeof(XCharStruct))
>   		      && buf.shm_segsz >= reply.shmsegoffset + reply.nCharInfos * sizeof(XCharStruct) + sizeof(CARD32)
>   		      && *(CARD32 *)(addr + reply.shmsegoffset + reply.nCharInfos * sizeof(XCharStruct)) == extcodes->serverSignature)) {
>   		    shmdt(addr);
>

Technically LONG_MAX is correct, and the check is right for 32-bit systems,
while a no-op on 64-bit.   However, I've gone ahead and applied this patch
with the added note in the commit comment to explain:

     [While LONG_MAX is correct, since it's used in size_t math, the
      numbers have to be limited to 32-bit range to be usable by 32-bit
      clients, and values beyond that range are far more likely to be
      bugs in the data from the server than valid numbers of characters
      in a font.   -alan- ]

     Reviewed-by: Alan Coopersmith <alan.coopersmith at oracle.com>
     Signed-off-by: Alan Coopersmith <alan.coopersmith at oracle.com>

And pushed: To ssh://git.freedesktop.org/git/xorg/lib/libX11
    383e2b0..6d92608  master -> master

-- 
	-Alan Coopersmith-              alan.coopersmith at oracle.com
	 Oracle Solaris Engineering - http://blogs.oracle.com/alanc