xf86-input-mouse signed/unsigned issue with Microsoft protocol

[Peter Hutterer]

On Fri, Aug 28, 2009 at 09:17:37AM -0500, Donald Kayser wrote:
> I am developing under a PPC embedded system running 2.6.30.2 Linux,  
> xserver from debian distribution of 7.3+19, xf86-input-mouse 1.3.0. I  
> have a custom board that I am sending Microsoft mouse 3 byte protocol  
> through a pipe to the input of xserver via the mouse driver. I was able 
> to make small movements, all positive, but if I moved negative, the mouse 
> would jump a large amount in the positive direction. I checked out from 
> git and built the mouse driver and turned on debug to find out and it is 
> not accepting negative numbers correctly in the case of PROT_MS. By 
> changing the cast on lines 1304/1305 in mouse.c from (char) to (signed 
> char), I fixed the problem. I don't know if this exists on other (non 
> PPC) platforms.  I also noticed the potential for the same problem in a 
> few more places. I am using gcc-4.3.
>
> The source I have is from:
> xf86-input-mouse-1.4.0-13-gf292f23
>
> diff --git a/src/mouse.c b/src/mouse.c
> index aff2512..b59a138 100644
> --- a/src/mouse.c
> +++ b/src/mouse.c
> @@ -1301,8 +1301,8 @@ MouseReadInput(InputInfoPtr pInfo)
>                 buttons = (pMse->lastButtons & 2)
>                         | ((int)(pBuf[0] & 0x20) >> 3)
>                         | ((int)(pBuf[0] & 0x10) >> 4);
> -           dx = (char)(((pBuf[0] & 0x03) << 6) | (pBuf[1] & 0x3F));
> -           dy = (char)(((pBuf[0] & 0x0C) << 4) | (pBuf[2] & 0x3F));
> +           dx = (signed char)(((pBuf[0] & 0x03) << 6) | (pBuf[1] &  
> 0x3F));
> +           dy = (signed char)(((pBuf[0] & 0x0C) << 4) | (pBuf[2] &  
> 0x3F));
>             break;
>         case PROT_GLIDE:        /* ALPS GlidePoint */

this seems to be a problem on your particular architecture and while this
may be the only point where it is obvious there are more places where char
is assumed to be signed (as you mentioned). try passing -fsigned-char to gcc. 
I wonder if there's a configure(.ac) option to do that.

Cheers,
  Peter