[Xorg] 64-bit applications and hostnames that start with a digit

[Paul Anderson]

I stumbled across a 64-bit issue in the transport layer that occurs
when connecting to hostnames that start with a digit (rather than a
letter).  This type of hostname appears to be allowed by RFC1123.

The problem occurs in xc/lib/xtrans/Xtranssock.c in
the function TRANS(SocketINETConnect)().  The variable tmpaddr
is defined as an "unsigned long".  Later in the routine, there
is a block that looks something like:

    /*
     * fill in sin_addr
     */

    /* check for ww.xx.yy.zz host string */

    if (isascii (host[0]) && isdigit (host[0])) {
	tmpaddr = inet_addr (host); /* returns network byte order */
    } else {
	tmpaddr = -1;
    }

    PRMSG (4,"TRANS(SocketINETConnect) inet_addr(%s) = %p\n",
	host, tmpaddr, 0);

    if ((long)tmpaddr == -1L)
    {
        ....call XGethostbyname(host,...) and then fill in sockname.sin_addr
        accordingly
    }
    else
    {
        sockname.sin_addr.s_addr = tmpaddr;
    }


The problem is that if the hostname starts with a digit, we'll
call inet_addr(), and this will fail with a value of -1 (because
the string isn't in the internet standard "dot" notation). 

In a 64-bit library, the "if" test that compares tmpaddr to -1L
will fail because tmpaddr is only 0xffffffff whereas -1L is
0xffffffffffffffff.  As a result, we'll end up taking the "else"
clause and copying -1 to sockname.sin_addr.s_addr, causing the 
subsequent call to connect() to fail.

To fix this, I'd like to use something more portable and get 
away from having to cast values, since those can cause problems,
such as in this situation.  Does anyone have any objections 
to something such as the following:

    in_addr_t         tmpaddr;

    ....

    /* check for ww.xx.yy.zz host string */

    if (isascii (host[0]) && isdigit (host[0])) {
	tmpaddr = inet_addr (host); /* returns network byte order */
    } else {
	tmpaddr = INADDR_NONE;
    }

    PRMSG (4,"TRANS(SocketINETConnect) inet_addr(%s) = %p\n",
	host, tmpaddr, 0);

    if (tmpaddr == INADDR_NONE)
    {
        ....call XGethostbyname(host,...) and then fill in sockname.sin_addr
        accordingly
    }
    else ...


>From the header files I've seen, INADDR_NONE is defined properly
to cast the -1 to (in_addr_t), so this seems like a safer way
to accomplish what the original code does.

Any comments?  Does anyone have any better ideas?

-paul