XChangeProperty accesses 32 bit values as 64 bit

[Mark Kettenis]

> Date: Wed, 2 Jan 2019 16:31:39 +0100
> From: Hanno Böck <hanno at hboeck.de>
> 
> Hi,
> 
> Trying to debug a crash (in gajim) I discovered that it was due to a
> stack buffer overread in gtk+/libX11.
> 
> Digging down I am not entirely sure how to interpret it and whether
> it's libX11's or GTK's fault. Here's what's going on:
> 
> Gtk+ calls XChangeProperty where the second last parameters are a
> pointer to a pid, see e.g. [1]. The "format" parameter is "32", which
> is the bit size.
> 
> Now in libX11 it ends up crashing in the function _XData32, because it
> tries to access the variable as a long, which is 64 bit.
> 
> Now this is kinda documented [2], where it says:
> "If the specified format is 32, the property data must be a long array."
> 
> However that is - to put it mildly - unexpected and confusing. If I
> have a function that lets me tell I want to process a 32 bit value then
> learning that I have to pass that 32 bit value as a 64 bit value is
> surely the last thing I expected.

Yes. the X11 API dates from the time when 32-bit computing was
relatively new and short meant 16-bit and long meant 32-bit.  And then
64-bit computers came around some years later...

> Given this API this probably needs to be fixed in gtk by using long
> instead of pid_t for the pid, but I strongly recommend rethinking that
> design in libX11 and make it accept 32 bit values.

POSIX defines pid_t as "integer type" of "an appropriate length".
There is no guarantee that it is 32 bits, so GTK needs to be fixed
anyway.

Changing the libX11 API isn't possible at this point as software that
uses the interface correctly would break on a 64-bit system.  There is
of course libxcb which (as far as I can tell) doesn't have this
weirdness.