VNC patch for Xserver 1.3

[Colin Guthrie]

Stefan Dirsch wrote:
> On Thu, Oct 18, 2007 at 09:21:21AM +0100, Alan Hourihane wrote:
>> On Wed, 2007-10-17 at 13:08 +0200, Stefan Dirsch wrote:
>>> On Thu, Aug 23, 2007 at 04:31:02PM +0200, Stefan Dirsch wrote:
>>>> Thanks. Unfortunately the Xserver crashes immediately now.
>>>>
>>>> (gdb) run
>>>>
>>>> Program received signal SIGSEGV, Segmentation fault.
>>>> [Switching to Thread 0x2ab337bb4290 (LWP 4737)]
>>>> 0x00000000004a0bdc in xf86CollectInputOptions ()
>>>> (gdb) bt
>>>> #0  0x00000000004a0bdc in xf86CollectInputOptions ()
>>>> #1  0x00002ab338d5a019 in xf86rfbKeybInit ()
>>>>    from /usr/lib64/xorg/modules//extensions/libvnc.so
>>>> #2  0x0000000000467fdc in InitInput ()
>>>> #3  0x0000000000439931 in main ()
>>>> (gdb) quit
>>> Tried again with X.Org 7.3 and current xf4vnc CVS. Still crashing. :-)
>>> Here are the results of my investigations.
>>>
>>> Backtrace:
>>> 0: X(xf86SigHandler+0x6a) [0x48ce3a]
>>> 1: /lib64/libc.so.6 [0x2b2b2e271ba0]
>>> 2: /lib64/libc.so.6(strlen+0x40) [0x2b2b2e2b6ba0]
>>> 3: X(xf86ActivateDevice+0x59) [0x47b389]
>>> 4: X(InitInput+0x4d) [0x46785d]
>>> 5: X(main+0x370) [0x436210]
>>> 6: /lib64/libc.so.6(__libc_start_main+0xf4) [0x2b2b2e25eb24]
>>> 7: X(FontFileCompleteXLFD+0x259) [0x435659]
>>>
>>> Program received signal SIGSEGV, Segmentation fault.
>>> 0x00002b587f48eba0 in strlen () from /lib64/libc.so.6
>>> (gdb) bt
>>> #0  0x00002b587f48eba0 in strlen () from /lib64/libc.so.6
>>> #1  0x000000000047b389 in xf86ActivateDevice (local=0x8be9d0)
>>>     at xf86Xinput.c:158
>>> #2  0x000000000046785d in InitInput (argc=<value optimized out>, 
>>>     argv=<value optimized out>) at xf86Init.c:960
>>> #3  0x0000000000436210 in main (argc=1, argv=0x7fff2ce5c1a8, envp=0x83a090)
>>>     at main.c:397
>>>
>>> 158             local->atom = MakeAtom(local->type_name,
>>> 159                                    strlen(local->type_name),
>>> 160                                    TRUE);
>>>
>>> (gdb) p* local
>>> $3 = {next = 0x8be900, name = 0x7ef4d0 "Mouse[3]", flags = 78, 
>>>   device_control = 0x2b0c70993820 <xf86rfbMouseControlProc>, read_input = 0, 
>>>   control_proc = 0, close_proc = 0, switch_mode = 0, conversion_proc = 0, 
>>>   reverse_conversion_proc = 0, set_device_valuators = 0, fd = -1, atom = 0, 
>>>   dev = 0x0, private = 0x0, private_flags = 0, first = 0, last = 0, old_x = 0, 
>>>   old_y = 0, type_name = 0x0, always_core_feedback = 0x2b0c709acb3c, 
>>>   conf_idev = 0x0, drv = 0x7f5000, module = 0x0, options = 0x0, 
>>>   history_size = 256}
>>>
>>> Looks like the pInfo structure gets not filled properly. I'm not sure,
>>> where this is done. Here is the configuration I'm currently using.
>>>
>>> Section "InputDevice"
>>>   Driver       "rfbkeyb"
>>>   Identifier   "Keyboard[2]"
>>>   Option       "InputFashion" "VNC"
>>> EndSection
>>>
>>> Section "InputDevice"
>>>   Driver       "rfbmouse"
>>>   Identifier   "Mouse[3]"
>>>   Option       "InputFashion" "VNC"
>>> EndSection
>>>
>>> Section "ServerLayout"
>>>   [...]
>>>   InputDevice  "Keyboard[2]" "ExtraKeyboard"
>>>   InputDevice  "Mouse[3]" "ExtraPointer"
>>> EndSection
>>>
>>> You get different results for pInfo for a regular mouse section and of
>>> course no crashes.
>>>
>>> Section "InputDevice"
>>>   Driver       "mouse"
>>>   Identifier   "Mouse[1]"
>>>   Option       "Buttons" "5"
>>>   Option       "Device" "/dev/input/mice"
>>>   Option       "Name" "MX310"
>>>   Option       "Protocol" "ExplorerPS/2"
>>>   Option       "Vendor" "Logitech"
>>>   Option       "ZAxisMapping" "4 5"
>>> EndSection
>>>
>>> (gdb) p* local
>>> $2 = {next = 0x8b9230, name = 0x7ee9d0 "Mouse[1]", flags = 78, 
>>>   device_control = 0x2b61f0cebea0 <MouseProc>, 
>>>   read_input = 0x2b61f0ceb1a0 <MouseReadInput>, control_proc = 0, 
>>>   close_proc = 0, switch_mode = 0, 
>>>   conversion_proc = 0x2b61f0ce86a0 <MouseConvert>, 
>>>   reverse_conversion_proc = 0, set_device_valuators = 0, fd = -1, atom = 0, 
>>>   dev = 0x0, private = 0x8be180, private_flags = 0, first = 0, last = 0, 
>>>   old_x = 0, old_y = 0, type_name = 0x2b61f0cedd55 "MOUSE", 
>>>   always_core_feedback = 0x0, conf_idev = 0x7f4e90, drv = 0x808070, 
>>>   module = 0x8be0a0, options = 0x8bda80, history_size = 256}
>>>
>>> Alan, can you at least reproduce this problem?
>> I'll try and get to this asap.
> 
> Could you try this on a x86_64 system? It doesn't seem to happen on a
> 32bit x86 system.

I should be able to play on my x86_64 this weekend assuming I get time
away from DIY :)

Col