Tech-Proposal for Server-side XCB impl of requests, replys, events etc ( Re: RFC: server-side XCB )

Christian Linhart chris at DemoRecorder.com
Sun Dec 21 02:58:06 PST 2014 

Overall task:
Make the server-side implementation of requests, events etc use
XCB-generated code.
(Instead of using the data structures from manually written proto-headers,
and instead of manual interpretation of the protocol in Proc functions.)

Overall strategy:
1. Use the generated code of libxcb and enhance the code-generator
   there as needed.

2. Server-side access to a data of request will be done in a similar way
   as the client-side accesses the data of a reply.
   Rationale: Use the same strategy for received data on both sides.

3. Server-side generation of replies, events and errors will be done
   in a similar way as the clients-side generates requests.
   Rationale: Use the same strategy for sending data both sides.


ad 1. Use libxcb
================

  Most of the code of libxcb can be used as-is.

  We just need to add support for the other direction of
  data-flow for requests, replies, events and errors.
  (structs can already handle both directions)

  That is, everything will become more symmetric
  because then, all protocol elements will have support
  for serialization and deserialization.


ad 2. Incoming data, i.e., Requests
===================================

  The client-side of XCB handles receiving data by essentially
  providing a pointer to raw protocol data.
  The pointer-type is such that it points to a struct
  where the fixed part of the protocol data can be directly
  accessed as struct members.

  All other data has be to accesses through deserialization
  functions or iterators.

  We can do the same for requests on the server side.

  The advantages are:
  * we already have a pointer to raw protocol data in
    all Proc-functions.
    So this requires only minimal refactoring.

  * This provides high performance because for the fixed part of a request
    no computation is needed and no data needs to be copied
    for deserialization.

  * We already have support for that in the generator of libxcb.
    Maybe we need to enable generation of some of the stuff for requests
    that's currently only generated for replies, events etc.
    This is probably a rather simple change in the generator
    because requests and replies have the same Python base class
    and only differ minimally fom each other.

  We may want to do some changes for type-safety, such
  as changing the parameter list of Proc functions from
  using a generic request-pointer to using the specific
  pointer-type of the XCB-struct. Therefore eliminating
  the need of the typecast that's hidden in the REQUEST macro.

  Also, for the var-sized part of requests, we may want to pass
  XCB-iterators and deserialized data to the Proc functions
  instead of requiring them to call the deserialization stuff
  themselves correctly.

  However we need to be careful with changing the parameter
  list of Proc functions because some of them are accessed
  via a function pointer, and there may be code in DDXes or
  extensions or somewhere else, which redirects these
  function pointers. So changing the parameter list of
  Proc functions can open a can of worms.

  So I am not sure whether we really should change the
  parameter list of Proc functions.

ad 3. Outgoing data such as replies, events, errors
===================================================

  This will be handled like requests are handled on the client-side.

  Instead of "xcb_connection_t*" we need some abstraction of ClientPtr,
  of course.

  We cannot use ClientPtr directly because we do not want to expose
  ClientPtr outside of the server.

  In any case, the abstraction of ClientPtr can be pretty simple
  because all we need is access to the function "WriteToClient"
  and a generic pointer which points to the ClientRec.

  So, the abstraction can be combination of
  a void pointer and a function-pointer.
  Both as members of a struct, so this is extensible
  and does not clutter parameter lists.
 
  Such as:
    typedef struct{   
        void* sender;
        int (*send)(void* sender, int count, const void* buf );
    } xcb_protocol_sender_t;

  So, e.g., the function for sending the reply of QueryTree
  can look somehow as follows:

    int
    xcb_send_query_tree_reply(
        xcb_protocol_sender_t *sender,
        uint16_t sequence,
        xcb_window_t root,
        xcb_window_t parent.
        uint16_t children_len,
        const xcb_window_t *children
    );

***

What do you think about this proposal?

Chris

P.S.: For this task we also need to fix problems and bugs
in XCB protocol definitions.
See my proposal for "checking and byte-swapping" for details.


On 12/21/14 09:36, Christian Linhart wrote:
> There is already a project underway to do this.
>
> We would have sent the technical proposal in a few days.
> I will send the technical proposal as an extra message, soon.
>
> In any case we should avoid that two teams are working on that independently.
>
> We have two interns at Xorg who are working on this under my mentorship:
> * Asalle works as intern under the OPW-program ( http://www.x.org/wiki/XorgOPW/ )
>   She will do the length checking and byte swapping.
>
> * Jaya is an intern without any formal program behind it.
>   She will implement server-side XCB in requests, replys etc,
>   i.e. will replace the usage of the manually written proto-headers
>   with using XCB.
>
> Currently, both are setting up their work environment and getting acquainted with the code.
> Both already have gained some experience with XCB during their qualification task etc.
>
> Me, I have 17 years experience with the X-Server source-code, especially with protocol handling,
> from work in a commercial project.
>
> I have recently volunteered to become the maintainer of the XCB project.
>
> And I do understand the code of the XCB codegenerator. ;-)
> I have already made several changes to it, and more are in the pipeline.
>
> I also have made lots of changes of the XCB protocol definition of XInput, with the goal of 100% coverage.
> I will also complete the protocol definitions of the XInput and XKeyboard extensions'along with further changes in the XCB code generator.
> (actually already finished with this. Just need to prepare this for posting on the XCB list.)
>
> Cheers,
>
> Chris
>
>
> On 12/20/14 21:27, Jamey Sharp wrote:
>> We've talked about doing the xserver equivalent of XCB for years--that is, auto-generating the protocol serialization and deserialization code from XCB's machine-readable descriptions of the X protocol and extensions.
>>
>> Considering the recent CVEs in that code, I think it's time. So I want to collect folks' thoughts on what server-side XCB should look like.
>>
>> At a high level, which code should we be generating? Off the top of my head, I'm thinking the dispatch tables and all the swap procedures, as a first target in order to get the codegen infrastructure merged and tested. Then it'd be nice to generate code that validates arguments and returns appropriate errors. Thoughts?
>>
>> XCB's code generator is implemented in Python. Can we make the same choice in the xserver build process?
>>
>> Anything else I should be thinking about?
>> Jamey
>>
>>
>>
>> _______________________________________________
>> xorg-devel at lists.x.org: X.Org development
>> Archives: http://lists.x.org/archives/xorg-devel
>> Info: http://lists.x.org/mailman/listinfo/xorg-devel
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: http://lists.x.org/mailman/listinfo/xorg-devel

More information about the xorg-devel mailing list