Tech-Proposal for checking and byteswapping ( Re: RFC: server-side XCB )
Christian Linhart
chris at DemoRecorder.com
Sun Dec 21 01:10:50 PST 2014
Overall Task:
Implement byte-swapping and size-checking in the X-Server by using
generated code based on XCB protocol definitions.
1. Write a new code-generator which creates byte-swapping and
size-checking code.
(Byte-swapping and size-checking can both be done with the same generator.)
2. The generator and helper-code will be in the Xserver source code tree.
Do not create an extra library for that so we can avoid the topics
of API and ABI compatibility.
3. Integrate the code into the X-Server.
4. Fix XCB protocol definitions.
ad 1. Code-generator:
=====================
The code-generator will be implemented in Python.
It will use the parser from xcb/proto.
The code-generator from xcb/libxcb is also based on the parser
from xcb/proto.
Essentially the generator creates three types of functions for each
xcb-object:
a) xxxx_SwapFromClient :
Swaps client byte-order to server byte-order, and does length-check
for everything, including the variable part.
Possibly this can later include some other consistency checks.
(like enum-values, mask-values, etc)
b) xxxx_SwapToClient :
Swaps server byte-order to client byte-order, and does checking as
mentioned for a).
c) xxxx_Check :
Just performs the checking but does not do any byte-swapping.
FAQ: Wouldn't it be better to have the byteswapping and checking functionality
separate?
Answer: No, here are the reasons:
* Performance:
Swapping needs to interpret lengths and iterate over lists.
Length checking essentially also has to do that. So when checking
and swapping is done separately, it will have do do many things twice.
* It's generated code.
Therefore, code-duplication is OK because code-duplication in generated
code is not a software maintenance issue.
* If these were separate, the checking would have to be done first.
Otherwise swapping might change memory after the end of request.
( e.g., wrong value of a length-field or mask ) So the checking would
have to work in client byte-order. This is, e.g., relevant for
length-fields and masks which determine the layout of subsequent data.
Therefore a standalone checking-code will already do most of the work
of byte-swapping. So it better does the full byte-swapping right away.
ad 2. Where to put the generator:
=================================
We need to determine which directory it should be in.
Probably a new top-level directory in xorg/xserver, named appropriately,
e.g., "proto".
ad 3. Integration:
==================
* Requests:
Integration for requests is rather straightforward: in the dispatch
functions of the respective extension.
The switch-case statement or dispatch-table in those dispatch functions
should also be automatically generated.
Requests are most important thing to support because proper
checking and byteswapping of requests is important for security.
Start this integration with a simple extension, so the generator need
not yet support all features at first.
The move to more complex extensions.
Of course the design already has to factor in the most complex
extensions. XInput and XKB seem to be the most complex ones.
I already have experience with this from my commercial project
where an XCB-based code-generator already handles all of XInput and XKB
for several protocol transformation tasks.
* Replies, Events, Errors:
This integration is more complicated.
This integration is only relevant for byteswapping.
Start this only when we are finished with requests for all extensions.
So, possibly, this will be beyond the scope of the internships.
Therefore, no more details provided here for now.
(One possibility would be to hook into "WriteToClient"
and parse and swap the complete outgoing protocol in the case of
a swapped client.)
ad 4. Fix XCB protocol definitions.
===================================
There are still quite some bugs and missing pieces in the XCB protocol
definitions of several extensions, judging from some random checking.
Server-Side XCB needs 100% coverage of all requests, replies, etc,
and should be bug-free. Otherwise we'd generate more problems
in the X-Server than we solve with this.
For some extensions, the generator must be enhanced to be able
to describe their protocol. I have already implemented several
enhancements of the generator, so that XInput can be supported.
State of fixing XCB protocol definitions:
* I handle the XInput and XKeyboard extensions because I have already
finished this, but not yet posted everything.
* For other extensions, we need help with reviewing and fixing the
current state of the protocol-definitions in XCB.
( going through all protocol definitions is beyond the scope of these
internships )
Also we need to make the padding explicit in the protocol definitions,
and fix the current problems with the implicit padding code.
There was consensus on the XCB list to make padding explicit,
and then to remove the implicit padding code.
***
What do you think about this proposal?
Chris
On 12/21/14 09:36, Christian Linhart wrote:
> There is already a project underway to do this.
>
> We would have sent the technical proposal in a few days.
> I will send the technical proposal as an extra message, soon.
>
> In any case we should avoid that two teams are working on that independently.
>
> We have two interns at Xorg who are working on this under my mentorship:
> * Asalle works as intern under the OPW-program ( http://www.x.org/wiki/XorgOPW/ )
> She will do the length checking and byte swapping.
>
> * Jaya is an intern without any formal program behind it.
> She will implement server-side XCB in requests, replys etc,
> i.e. will replace the usage of the manually written proto-headers
> with using XCB.
>
> Currently, both are setting up their work environment and getting acquainted with the code.
> Both already have gained some experience with XCB during their qualification task etc.
>
> Me, I have 17 years experience with the X-Server source-code, especially with protocol handling,
> from work in a commercial project.
>
> I have recently volunteered to become the maintainer of the XCB project.
>
> And I do understand the code of the XCB codegenerator. ;-)
> I have already made several changes to it, and more are in the pipeline.
>
> I also have made lots of changes of the XCB protocol definition of XInput, with the goal of 100% coverage.
> I will also complete the protocol definitions of the XInput and XKeyboard extensions'along with further changes in the XCB code generator.
> (actually already finished with this. Just need to prepare this for posting on the XCB list.)
>
> Cheers,
>
> Chris
>
>
> On 12/20/14 21:27, Jamey Sharp wrote:
>> We've talked about doing the xserver equivalent of XCB for years--that is, auto-generating the protocol serialization and deserialization code from XCB's machine-readable descriptions of the X protocol and extensions.
>>
>> Considering the recent CVEs in that code, I think it's time. So I want to collect folks' thoughts on what server-side XCB should look like.
>>
>> At a high level, which code should we be generating? Off the top of my head, I'm thinking the dispatch tables and all the swap procedures, as a first target in order to get the codegen infrastructure merged and tested. Then it'd be nice to generate code that validates arguments and returns appropriate errors. Thoughts?
>>
>> XCB's code generator is implemented in Python. Can we make the same choice in the xserver build process?
>>
>> Anything else I should be thinking about?
>> Jamey
>>
>>
>>
>> _______________________________________________
>> xorg-devel at lists.x.org: X.Org development
>> Archives: http://lists.x.org/archives/xorg-devel
>> Info: http://lists.x.org/mailman/listinfo/xorg-devel
> _______________________________________________
> xorg-devel at lists.x.org: X.Org development
> Archives: http://lists.x.org/archives/xorg-devel
> Info: http://lists.x.org/mailman/listinfo/xorg-devel
More information about the xorg-devel
mailing list