RFC - GLX Extension to control GLXVND dispatching for PRIME GPU offloading
Andy Ritger
aritger at nvidia.com
Tue Apr 30 23:37:48 UTC 2019
Thanks, Kyle.
For a little more context for others on xorg-devel: NVIDIA's GLX
implementation sends GLX protocol even when direct rendering, to
coordinate between client-side and server-side components. For PRIME
render offload, we'd like those GLX protocol requests to get routed to
NVIDIA's GLX implementation, regardless of the X driver of the X screen.
I'm curious what others think about a few general questions:
(1) Does it seem reasonable to update GLXVND within the X server
to be able to dispatch to different vendors on a per-client basis?
An alternative could be for NVIDIA's implementation to just tunnel
its GLX requests through private protocol. But, dispatching in GLXVND
seems cleaner to me.
(2) Does it seem reasonable to add a GLX extension to control this?
Even though Mesa doesn't need this, we thought it would be cleanest
to control through an extension. Though, maybe it is odd to add a
GLX extension intended to only be used by GLX client-side libraries?
(Or maybe that is no different than implementation-specific X extensions
like DRI3, Present, or NV-GLX).
I suppose an alternative to a full-blown GLX extension would just
be an exported function within the server, such that server-side GLX
implementations could update GLXVND's dispatching per-client, if needed.
Anyway, I know it can be hard to get excited about GLX in 2019, but it
would be nice to hear if others think GLX_EXT_server_vendor_select is
a sane direction.
Thanks,
- Andy
On Wed, Apr 24, 2019 at 09:26:56AM -0600, Kyle Brenneman wrote:
> On 4/23/19 4:28 PM, Aaron Plattner wrote:
> > On 4/17/19 8:51 AM, Kyle Brenneman wrote:
> > > For GPU offloading in libglvnd, where individual clients can run
> > > with an alternate GPU and client-side vendor library, we'd need some
> > > way for that alternate vendor library to communicate with its
> > > server-side counterpart. Normally, the server's GLXVND layer would
> > > dispatch any GLX requests to whichever driver is running an X
> > > screen. This is a GLX extension that allows a client to tell the
> > > server to send GLX requests to a different driver instead.
> > >
> > > The basic idea is that the server keeps a separate (screen ->
> > > GLXServerVendor) mapping for each client. The current global mapping
> > > is used as the default for each new client, but the client can send
> > > a request to change its own mapping. That way, if the client uses a
> > > different vendor library, then the client-side vendor can arrange
> > > for any GLX requests to go to the matching server-side driver.
> > >
> > > The extension uses Atoms as an ID to identify each GLXServerVendor,
> > > using a string provided by the driver. That way, the client-side
> > > driver can know which Atom it needs to use without having to define
> > > an extra query. The client can send a request with a screen number
> > > and a vendor ID to tell the server to dispatch any GLX requests for
> > > that screen to the specified vendor. A client can also send None as
> > > a vendor ID to revert to whatever GLXServerVendor would handle that
> > > screen by default.
> > >
> > > I also added a GLXVendorPrivate/GLXVendorPrivateWithReply-style
> > > request, which sends a request to a specific vendor based on a
> > > vendor ID, without having to worry about which vendor is assigned to
> > > a screen at the moment. Strictly speaking, a vendor library could
> > > get the same result by adding a regular GLXVendorPrivate request,
> > > and providing a dispatch function that always routes the request to
> > > itself, but that seems like it's more of an implementation detail of
> > > GLXVND.
> > >
> > > Also, this extension doesn't define any errors or queries to check
> > > whether a GLXServerVendor can support a given screen. These requests
> > > would be sent by a client-side vendor library (not by libglvnd or an
> > > application), so each driver would be responsible for figuring out
> > > on its own which screens it can support.
> > >
> > > Anyway, I've got a draft of the extension spec here, and I've
> > > written up a series of patches for the X server to implement it
> > > here:
> > > https://gitlab.freedesktop.org/kbrenneman/xserver/tree/GLX_EXT_server_vendor_select
> >
> >
> > Hi Kyle,
> >
> > Have you gotten any feedback on the commits there? It might help to
> > create an xorg/xserver merge request for them. You can use the "WIP:"
> > prefix on the MR title if you just want to request feedback without
> > actually getting it merged.
> Not a bad idea. I've posted a merge request with the patches here, and
> attached the extension spec:
> https://gitlab.freedesktop.org/xorg/xserver/merge_requests/179
>
> -Kyle
>
> > > Comments and questions welcome.
> > >
> > > -Kyle Brenneman
> > >
> > >
> > > Name
> > >
> > > EXT_server_vendor_select
> > >
> > > Name Strings
> > >
> > > GLX_EXT_server_vendor_select
> > >
> > > Contact
> > >
> > > Kyle Brenneman, NVIDIA, kbrenneman at nvidia.com
> > >
> > > Contributors
> > >
> > > Kyle Brenneman
> > >
> > > Status
> > >
> > > XXX - Not complete yet!!!
> > >
> > > Version
> > >
> > > Last Modified Date: April 11, 2019
> > > Revision: 1
> > >
> > > Number
> > >
> > > OpenGL Extension #???
> > >
> > > Dependencies
> > >
> > > GLX version 1.2 is required.
> > >
> > > This specification is written against the wording of the GLX 1.3
> > > Protocol Encoding Specification.
> > >
> > > Overview
> > >
> > > In multi-GPU systems, a client may decide at runtime which device
> > > and driver to use for GLX, for example to choose between a
> > > high-performance and low-power device.
> > >
> > > This extension defines a set of requests that allow a client to
> > > specify which server-side driver should handle GLX requests
> > > from the
> > > sending client for a particular screen.
> > >
> > > IP Status
> > >
> > > No known IP claims.
> > >
> > > New Procedures and Functions
> > >
> > > None
> > >
> > > New Tokens
> > >
> > > None
> > >
> > > Additions to the GLX Specification
> > >
> > > None. These requests are intended to be used by a client-side GLX
> > > implementation, not by an application. Therefore, this extension
> > > does not define any new functions or changes to the GLX
> > > specification.
> > >
> > > GLX Protocol
> > >
> > > Get a List of Server-Side Drivers
> > >
> > > Name: glXQueryServerVendorIDsEXT
> > >
> > > Description:
> > > This request fetches a list of available server-side
> > > drivers, and the current vendor ID selected for each
> > > screen.
> > > Each driver is identified by an Atom, with a string chosen
> > > by the driver.
> > >
> > > The reply contains a list of the currently selected vendors
> > > first, with one Atom for each screen. This will be the
> > > vendor selected with the glXSelectScreenServerVendorIDEXT
> > > request, or the default vendor if the client has not sent a
> > > glXSelectScreenServerVendorIDEXT request for a screen.
> > >
> > > If a screen is using the default vendor, and the vendor
> > > does
> > > not have a vendor ID, then the corresponding Atom in the
> > > reply will be None.
> > >
> > > After the currently selected vendors, the reply will
> > > contain
> > > a list of all available vendor ID's.
> > >
> > > Note that the list of available vendors is global, not
> > > per-screen. The client-side driver is responsible for
> > > determining which screens it can support.
> > >
> > > Encoding:
> > > 1 CARD8 opcode (X assigned)
> > > 1 17 GLX opcode (glXVendorPrivateWithReply)
> > > 2 3 request length
> > > 4 1417 vendor-specific opcode
> > > 4 unused
> > > =>
> > > 1 1 reply
> > > 1 unused
> > > 2 CARD16 sequence number
> > > 4 n+s reply length
> > > 4 CARD32 num_vendors
> > > 20 unused
> > > 4*s LISTofATOM current_vendor_ids (s =
> > > ScreenCount(dpy))
> > > 4*n LISTofATOM vendor_ids
> > >
> > > Select a Server-Side Driver For a Screen
> > >
> > > Name: glXSelectScreenServerVendorIDEXT
> > >
> > > Errors: BadValue
> > >
> > > Description:
> > > This request specifies that GLX requests for a screen
> > > should
> > > be dispatched to a particular server-side vendor.
> > >
> > > This only applies to requests from the client that
> > > sends the
> > > glXSelectScreenServerVendorIDEXT request. Requests from
> > > other clients are unaffected.
> > >
> > > As a special case, if vendorID is None, then the server
> > > will
> > > revert to the default vendor for a screen. This applies
> > > even
> > > if the screen's default vendor does not have a vendorID.
> > >
> > > A BadValue error is generated if vendorID is not None or a
> > > valid vendor ID.
> > >
> > > Encoding:
> > > 1 CARD8 opcode (X assigned)
> > > 1 16 GLX opcode (glXVendorPrivate)
> > > 2 5 request length
> > > 4 1418 vendor-specific opcode
> > > (glXSelectScreenServerVendorIDEXT)
> > > 4 unused
> > > 4 CARD32 screen
> > > 4 ATOM vendorID
> > >
> > > Vendor-Specific Private Request
> > > Name: glXNamedVendorPrivateEXT
> > >
> > > Errors: BadValue
> > >
> > > Description:
> > > This request sends a vendor-specific command to a vendor
> > > based on its vendor ID. The named vendor need not be
> > > assigned to any screen.
> > >
> > > A BadValue error is generated if vendorID is not a valid
> > > vendor ID. GLX vendors may also generate other errors.
> > >
> > > Encoding:
> > > 1 CARD8 opcode (X assigned)
> > > 1 16 GLX opcode (glXVendorPrivate)
> > > 2 4+(n+p)/4 request length
> > > 4 1419 vendor-specific opcode
> > > (glXNamedVendorPrivateEXT)
> > > 4 unused
> > > 4 ATOM vendorID
> > > n LISTofBYTE vendor-specific data
> > > p unused, p=pad(n)
> > >
> > > Vendor-Specific Private Request with Reply
> > > Name: glXNamedVendorPrivateWithReplyEXT
> > >
> > > Errors: BadValue
> > >
> > > Description:
> > > This request is identical to glXNamedVendorPrivateEXT,
> > > except that it generates a reply.
> > >
> > > Note that the request is identical except for using the GLX
> > > opcode glXVendorPrivateWithReply.
> > >
> > > Encoding:
> > > 1 CARD8 opcode (X assigned)
> > > 1 17 GLX opcode (glXVendorPrivateWithReply)
> > > 2 4+(n+p)/4 request length
> > > 4 1419 vendor-specific opcode
> > > (glXNamedVendorPrivateEXT)
> > > 4 unused
> > > 4 ATOM vendorID
> > > n LISTofBYTE vendor-specific data
> > > p unused, p=pad(n)
> > > =>
> > > 1 1 reply
> > > 1 unused
> > > 2 CARD16 sequence number
> > > 4 n reply length
> > > 24 LISTofBYTE returned data
> > > 4*n LISTofBYTE more returned data
> > >
> > > Errors
> > >
> > > None
> > >
> > > Issues
> > >
> > > 1) Should server-side drivers be identified by Atoms or XIDs?
> > >
> > > PROPOSED: Use an Atom. Using an XID seems somewhat cleaner, but
> > > it would require an additional query for the client to know
> > > which XID corresponded to which server-side driver.
> > >
> > > Using an Atom, each driver can simply define the string to
> > > use a
> > > priori.
> > >
> > > 2) Should glXQueryServerVendorIDsEXT send back the currently
> > > selected vendor ID for each screen?
> > >
> > > PROPOSED: Yes. Being able to query the current screens might be
> > > useful for a client to restore the previous assignments in case
> > > of an initialization failure.
> > >
> > > 3) Are the glXNamedVendorPrivateEXT requests needed?
> > >
> > > PROPOSED: Yes. The two glXNamedVendorPrivateEXT requests
> > > provide
> > > a well-defined communication channel between the client and
> > > server sides of a driver.
> > >
> > > Sending a request to a specific driver would still be possible
> > > without these requests, but would require either defining a
> > > separate X11 extension or relying on implementation details of
> > > GLXVND.
> > >
> > > Revision History
> > >
> > > 1. 11 April 2019
> > > - Initial draft.
> > >
>
More information about the xorg-devel
mailing list