PCI rework

[Dave Airlie]

> > On Sunday, April 30, 2006 5:22 am, Mark Kettenis wrote:
> > > Matthieu Herb, asked me to have a look at libpciaccess some time ago.
> > > My conclusion then was that it is too tightly bound to Linux.  It
> > > takes the Linux sysfs pci access interface as a model instead of the
> > > actual pci hardware.  I especially feel the support to map memory
> > > associated with the PCI BAR's does not belong in the library.  It is
> > > not impossible to implement the libpciaccess backend for other
> > > operating systems, but it would be rather awkward.
> >

Well I've been taking the time recently to try and salvage the current
X PCI interfaces and really its a quite large unholy mess.. and I've
just added to it by fixing the ROMs for Linux....

> > X already assumes it can mmap memory BARs, do you think it needs to
> > change too?  It seems to me that the interface provided by libpciaccess
> > is fairly flexible, easy to use, and easily ported to new platforms (it
> > should be, it's based on the Linux internal PCI interfaces which have
> > prooven highly portable).  Do you have other examples where you think
> > Linux specific implementation details have crept into the interface?
>
> I'm talking specifically about pci_device_map_region() and
> pci_device_unmap_region().  These interfaces are clearly there because
> the Linux sysfs provides these "regions" as files which you can open
> and then mmap.  Other operating systems don't support this view, and
> there's a good reason not to do it: Many PCI devices put mappable
> memory addresses in config space outside the standard PCI BARs.  What
> you really need is an interface to read from PCI config space and an
> interface to map physicall addresses on the pci bus into memory.  Some
> standard helper functions to decode the standard BARs is certainly
> desirable.

Can you point me out some of those PCI devices, as I can't see how
they can possibly work with any OS that does PCI, because there would
be nothing stopping the OS from mapping something else in that area,
granted I can see times when we might want to map 0xa0000 or 0xc0000
areas for things, I've been trying to make it so I can mmap the
resource files from sysfs for Linux recently, to be honest the
interface should be dealing with BARs, PCI is about BARs, if there is
another sneaky interface for direct memory mapping so be it, but from
a device driver POV, it should be able to request a BAR resource
mapping, if the underlying code has to fallback to direct mapping
/dev/mem it should be easy, it can store the PCI space info to do the
conversion.. I started implementing this in os-support, before I found
out that os-support doesn't have enough info, common/xf86pciBus.c has
it..

Another reason the current code is horribly broken is it doesn't
respect the E820 maps, I don't want X looking at E820 maps at all, but
really it should if it is going to do think it can re-configure PCI
addresses, on Linux the kernel does this, my goal is to remove all
remapping code paths from X for Linux using systems, so it does
nothing but read the PCI spaces. The kernel is adding code to enable
devices from userspace so that X can do that first, which will sort
out all the BARs etc....

> >
> > Note that the prototype is different in that it takes a full pci info
> > structure rather than just a tag.  This gives arch specific
> > implementations more flexibility and eases porting.
>
> But a PCITAG is already opaque; there's no reason why you could extend
> it to include the additional information you might need.

It's not, passing domain info around in the TAG isn't correct, I don't
think there is enough bits to fill everything in a TAG correctly, I've
got a patch from last year that rips out all the TAGs and just adds
domain to the PciInfo...

> > > I really don't see any major flaws in the
> > > current xf86Pci.h interfaces, but I agree that some of the
> > > implementation could use some cleanup.
> >
> > I disagree with this, the xf86Pci interface is pretty screwy:
> >   o X does things with PCI devices it has no business doing (e.g.
> >     remapping BARs)
>
> Remapping as in writing different addresses into them?  X might need
> to do that if the firmware doesn't properly initialize them.  I've
> seen many, many buggy firmware implementations (ok they're mostly
> BIOSes) that don't do this properly.

Look on Linux the kernel does this, this is an OS'es job, if your OS
doesn't do it then implement it in the OS abstraction library which is
libpciaccess, X is not an OS. Also X is incapable currently of dealing
with PCI domains in any reasonable fashion all the code that deals
with domains is a big big hack, I enabled it on 32-bit x86 recently
and everything breaks. This shouldn't happen, the code should work on
all systems, if domains don't exist it should figure it out, it
doesn't it requires compile time switches, DRI is very broken with it,
(shall I go on???)


> >   o the distinction between mapping domain and regular PCI memory is
> >     arbitrary and should be removed
>
> I'm not quite sure what is actually meant by a "domain".  I'm
> presuming it's similar to what the ACPI specification calls a
> "segment": a completely seperate PCI bus hierarchy.  Yes, the way the
> current interface handles that is awkward, but it should be easy to
> fix this if you add the domain to the PCITAG.

Been done, I personally think we don't have enough bits to fit
potential new crazy ass machines with lots of buses...

>
> I'm not so sure about that.  Some amount of complexity will be needed
> to deal with badly designed or buggy hardware and firmware.  Most of
> these issues will be specific to particular PCI hardware.  Shoving
> those into the domain of a seperate library, or the further down into
> the operating system isn't a solution.
>
> >   o ROM mapping is hard to port and buggy in some cases
>
> By its very nature ROM's are unportable.  I really can't see how
> libpciaccess would alleviate that situation.  The X hardware drivers
> really should try to avoid depending on ROMs.

They have to read the ROMs to get the tables out of them, you can't do
most things any other way, the problem is X assumed it owned the PCI
bus, it doesn't, so it would remap ROMs and crash the system, I've
fixed that now with a proper sysfs ROM patch.

> > libpciaccess directly addresses some of the above problems and makes the
> > others much easier to fix.  And its fundamental direction is correct as
> > well:  X should *not* have a big PCI abstraction layer of its own, it
> > should use the underlying OS via an external library for PCI related
> > operations.
>
> There are not many applications that need a PCI abstraction layer.
> Apart from X and some debugging tools to look at PCI config space I
> can't really think of any applications that need them.  The debugging
> tools used on Linux, pciutils, already come with their own abstraction
> layer.  So libpciaccess was basically developed just for X, and is
> unlikely to be used for any other software packages.  I really don't
> see your point, especially since adapting them requires changes to
> almost all X drivers, even those that don't abuse the current PCI
> abstraction layer.

Well I used it recently in miniglx, and a new driver framework / EGL
will use it if we get around to it.

Dave.