PCI rework

[Benjamin Herrenschmidt]

> But no, I've seen a few PCI devices that have "normal" memory BAR's in
> nonstandard locations, i.e. at offset 0xe0 or 0xf0 in PCI config
> space.  The libpciaccess interface can only deal with the standard
> BARs at offsets 0x10 through 0x28.

Those devices vendors should be shot :) Seriously, even the kernel will
not deal with that properly, and it will just plain wreck havok. I don't
think we need to even bother supporting these as "PCI" devices. They
don't follow the PCI standard, let's them have their own hacks to map
their stuff and break and every second platform.

> Anothor major flaw in the libpciaccess interface is that it doesn't
> allow the caller to specify a size for the mapping.  This is
> important, since a video driver might want to map only part of a
> region.

Agreed. We want size & offset.

We also need (maybe there is, I haven't looked closely enough) routines
for accessing/mapping the VGA IO and MEM segments on a given PCI bus.
They might be different per domain or they may just not be accessible at
all on a given bus/domain). The driver needs to know that. That's also
where VGA arbitration kicks in and I'm not sure libpciaccess currently
handles that correctly, but again, I haven't had time to read/understand
it all.

Because of VGA arbitration, some drivers will "lose" the MMIO/IO enable
bits in their config space, potentially blowing up if, for example, they
have a DRI driver running which might be taking interrupts at any time.
I think drivers should be able to inform the PCI layer that they are
disabling their legacy access (and do so) so that they get out of the
normal arbitration and don't get their MMIO access disabled. That should
be done separately for MMIO and IO (since some drivers just don't care
about VGA memory once switched to native mode but might still tap legacy
IOs, though they also don't care about IOs when handling interrupts).

It's a complicated process (see my attempts at writing a kernel based
arbiter a while ago).

> If the firmware properly initialized things, PCI config does give you
> enough information.  X should only be looking at the PCI config space
> for the graphics card.  It certainly shouldn't need to walk PCI
> bridges and assigning memory addresses.

Indeed. The problem is that converting from a config space BAR value to
something you can "map" is not a service X can provide. You can't just
take the value and pass it to mmap on /dev/mem for example :) This is
platform and operating system specific and requires platform knowledge
that we don't want in X. Linux provides a per-BAR interface that we
should use when running on linux for example. It also provides a
separate interface to access the legacy areas.

In addition, we might want to try carrying around some access flags as
well (cacheability, write combining etc... hints) even if we don't
actually implement them in the back end right now. The old /proc/
bus/pci interface in linux had an ioctl to enable write combining on
some platforms prior to doing a mapping. The sysfs interface doesn't
have an equivalent at this point though there have been various talks
about how we could replace it.
 
> You're probably right.  But you releaze that this means dropping
> support for operating systems that don't fix things up right.  There's
> no way X can fix things up for you.  The same thing goes for new
> hardware that the OS doesn't know how to fix up yet.

No. That support would simply move into the libpciaccess backend for
that operating system.

Ben.