VGA arbitration: API proposal

[Egbert Eich]

Benjamin Herrenschmidt writes:
 > > X uses an arbiter like this. We distinguish between different states:
 > > decoding_disabled, unused, used. Furthermore we distinguish between
 > > 'setup' and 'operating' state as some chips may need legacy resources
 > > for setup while during normal operation they may not be required.
 > > Also we treat 'memory' and 'io' resources separately.
 > 
 > Yes, I want to treat memory and io separately too. I think I don't need
 > to make a difference between setup and operating. As I wrote below, a
 > card that indicates that it doesn't do legacy decoding (safe) can still
 > use vga_get/put for a 'temporary' lock on the vga space, typically for
 > things like int10

We did this distiction to keep switching cycles low. It doesn't make
sense in the general arbiter context but maybe it would make sense
if driver could register required resource with a finer granularity
than just saying 'we need legacy' to minimize the number situations 
where PCI cards need to be fully disabled.

 > 
 > > Based on this we decide which devices can stay enabled and which 
 > > driver instances need to call the arbiter before they are allowed to
 > > access resources.
 > 
 > Yup. One thing I want to avoid tho is callbacks to drivers, since it
 > becomes very difficult for a kernel based arbiter. So I think a driver
 > that calls vga_set_legacy_decoding() indicating it has set the card not
 > to decode legacy space can then safely not call the arbiter anymore
 > unless it needs temporary VGA lock for int10.

Right. A driver that 'knows' that it doesn't need any resources that
may be used by someone else could do this.
When we implemented this in X we originally even allowed PCI resources
to conflict (in case we run out of available resources) but abandoned 
this idea at one point.

 > > I would like to come up with a solution that works with every card.
 > > The big breakthru of multihead setups came when we had this resource 
 > > manager (arbiter) in X which that was able to handle whatever cards
 > > were available on the market.
 > 
 > I would keep the arbiter API simple, and have the knowledge of the card
 > specific up to the card's driver tho. That is by default assume a card
 > does full legacy decoding unless the card's specific driver informs us
 > that this is not the case (typically because the driver disabled it on
 > the card).

We need to see what the tradeoff actually would be.
In X we have a simple way to register resources which does not create much
driver overhead at all. Macros for most standard legacy address ranges are 
available.
The driver API is really simple.
 > > If we want to allow more than one instance (process) to access this we 
 > > may have to let this wait here until another instance has given up
 > > the resource. Currently we don't have to do this in X as everything 
 > > happens in a single thread.
 > 
 > Yes. I'm thinking in breaking this into a couple of versions in fact, a
 > blocking one and a non-blocking one (that just fails). Eventually a
 > blocking one with a timeout too.

OK, what do we do when a timeout occurs?

 > >  > 
 > >  >  - vga_put(cookie);
 > >  > 
 > >  >    Release.
 > >  > 
 > >  > I don't think we need a callback to "clients" to inform them that they
 > >  > are losing VGA access, this would be to complicated to deal with as
 > >  > client can be separate processes and/or kernel land.
 > > 
 > > How about the interrupt case?
 > 
 > Explained later in the email. There is no way to deal cleanly with
 > interrupts other than only allowing them on a card that has permanent
 > access to resources, unless we want to introduce a callback mecanism for
 > "lost ownership". Possible, but complexifies the API. Do we really need
 > that ?

I certainly would like to avoid this. You would have to be able to 
preempt the user space application at any point.

 > >  > Typically, a driver could only enable IRQ generation on the card in on
 > >  > those circumstances:
 > >  > 
 > >  >  - It does have VGA decoding enabled (it called vga_set_legacy_decoding
 > >  > with no IO and no MEM decoding set). It basically puts the card out of
 > >  > the arbitration domain.
 > > 
 > > Quite restrictive....
 > 
 > Yes, but is this a problem in practice ? We mostly need interrupts with
 > modern cards that do DRI. Can't they all disable legacy VGA decoding ?

I hope so.

 > 
 > If not, I'll have to come up with a callback based mecanism, so that the
 > current "owner" gets notified that it's about to lose it, so it can
 > disable interrupt generation on the card. I do _not_ intend to have an
 > API that allows switching the owner at interrupt time, that would be a
 > nightmare. If a driver really don't want to disable interrupts, it can
 > go the dirty way, and if it takes an interrupt while not the owner, just
 > call disable_irq() on it, trigger an asynchronous event (work queue)
 > and, from there, in process context, do the vga_get() to re-acquire
 > ownership and re-enable the IRQ. This will add latency to the interrupt
 > but I expect it to happen quite rarely.

OK, that would be a way for legacy cards to deal with this.
The most common source of interrupts there are v-retraces.
They would happen less 100 times per second on these cards.
would this qualify for 'quite rarely'?
 > 
 > I'm no fan of this solution tho, but if it's considered acceptable, then
 > the only API addition needed is a vga_tryget() that is callable at irq
 > time, that doesn't block and basically returns true if we still own the
 > VGA, or even eventually proceeds to the switch if nobody else has the
 > VGA semaphore held.
 > 
 > Such a tryget mecanism is needed anyway if we want to keep vgacon
 > ability to do printk at interrupt time.

Right!
 > 
 > > Another set of leagacy resources are the 8514 registers which are 
 > > employed by the Mach64 driver (at least Mach64 decodes them).
 > > 8514 resources are a PITA as they are sparse and scattered
 > > accross PIO space.
 > > We've got all the code to handle that it would not be too difficult
 > > to extend it to general resource registration.
 > 
 > Hrm... I have to think about it. I could split, in my API, "IO" into
 > several classes of IOs, but i'd rather avoid adding too much complexity,
 > even if that ends up beeing at the expense of a little bit of perfs on
 > old card when multiple of them are used simultaneously.

It is actually not so bad. The current complexity in X determining
if a resource conflicts arises from the fact that we have an incomplete
knowledge about some resources and need to determine conflicts depending
on the situation we are in. The rest of the arbiter code is really
not that complicated.

Cheers,
	Egbert.