Bad DMA from CEDAR card

[Benjamin Herrenschmidt]

On Tue, 2012-10-23 at 09:34 -0400, Alex Deucher wrote:
> >
> > I'll do some more tests tomorrow to confirm whether that is the problem
> > or not at which point, if it is, we'll need some kind of quirk to
> > indicate that it supports only MSI32 and not MSI64 or something along
> > those lines. Guys, go shoot your HW engineers please.
> >
> 
> Well, we only support a 40 bit DMA mask, so I suspect MSIs are limited
> to 40 bits as well.

Well, there is a big difference here... limiting yourself to
configurations up to 1TB of RAM for the purpose of DMA is somewhat
understandable (ignoring that in NUMA setups etc... memory might
actually cover more address space than the amount of RAM etc...) but
MSI's are a whole different thing.

They do not hit memory, they are decoded as special addresses by the
bridge , and can potentially be anywhere.

Some of our bridges only support MSIs high up in 64-bit space for
example.

Anyway, I will double check today whether that is indeed the problem I'm
hitting though from what I can tell from here, it looks like it.

Cheers,
Ben.