RFC: Video decode acceleration API

[Zack Rusin]

On Tuesday 27 March 2007 17:34, Bian, Jonathan wrote:
> The current interface is focused on video decode only and is window
> system independent, so that potentially it can be used with graphics
> sub-systems other than X.  In a nutshell it is basically a scheme to
> pass various types of data buffers from the application to the GPU for
> decoding a compressed bit-stream.  This is very much a work-in-progress
> and the current draft only defines data structures for MPEG-2 and VC-1
> decode at the VLD (or slice level) entry-point.  I would like to get
> this out for critic and feedback before going too much further, and also
> hopefully make it a community collaborative effort.

Hey, it's great to see someone actually having the time to look at this.

I looked over your proposal. The API looks heavily OpenVG influenced and I'm 
assuming it was meant to be used with the windowing system bindings that you 
guys also use for your OpenGL ES implementation available for xscale. 
The interface, altough pretty nice, doesn't fit all to well with the rest of X 
extensions, we'd need to rework it a bit.

A few thoughts:
- in VAProfile I'd suggest addition a VAProfileCustom flag with some callback 
allowing clients to stream in other formats. i think that ideally instead of 
VA_RT_FORMAT defines we'd have vaQueryIsFormatSupport(VADisplay, int fourcc) 
vaQuerySupportFormats(VADisplay, int **fourccs, int *num); and maybe even 
vaQueryNative/PreferredFormat or such.
- either VAContext and vaCreateContext or VASurface or a new member or 
additional flag for the flags member would be nice to specify whether a video 
is actually fullscreen (knowledge that picture width == screen width and 
picture height == screen height isn't enough.) in those fullscreen cases we 
could use backened scalers, plus do some other magic so it'd be nice to have 
it explicit in the flags.
- i'm not sure how nicely the surface management section converts to X 
semantics. although i think that a whole new rendering surface might be 
better than additional formats in Xrender pictures.
- can the vacontext be created with a different width/height than vasurface? 
it would seem pretty awkward to me. if that's the case, than we should have 
xOffset/yOffset in there as well.
- in surfaces using fourcc for format would a lot more convenient than just 
420/422/44. most decoding engines i've seen operates with and passes fourcc 
around anyway it would make streaming between them a little smoother. going 
back to that point, imho it would be more consistent to just uniformly use 
fourcc across the framework. including creation of VAPictureParameterBuffer 
and VAIQMatrixBuffer. 

Zack