Multimedia support (Re: X Developers' Summit)

[Helge Bahmann]

> More details are on the wiki page:
> http://www.x.org/wiki/Events/XDS2007

Hello X developers,

I would like to get involved with X.org and its future in the area 
of "multimedia". Maybe the X developers' summit  would be a good occasion to 
meet and discuss ideas, if you have a slot to spare I would be happy to give 
a short talk on what I have done and what could be done in the future.

My interest is in extending the X Window System to provide support for 
network-transparent multimedia. To demonstrate that this is conceptually 
viable I have written three X extensions, providing: timing support, audio 
support, compressed media support. The extensions are minimalistic but 
sufficient to enable many multimedia applications, including for expamle

- network-transparent, robustly synchronized video/audio playback
- network-transparent video editing (cutting, effects) with real-time preview

Conceptually I would like the X server to provide

- timing support: Schedule X drawing (or other) operations to a later point in 
time (XSync in its present form is unfortunately insufficient)

- audio support: well, the title says it all. It includes playback, recording 
and simple processing (arthimetics, FIR filters, resampling). This is 
probably highly controversial, but basically I think that integrating audio 
at the X protocol level has significant advantages over a separate audio 
server (like MAS etc), especially when it comes to synchronization

- compressed media support: quite obvious as well, I would like to transfer 
compressed images and audio within the X protocol. Unlike XIE (X imaging 
extension) this should include support for nonintra-coded pictures

I have a proof-of-concept implementation that demonstrates all of the above 
can be done (will be relased by beginning/mid-August, still some code cleanup 
to do, including working demo applications), but lots of implementation 
issues remain with the core X server.

I would very much like to discuss if this form of multimedia support is a 
worthwhile thing to provide in the X server (or, rather, convince that it 
is :), and how I could proceed in integrating this support in X.org, so let 
me know what you think.

Best regards,
Helge Bahmann
-- 
Mathematicians stand on each other's shoulders while computer scientists stand 
on each other's toes.
-- Richard Hamming