Video overlay - Window updating

[Aaron Plattner]

On Mon, Mar 08, 2010 at 04:41:46AM -0800, Iban Rodriguez wrote:
>   Good Morning,
> 
>   I have a question about video overlay and window repainting. I have
> a project in which I need to show a video on a non-rect part of an
> application window. After some attempts I decided to use play the
> video on one window (window 1) and over it put the application window
> (window 2) where the zone I want the video to show is painted using
> the color key of the video overlay. With this configuration the video
> should be shown as if it were embedded in window 2. However, it works
> only in some situations, which I describe bellow:
> 
>   1.- If window 1 (video) is completely covered by window 2, the video
> doesn't show.
>   2.- If window 1 (video) is not completely covered by window 2 (app)
> by the left, the video is only show on the part that is not covered.
>  3.- If window 1 (video) is not completely covered by window 2 (app)
> by the bottom, the video is only show on the part that is not covered.
>  4.- If window 1 (video) is not completely covered by window 2 (app)
> by the bottom and by the left simultaneously, the video is shown on
> the parts not covered and also on the parts of window 2 (app) which
> are painted using the color key of the video overlay.

>  For my project, I need to reproduce the behaviour of case 4 in the
> situation of case 1 but I don't know how to do it. I have tried it
> with and without a window manager with the same result. I don't know
> very much about the X server but the problem seems to be that when it
> needs to update a window, it repaints the minimum rectangle which
> cover all pixels that need to be updated so it only shows the video in
> case 4 where the minimum rectangle is the entire window 1 (because of
> the parts not covered). So my question is, is there a solution for
> this problem? Can I tell the X server that some windows must be always
> completely repainted? Is there any other way for managing the video
> overlay that avoids this problem?

The problem is that this is not a valid use of the overlay.  Among other
things, you can't assume that the driver really uses a hardware overlay or
just fakes it.  I suspect what's happening here is that the server is
clipping the rendering against the occluding window, and simply skips it
for the parts that it thinks are not visible.  The X server doesn't know
that the hardware would let the video show through into parts of the
occluding window.

If you want to overlay stuff on top of the video, you'll need to either
render it into the original video stream before sending it to Xv or use
something that explicitly supports sub-pictures, like VDPAU or XvMC.

-- Aaron