Making Composite better for interactive apps

[Giuseppe Bilotta]

On Tue, Feb 6, 2018 at 3:39 AM, Keith Packard <keithp at keithp.com> wrote:
> Giuseppe Bilotta <giuseppe.bilotta at gmail.com> writes:
>> I'm coming at this from the assumption that a Compositing Manager
>> would try to pump out frames as frequently as possible, in order to
>> minimize latency, which would lead to changes being disabled more
>> often than not.
>
> I think that's the wrong assumption. Given that you have monitors at a
> fixed refresh rate, you want each frame on the monitor to contain the
> most recent changes for all windows. That means painting the screen
> after all applications have updated for the current frame. Because we
> can't really tell when applications are done updating, as some may skip
> a frame now and then, what we can do is to delay painting until there's
> just barely enough time left before vblank to paint the frame. We've got
> lots of information that can be used to help predict when that time
> should be.

So it's generally true that, as far as compositing is concerned, the
time to prepare a frame is low enough that even on higher frequency
monitor there should be “plenty” of time between frames?

Actually duh it obviously is, since even assuming everything had to be
copied over, 1080p at 144Hz is in the order of 1GB/sec, unless I
completely jumbled my math, and of course knowing the worst-case
bandwidth requirements and latency one can predict the time.

Ok, I'm convinced. Just as a curiosity though, are there any plans (or
anything special that needs planning) to support things such as
adaptive sync?

> One option would be to bury heuristics like this into the X server and
> have it send an event when it thinks the Compositing Manager should
> start painting; probably not ideal, but workable. Or even have the
> Compositing Manager tell the X server how much lead time it needs. In
> either case, the X server could suspend the other clients and send an
> event to the Compositing Manager at that time, unblocking when the
> Compositing Manager was finished.

I'd say that the idea of the Compositing Manager doing it sounds more
in line with the general principles of X. Also arguably nobody knows
better what's needed for the generation of a frame than the CM itself,
so they may have different heuristics better tuned for their specific
modus operandi. Why would there be a need for the server to send an
event to the CM?

> Sure, it would be great to suspend applications only when they try to
> manipulate resources on the target CRTC. We can do that in X, it's just
> more work to go through the window system code and stick in checks to
> suspend clients in relevant requests. Completely enforceable, in terms
> of geometry and rendering requests.

I was actually more worried about potential dangers of deadlocks,
actually, or something like that. Like, a client sending multiple
events, some affecting windows on a blocked CRTC, others affecting
windows on a non-blocked CRTC, and there being some kind of implicit
assumption somewhere that the events would be processed in order,
whereas during lockdown the server might end up serving them in a
different order.

>> The question of the century is then: how much time will the server
>> spend in grabbed state? How much does this affect the server behavior?
>
> From the users' perspective, there is no effect at all -- you can't see
> anything that clients do until the Compositing Manager draws the
> screen. And, the server should spend very little time in the grabbed
> state; only long enough for the Compositing Manager to draw the
> frame. A Direct Rendered application would probably see no impact at
> all; if triple buffered, it will have all of the necessary resources to
> continue drawing without waiting for the PresentPixmap to finish.

While you can't _see_ anything until a screen update, applications may
behave differently, since the timings may be different.

>> Would it be more lightweight to have a specific message instead? This
>> could be used to carry extra information. For example, the AutoList
>> for this present may be sent with this message.
>
> For this to work without changing existing applications, I think the
> only thing we can do is stop executing requests from them.

Sorry, I meant for the Compositing Manager to send a specific message.
As in, instead of doing an actual grab/ungrab, have something like a
PreparePresentNotify that the Compositing Manager sends to the server,
with the information about the AutoList for the (upcoming)
PresentPixmap and the CRTC to lock down.

-- 
Giuseppe "Oblomov" Bilotta