[RFC PATCH v2] Add xdg-output protocol

[Pekka Paalanen]

On Fri, 7 Jul 2017 04:21:57 -0400 (EDT)
Olivier Fourdan <ofourdan at redhat.com> wrote:

> Hi Pekka,
> 
> > it's very hard for me to wrap my head around this, so the below may
> > sound a bit harsh, sorry. I don't mean to rant, but I feel there is
> > something fundamental amiss. I am diving back into the high-level
> > design which is fairly separated from the xdg_output interface.  
> 
> No worries, but my goal with the protocol proposal (being an Xwayland
> specific proposal initially, or merely an additional wl_output event
> or now a more generic xdg_output protocol to be extended in the
> future with whatever people want for desktop output uses) is to make
> Xwayland work with the current existing design/implementations, not
> to advocate for or against an existing design or particular
> compositor implementation.

Hi Olivier,

if the intention is not to evaluate whether the current implementations
are actually workable in the long run, then should the new interface be
named so that we can throw it away if it actually doesn't work? And not
design anything else directly on top of it, so that it remains
relatively easy to throw away, i.e. stop using?

> 
> WRT fractional scaling, my understanding of the design in mutter is
> based on this document:
> 
> https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html
> 
> I cannot really comment on the design decisions, nor how those
> decisions were made, Jonas would probably be in a better position for
> this.

That email talks about a new feature, separated coordinate spaces:
logical and physical. I believe that is the correct design.

But I have understood that you want to make Xwayland work with the
design before that one, which I cannot see how it could work. Do you
really need to make Xwayland work well with that old design?

I fully agree with section 1, Overall approach.

Section 5 mentions a hwdb-alike database but keyed by EDID data. This is
something Keith Packard needs for his VR work to identify HMDs and I
discussed with him about it a bit. I think the scaling info would be a
good fit there.

Section 7 touches the game and fullscreen topic, which I believe is
important for deciding how RandR should advertise output resolutions.

Section 8 is exactly what we are talking about in this thread, right?


What resolution do you want RandR to advertise to X11 apps?

If it is the physical output resolution, the window size in logical
units will be wrong unless XWM can reliably detect the X11 application
is actually intending to use the physical resolution. How would one do
that?

If it is the logical output size, the window size in logical units will
be correct, but the X11 application cannot provide a buffer of the
physical output resolution, which means the Wayland compositor must
scale the image. If the buffer can hit direct scanout, that should be
no problem, but otherwise it could be a performance issue. (I agree to
prefer this option.)

In any case, there is a single (default) "scaling factor" covering all
X11 clients in an Xwayland instance, because neither Xwayland nor XWM
have per-window knowledge of the scaling the X11 app used. Also, X11
apps are laid out in a global coordinate space spanning all outputs
(within a X11 SCREEN), and also input works in that global coordinate
space with trivial conversions to per-window coordinates. I assume
Xwayland converts per-wl_surface input coordinates to global input
coordinates in the X11 space before dispatching to the X11 windows,
which means that the Wayland server must send input events taking that
into account. How will input work in the proposals?

I do not like the idea of special-casing the Xwayland Wayland client in
the Wayland compositor outside of XWM, even less if it requires special
mangling for input as well. How much special casing do you do? Or maybe
I'm just biased and it would actually be fairly easy to implement in
Weston with a little bit of API... Quentin might have some opinions
here.

> Regarding this particular discussion about the need of xdg_output and
> logical size/position, I am completely open to suggestions, how would
> see Xwayland work with both weston and mutter as they are now and
> mutter in the future when it implements fractional scaling, without
> adding logical size in xdg_output (or even not adding xdg_output at
> all)?

I wouldn't say "without", I just do not understand how it all is
supposed to work in a mixed-dpi hardware setup. If we design an
interface that just cannot work in a mixed-dpi setup, we should be
able to throw it away later.

So I'm starting to think that the way forward is to design the new
interface such that it is easy to deprecate if it turns out to be a
dead end, or keep it if turns out future-proof.

Yeah, I'm going in circles here as I'm trying to find out what the
goals are.


> 
> Mixed dpi doesn't work in GNOME/x11, does it?

I guess - do you want to keep that status quo?

> GNOME implementation of HiDPI on x11 uses an xsettings, which by
> definition is per screen (in x11 terminology), thus not per monitor.
> 
>   https://wiki.gnome.org/HowDoI/HiDpi
> 
> So fixing mixed DPI in Xwayland will require additional mechanisms
> (possibly new EWMH properties?), but this is not part of this
> proposal.

I would not want to go there. I would prefer to keep all of X11 in a
single scale factor, and scale accordingly in the compositor. That is
what I call a working mixed-dpi setup - it may not be pixel-perfect
pretty, but at least everything is readable and not blown up.

Maybe the definition of "mixed-dpi support" is what we disagree here.

In my mind, Xwayland cannot provide full mixed-dpi support, because X11
is simply incapable of it with current toolkits. It can provide partial
mixed-dpi support however, where the Wayland compositor is
automatically scaling, assuming all X11 apps are configured to use the
same scale factor.

It same as with Wayland apps: HiDPI unaware apps get partial mixed-dpi
support, while HiDPI aware apps get the full mixed-dpi support.

This would classify all HiDPI-aware X11 apps as being somewhere between
HiDPI unaware and aware Wayland apps: they cannot communicate their
scaling factor, but they can be configured as a group to always draw
with a given scaling factor.

> However, nothing would stop an x11 client or toolkit from querying
> the monitor's mode and physical size (using xrandx), computing the
> current DPI depending on the monitor it resides and adapting its
> rendering based on that. X11 has all the mechanisms in place for
> that, it's just that most clients won't do that (iirc, Firefox has
> something like that at some point, not sure it's still used though)

That's actually a case I didn't even think of. I've only been thinking
about apps that want to make their window cover an output, like games.
This is a very good topic to bring up.

> That makes a wide range of different x11 clients who behave
> differently:
> 
>  - Plain old x11 clients, who don't know anything about DPI
>  - gtk+/GNOME clients, who base their rendering on a specific
> xsettings Gdk/WindowScalingFactor and/or a envvar read by clutter,
>  - Some other app trying to compute DPI themselves (very few, I think
> of Firefox, LibreOffice maybe? not sure at all about those)
> 
> I'm don't see how Xwayland could change its behavior for each app,
> xrandr is not per client. I guess we could come up with a new window
> property that Xwayland could monitor and set the buffer scale
> accordingly, maybe? Anyhow, that's going off topic wrt the xdg_output
> protocol, I'm afraid.

I agree, I don't see the worth in implementing per-X11-window scales.

Therefore we need to stick with one scale over all X11 clients. If
there are plain old clients in that set, then the only workable factor
for X11 is scale=1. Would you agree with that?

> The "benefit" I see in "lying" to the clients in xrandr by
> advertising a lower mode than actual is that it solves the problem in
> a consistent way for all these cases even those clients who try to
> compute the DPI themselves).

How does it solve it for all the cases? Does it not help only with the
third case, while the first and second are unaffected? Not to belittle
the benefit, just wanting to be accurate.

> There are some additional discussions in
> https://bugs.freedesktop.org/show_bug.cgi?id=93315

> Xwayland computes the screen size by adding up the output size and
> position. So it all comes from the same wl_output description (for
> now). 
> 
> Plain x11 screen width/height, Xrandr, Xinerama and even Xvidmode
> will remain consistent. 

That's nice, it's all consistent. We just need to be similarly
consistent in what we advertise as size in RandR. It means it has to
make sense in a mixed-dpi setup.



> If randr advertise the output mode 3840×2160 as 2560×1620 with a
> fractional scale of 1.5, then the compositor would scale the buffer
> up by that fractional scaling factor, i.e. 1.5, which gives the
> expected 3840×2160.
> 
> I don't see how downscaling an already tiny x11 window would work.
> Take a basic x11 app, say xterm, with its default fixed font size. It
> looks "fine" on a low DPI monitor, but is completely unreadable on a
> hidpi screen (at least for my old eyes), so the goal would be to
> scale it up, not down.
> 
> The scale factor of 2 advertised in wl_output would be for Wayland
> native clients, so they can set their buffer scale to that scale 2,
> and then be downscaled by the compositor to achieve the expected
> fractional scale of 1.5 with best rendering on screen.

Oh yes, I forgot that the X11 apps in this case are always drawing with
scale=1, so they are expected to be scaled up indeed. I must have been
thinking about all of X11 land being configured with scale=2.

> If the compositor was to scale Xwayland surfaces by 2 as well and

I think you mean the X11 apps are configured to draw in scale=2, right?

> then downscale to achieve the expected fractional scale of 1.5, it
> would downscale by a factor of (2÷1.5) = 1.33, in which case it would
> simply advertise a logical size of 5120×2880 (i.e [3840 × 2 ÷
> 1.5]×[2160 × 2 ÷ 1.5]) - That's the whole idea behind the logical
> size, whatever the compositor does eventually, Xwayland gets it right
> as long as the compositor advertise the expected size once the
> scaling factor -whatever it is- is applied, if that makes any sense.
> 
> At least it's how I understood it from the description from:
> 
> https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html
> 
> But I might be wrong in my understanding :)

Yes, now we are onto something. This feels good, considering outputs in
isolation. I didn't actually go through the numbers yet to make sure the
example is right, though.

> > I am so confused, because it feels like you are inventing scaling
> > factors here and there to get the numbers you want, without any
> > design on what the coordinate systems actually are. OTOH, I always
> > think in coordinate systems and how to map from one to another, how
> > you move on the 2D plane and how that motion translates to the
> > other one, especially when crossing from a low-dpi output to a
> > high-dpi output. I would love to see the coordinate systems drawn
> > out, and the mappings between them written down, parametrized with
> > what we intend to have in protocol and in compositor scaling
> > modes.  
> 
> I read this as a question about fractional scaling design, not how
> Xwayland would advertise output size.

They are all related. Xwayland itself together with all X11 clients use
a coordinate system, and everything needs to be consistent: RandR,
window placement, buffers, input.

> > I also cherish the idea of a chain of coordinate transformations.
> > Start with coordinate system 1, apply transformation A, end up in
> > coordinate system 2, apply transformation B, end up in coordinate
> > system 3. All transformations are invertible. If there is more than
> > one chain of transformations between two coordinate systems, the
> > total transformations must be identical. The less exceptions on
> > when each coordinate system applies, the better. This might be
> > obvious, but it must also hold when traveling in a coordinate
> > system and crossing window and output boundaries, and that is where
> > I think we will hit problems. The transformations do not apply only
> > to points, they must also apply to vectors (motion).  
> 
> That's up to the compositor, not Xwayland, innit? Xwayland is just a
> Wayland client and has no control on that.

No, I believe Xwayland converts input coordinates from per-wl_surface
to global X11 coordinates, and then does everything in the global X11
coordinate system which is all that X11 apps ever know about.

ISTR there have been Xwayland patches going back and forth on that
design, so I'm not sure what the current state actually is. It
definitely is tricky code in Xwayland.


> > > Yes, but that's for "regular" Wayland clients, Xwayland surface
> > > aren't scaled in mutter even though Xwayland doesn't use
> > > wl_surface.set_buffer_scale (mutter doesn't support
> > > set_buffer_transform, either)  
> > 
> > Right, I think that's a mistake. Or at least it makes me not
> > understand at all how Mutter can work in a mixed-dpi setup.  
> 
> You mean mutter or Xwayland with mutter? Either way, it's a comment
> for mutter devs.

Mutter with Xwayland, yes, and more precisely the partial HiDPI support
which is fundamental in getting X11 apps show up readable and not blown
up.

We actually can make even plain old X11 apps scale up properly on HiDPI
outputs, if the Wayland compositor's design allows it. And it sounds
like the new coordinate systems work in Mutter is going to allow that,
while the old way of never scale any X11 window was as bad as native
X11 on HiDPI.

So my criticism here is against caring at all about the old way of
never scaling any X11 window in the Wayland compositor. Maybe I have
misunderstood and you don't care about it?


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.x.org/archives/xorg-devel/attachments/20170712/fe151c5d/attachment-0001.sig>