[RFC PATCH v2] Add xdg-output protocol

[Olivier Fourdan]

Hi Pekka,

> it's very hard for me to wrap my head around this, so the below may
> sound a bit harsh, sorry. I don't mean to rant, but I feel there is
> something fundamental amiss. I am diving back into the high-level
> design which is fairly separated from the xdg_output interface.

No worries, but my goal with the protocol proposal (being an Xwayland specific proposal initially, or merely an additional wl_output event or now a more generic xdg_output protocol to be extended in the future with whatever people want for desktop output uses) is to make Xwayland work with the current existing design/implementations, not to advocate for or against an existing design or particular compositor implementation.

WRT fractional scaling, my understanding of the design in mutter is based on this document:

https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html

I cannot really comment on the design decisions, nor how those decisions were made, Jonas would probably be in a better position for this.

Regarding this particular discussion about the need of xdg_output and logical size/position, I am completely open to suggestions, how would see Xwayland work with both weston and mutter as they are now and mutter in the future when it implements fractional scaling, without adding logical size in xdg_output (or even not adding xdg_output at all)?
 
> > > [...]
> > > IOW, this event is a different way of sending the output scale,
> > > supporting arbitrary fractional values. Except that we expect most
> > > clients to ignore this and draw with the integer scale factor,
> > > instead of using this information and wp_viewport to realize the
> > > fractional scale factor. Is that correct?
> > 
> > Yes, but not just that, this is also to cope with the different ways
> > weston and mutter (< 3.26) deal with Xwayland surfaces, what you
> > mentioned about Xwayland not downscaling the advertised output size
> > (needed for weston, but not for mutter).
> 
> Right... let me think out loud once again.
> 
> X11 essentially uses a single global coordinate space for everything.
> Traditionally it has no scaling, so window and output sizes are in the
> same units. It must keep a single concept of a unit pixel to be
> consistent.
> 
> But we could do the scaling two ways: a) consider everything X11 to be
> scale=1, or b) consider everything X11 to be scale=N. Both are
> consistent. However, using mixed-dpi setup, it would be inconsistent to
> advertise all outputs with the hardware resolutions unscaled.
> 
> ...
> 
> Ok, I'm not sure where I was going with that.

Mixed dpi doesn't work in GNOME/x11, does it?

GNOME implementation of HiDPI on x11 uses an xsettings, which by definition is per screen (in x11 terminology), thus not per monitor.

  https://wiki.gnome.org/HowDoI/HiDpi

So fixing mixed DPI in Xwayland will require additional mechanisms (possibly new EWMH properties?), but this is not part of this proposal.

However, nothing would stop an x11 client or toolkit from querying the monitor's mode and physical size (using xrandx), computing the current DPI depending on the monitor it resides and adapting its rendering based on that. X11 has all the mechanisms in place for that, it's just that most clients won't do that (iirc, Firefox has something like that at some point, not sure it's still used though)

That makes a wide range of different x11 clients who behave differently:

 - Plain old x11 clients, who don't know anything about DPI
 - gtk+/GNOME clients, who base their rendering on a specific xsettings Gdk/WindowScalingFactor and/or a envvar read by clutter,
 - Some other app trying to compute DPI themselves (very few, I think of Firefox, LibreOffice maybe? not sure at all about those)

I'm don't see how Xwayland could change its behavior for each app, xrandr is not per client. I guess we could come up with a new window property that Xwayland could monitor and set the buffer scale accordingly, maybe? Anyhow, that's going off topic wrt the xdg_output protocol, I'm afraid.

The "benefit" I see in "lying" to the clients in xrandr by advertising a lower mode than actual is that it solves the problem in a consistent way for all these cases even those clients who try to compute the DPI themselves).

There are some additional discussions in https://bugs.freedesktop.org/show_bug.cgi?id=93315

> > > I suppose we could really use a good write-up (with examples and
> > > pictures!) on how Xwayland and compositors are intended to use this
> > > information, how it affects RandR parameters and how X11 clients
> > > will use this, and how their windows go through Xwayland to the
> > > compositor,
> > 
> > Well, X11 clients could never use that directly of course.
> 
> If this affects what sizes RandR exposes and if X11 apps use RandR as
> the base to size and position their windows, then this does fairly
> explicitly affect how X11 apps work. But that's only one thing they
> might do, they might look at SCREEN dimensions(?) and whatever... does
> Xinerama protocol provide yet another way to get "the same" information?

Xwayland computes the screen size by adding up the output size and position. So it all comes from the same wl_output description (for now). 

Plain x11 screen width/height, Xrandr, Xinerama and even Xvidmode will remain consistent.
 
> > > to achieve compositor-bypass and hit the direct scanout path in a
> > > mixed-dpi environment. I get dizzy when I try to think of that.
> > > IOW, I feel I cannot really evaluate the suitability of this
> > > solution for the underlying Xwayland issue.
> > 
> > The Xwayland issue this is aimed at solving is current, no need to go
> > as far as achieving compositor bypass or direct scan out (I am not
> > sure either if the logical_size would help at all with that)
> 
> Well, when we think about what we should expose in RandR, and assuming
> there are apps, e.g. fullscreen games, that use that information for
> window size, and window size determines buffer size, which eventually
> is related to whether one can scan it out without scaling as
> intended... or deliberately with scaling when picking a "non-native"
> resolution via RandR... luckily modern hardware is usually capable of
> scaling at scanout, so a wrong scale is not a performance-cliff.
> 
> Yes, that is not the current issue, but I have a feeling that the
> solutions to both issues are intertwined, which makes me fear of an
> arbitrary decision making something else impossible. But that's me.
> 
> > Let's take an example, a hidpi monitor 3840×2160 at scale 2.
> > 
> > Right now, advertised as its mode resolution in Xwayland (which
> > doesn't take into account the wl_output scale), so Xrandr reports:
> > 
> >   XWAYLAND1 connected 3840x2160+0+0
> >      3840x2160     60*+
> > 
> > In weston, an X client trying to configure its window the size based
> > on what Xrandr reports for the monitor ends up with a window of size
> > 7860×4320 because weston will scale the Xwayland surface buffer by 2,
> > whereas in mutter, the resulting window is 3840×2160 because mutter
> > (up to 3.26) doesn't scale Xwayland surfaces. But that will
> > eventually be configurable in mutter, so there is no "one fits all"
> > solution other than the compositor itself telling the "logical"
> > output size.
> 
> Ok, I get that. It the difference in the chosen global scale factor
> between X11 coordinate system and the compositor logical coordinate
> system.
> 
> > In such a case, weston would set the "logical_size" as 1920×1080
> > whereas mutter current or future version without
> > "scale-monitor-framebuffer" set would set a logical size of 3840×2160.
> 
> Don't forget that it will also affect input coordinates. Setting the
> logical_size effectively chooses a single scale factor for all X11, and
> even in a mixed-dpi setting, outputs must not disagree on what the
> scale factor is, otherwise I would expect e.g. input to become nearly
> impossible to get right.
> 
> > With mutter implementing fractional scaling and using a scale of 1.5
> > (for example), it would set a wl_output.scale of 2 and set a logical
> > size of 2560×1620.
> 
> RandR would advertise 2560×1620, X11 apps will create windows and
> buffers of that size, and then mutter will... scale them how?
> 
> If it assumes the Xwayland windows are already in scale=2, the
> wl_surface size becomes 1280×810, and that scaled up to the output
> factor 1.5 gets us to 1920×1215.
> 
> If it assumes Xwayland windows are not to be scaled at all, the surface
> size becomes 2560×1620, which on the 1.5 output will be 3840×1620. That
> means that the compositor is up-scaling for the output, not
> down-scaling as I understood was intended.
> 
> If it assumes Xwayland windows are literally not to be scaled at all,
> the size on the 1.5 output will be 2560×1620, which calculating
> backwards would mean surface size 1706.7×1080.

If randr advertise the output mode 3840×2160 as 2560×1620 with a fractional scale of 1.5, then the compositor would scale the buffer up by that fractional scaling factor, i.e. 1.5, which gives the expected 3840×2160.

I don't see how downscaling an already tiny x11 window would work. Take a basic x11 app, say xterm, with its default fixed font size. It looks "fine" on a low DPI monitor, but is completely unreadable on a hidpi screen (at least for my old eyes), so the goal would be to scale it up, not down.

The scale factor of 2 advertised in wl_output would be for Wayland native clients, so they can set their buffer scale to that scale 2, and then be downscaled by the compositor to achieve the expected fractional scale of 1.5 with best rendering on screen.

If the compositor was to scale Xwayland surfaces by 2 as well and then downscale to achieve the expected fractional scale of 1.5, it would downscale by a factor of (2÷1.5) = 1.33, in which case it would simply advertise a logical size of 5120×2880 (i.e [3840 × 2 ÷ 1.5]×[2160 × 2 ÷ 1.5]) - That's the whole idea behind the logical size, whatever the compositor does eventually, Xwayland gets it right as long as the compositor advertise the expected size once the scaling factor -whatever it is- is applied, if that makes any sense.

At least it's how I understood it from the description from:

https://mail.gnome.org/archives/gnome-shell-list/2017-June/msg00000.html

But I might be wrong in my understanding :)

> I am so confused, because it feels like you are inventing scaling
> factors here and there to get the numbers you want, without any design
> on what the coordinate systems actually are. OTOH, I always think in
> coordinate systems and how to map from one to another, how you move on
> the 2D plane and how that motion translates to the other one,
> especially when crossing from a low-dpi output to a high-dpi output. I
> would love to see the coordinate systems drawn out, and the mappings
> between them written down, parametrized with what we intend to have in
> protocol and in compositor scaling modes.

I read this as a question about fractional scaling design, not how Xwayland would advertise output size.

> I also cherish the idea of a chain of coordinate transformations. Start
> with coordinate system 1, apply transformation A, end up in coordinate
> system 2, apply transformation B, end up in coordinate system 3. All
> transformations are invertible. If there is more than one chain of
> transformations between two coordinate systems, the total
> transformations must be identical. The less exceptions on when each
> coordinate system applies, the better. This might be obvious, but it
> must also hold when traveling in a coordinate system and crossing
> window and output boundaries, and that is where I think we will hit
> problems. The transformations do not apply only to points, they must
> also apply to vectors (motion).

That's up to the compositor, not Xwayland, innit? Xwayland is just a Wayland client and has no control on that.

> I still think that handling Xwayland as a special client that does not
> obey the normal Wayland protocol semantics like how the size of a
> wl_surface is determined from buffer size, buffer_scale and
> buffer_transform, maybe via wp_viewport, is not a good design. I'm not
> sure if Mutter does that or not.

If you want to see how current mutter works with mixed DPI and Xwayland, you can try mutter nested with:

 $ MUTTER_DEBUG_NUM_DUMMY_MONITORS=2 MUTTER_DEBUG_DUMMY_MONITOR_SCALES=2,1 mutter --wayland --nested

(from the HiDPI GNOME wiki page mentioned above)

And then run an xterm on the correspnding X11 display:

 $ DISPLAY=:1 xterm

> > I'll add that example in the description for logical_size.
> 
> I think this would need a separate design document with pictures,
> cramming it into the protocol doc doesn't seem feasible to me. Ideally
> we'd have it in the Wayland docbook. I see no problem having
> desktop-specific chapters there.
> 
> > > > +
> > > > +	The logical size being in global compositor space
> > > > implies that the
> > > > +	client does not need to apply any wl_output.scale or
> > > > wl_output.rotation
> > > > +	to the given logical_size to figure the size of a
> > > > surface when
> > > > +	displayed on the output.
> > > 
> > > The only way you can actually directly set the size of a surface is
> > > via wp_viewport, and in all other cases the surface size is
> > > computed from the buffer size via the client-set buffer_scale and
> > > buffer_transform.
> > 
> > Yes, but that's for "regular" Wayland clients, Xwayland surface
> > aren't scaled in mutter even though Xwayland doesn't use
> > wl_surface.set_buffer_scale (mutter doesn't support
> > set_buffer_transform, either)
> 
> Right, I think that's a mistake. Or at least it makes me not understand
> at all how Mutter can work in a mixed-dpi setup.

You mean mutter or Xwayland with mutter? Either way, it's a comment for mutter devs.

> > > I think you wanted to talk about the output size here, somehow, that
> > > you are given it directly and not need to take into account
> > > output_scale and output_transform. If one then arranges a
> > > wl_surface to have that size, it will be the same size as the
> > > output area. So it's about finding out what size a client wants to
> > > have, not how the size of a wl_surface is determined. Can you see
> > > what I mean?
> > 
> > I think I kinda see what you mean, I would remove that sentence
> > altogether (it's convoluted and confusing) and rephrase the
> > introductory sentence as:
> > 
> >  "For example, a surface without any buffer scale, transformation
> >   nor rotation set, with the size matching the logical_size will
> >   have the same size as the corresponding output when displayed."
> 
> I think it could be simplified even further. A wl_surface whose size
> equals the logical_size of an output, will be shown at the size of the
> output.
> 
> There is no reason to rule out buffer_scale, buffer_transform, or
> wp_viewport, because all they do is affect what size the wl_surface
> will be based on the buffer size. A client can set all those any way it
> wants if the resulting wl_surface size equals the output logical_size.
> 
> > > > +    <event name="done">
> > > > +      <description summary="Sent all information about output">
> > > > +	This event is sent after all other properties have
> > > > been	sent, after
> > > > +	binding to an xdg_output interface and after any other
> > > > property changes
> > > > +	done after that.
> > > > +
> > > > +	This allows changes to the xdg_output properties to be
> > > > seen as atomic,
> > > > +	even if they happen via multiple events.
> > > 
> > > A good idea.
> > 
> > Not mine of course, this is from wl_output :)
> 
> Yes, it is good to not make the same mistake twice. ;-)

Cheers,
Olivier