[PATCH xserver] xwayland: rotate logical size for RRMode

[Simon Ser]

On July 13, 2018 11:18 AM, Olivier Fourdan <ofourdan at redhat.com> wrote:
> Hi,
>
> On Wed, Jul 11, 2018 at 9:22 AM Simon Ser <contact at emersion.fr> wrote:
> >
> > From: emersion <contact at emersion.fr>
> >
> > The logical size is the size of the output in the global compositor
> > space. The mode width/height should be scaled as in the logical
> > size, but shouldn't be transformed. Thus we need to rotate back
> > the logical size to be able to use it as the mode width/height.
> >
> > This fixes issues with pointer input on transformed outputs.
> >
> > Signed-Off-By: Simon Ser <contact at emersion.fr>
> > ---
> > This supersedes my previous patch [1], and rotates the logical size
> > instead of using the raw wl_output mode (so that scaling still
> > works).
> >
> > [1]: https://lists.x.org/archives/xorg-devel/2018-July/057285.html
> >
> >  hw/xwayland/xwayland-output.c | 12 +++++++++++-
> >  1 file changed, 11 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/xwayland/xwayland-output.c b/hw/xwayland/xwayland-output.c
> > index 379062549..26e055867 100644
> > --- a/hw/xwayland/xwayland-output.c
> > +++ b/hw/xwayland/xwayland-output.c
> > @@ -213,6 +213,7 @@ apply_output_change(struct xwl_output *xwl_output)
> >  {
> >      struct xwl_screen *xwl_screen = xwl_output->xwl_screen;
> >      struct xwl_output *it;
> > +    int mode_width, mode_height;
> >      int width = 0, height = 0, has_this_output = 0;
> >      RRModePtr randr_mode;
> >      Bool need_rotate;
> > @@ -224,7 +225,16 @@ apply_output_change(struct xwl_output *xwl_output)
> >      /* xdg-output sends output size in compositor space. so already rotated */
> >      need_rotate = (xwl_output->xdg_output == NULL);
> >
> > -    randr_mode = xwayland_cvt(xwl_output->width, xwl_output->height,
> > +    /* We need to rotate back the logical size for the mode */
> > +    if (xwl_output->rotation & (RR_Rotate_0 | RR_Rotate_180)) {
> > +        mode_width = xwl_output->width;
> > +        mode_height = xwl_output->height;
> > +    } else {
> > +        mode_width = xwl_output->height;
> > +        mode_height = xwl_output->width;
> > +    }
>
> But shouldn't this check `need_rotate` as well?
>
> I mean, we should not do this in the case of `xdg-output` not being available.

Ah, you're right. Will send a patch to fix that, and will make sure it
works in Weston (which doesn't implement xdg-output).

> > +
> > +    randr_mode = xwayland_cvt(mode_width, mode_height,
> >                                xwl_output->refresh / 1000.0, 0, 0);
> >      RROutputSetModes(xwl_output->randr_output, &randr_mode, 1, 1);
> >      RRCrtcNotify(xwl_output->randr_crtc, randr_mode,
> > --
>
> This is where I get surprised, because if I understand the problem,
> you where stating that the pointer events stopped working below a
> certain limit.

Yeah. Events per se still work, but the y axis gets stuck at half of
the screen (so clicking at the 3/4 of the screen will actually trigger
a click in the middle of the screen).

> Those are limited by the actual “screen size” as set by
> `update_desktop_dimensions()` at the end of `update_screen_size()`, I
> don't see how changing the size passed to compute the randr mode will
> help with the bug you're trying to address here...

The screen size is indeed correct, but there are two more pieces of
information: the mode size and the output logical size. It seems to me
that the logical size is computed by X11 from the mode size and output
transform (see my previous email [1]). I'm also not an X11 expert and
don't know how the logical size participates in input events handling,
but I do get bad input events with a wrong logical size and setting a
correct logical size fixes it.

[1]: https://lists.x.org/archives/xorg-devel/2018-July/057292.html