[PATCH] modesetting: Update fb_id from shadow allocate and destroy if not set

[Tony Lindgren]

* Lyude Paul <lyude at redhat.com> [180621 22:55]:
> Hey, was a patch updated re: Keith's comments ever posted for this? Was going
> to review this, but I can't find any versions of this patch other than this
> one

You got the second version already, that's patchworks:

https://patchwork.freedesktop.org/patch/203834/

Looks like it's missing the description, do you want
me to resend it with the description below?

Regards,

Tony

> On Sat, 2018-02-03 at 10:34 -0800, Tony Lindgren wrote:
> > Looks like drmModeDirtyFB() stopped working at some point and we now
> > call it with fb_id of zero for rotated displays. This will stop displays
> > relying on DRM_IOCTL_MODE_DIRTYFB ioctl to display anything.
> > 
> > This regression probably with commit 3dcd591fa9b7 ("modesetting: Add
> > support for using RandR shadow buffers") that inroduced rotate_fb_id.
> > 
> > Let's fix this issue by copying rotate_fb_id to fb_id if zero and then
> > reset it back to zero after we're done with the rotation.
> > 
> > After the fix we then call drmModeDirtyFB() we then have the
> > following fb_id changes:
> > 
> > $ startx
> > drmmode_get_default_bpp: fb_id: 51
> > drmmode_get_default_bpp: cleared fb_id
> > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > drmmode_set_mode_major: drmmode->fb_id: 52
> > dispatch_dirty: ms->drmmode.fb_id: 52 fb_id: 52
> > ...
> > 
> > $ xrandr --output DSI-1 --rotate right
> > drmmode_xf86crtc_resize: cleared old_fb_id
> > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > drmmode_shadow_allocate: drmmode_crtc->rotate_fb_id: 50
> > dispatch_dirty: ms->drmmode.fb_id: 50 fb_id: 50
> > ...
> > 
> > $ xrandr --output DSI-1 --rotate normal
> > drmmode_shadow_destroy: cleared drmmode_crtc->rotate_fb_id
> > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > drmmode_set_mode_major: drmmode->fb_id: 51
> > dispatch_dirty: ms->drmmode.fb_id: 51 fb_id: 51
> > ...
> > 
> > Cc: Hans De Goede <hdegoede at redhat.com>
> > Cc: Jason Ekstrand <jason.ekstrand at intel.com>
> > Cc: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> > Cc: Sebastian Reichel <sebastian.reichel at collabora.co.uk>
> > Cc: Tomi Valkeinen <tomi.valkeinen at ti.com>
> > Signed-off-by: Tony Lindgren <tony at atomide.com>
> > ---
> >  hw/xfree86/drivers/modesetting/drmmode_display.c | 10 ++++++++++
> >  hw/xfree86/drivers/modesetting/drmmode_display.h |  1 +
> >  2 files changed, 11 insertions(+)
> > 
> > diff --git a/hw/xfree86/drivers/modesetting/drmmode_display.c
> > b/hw/xfree86/drivers/modesetting/drmmode_display.c
> > --- a/hw/xfree86/drivers/modesetting/drmmode_display.c
> > +++ b/hw/xfree86/drivers/modesetting/drmmode_display.c
> > @@ -997,6 +997,12 @@ drmmode_shadow_allocate(xf86CrtcPtr crtc, int width,
> > int height)
> >          return NULL;
> >      }
> >  
> > +    /* Must have proper drmmode->fb_id for drmModeDirtyFB() */
> > +    if (!drmmode->fb_id) {
> > +        drmmode->fb_id = drmmode_crtc->rotate_fb_id;
> > +        drmmode_crtc->reset_fb_id = TRUE;
> > +    }
> > +
> >  #ifdef GLAMOR_HAS_GBM
> >      if (drmmode->gbm)
> >          return drmmode_crtc->rotate_bo.gbm;
> > @@ -1084,6 +1090,10 @@ drmmode_shadow_destroy(xf86CrtcPtr crtc, PixmapPtr
> > rotate_pixmap, void *data)
> >  
> >      if (data) {
> >          drmModeRmFB(drmmode->fd, drmmode_crtc->rotate_fb_id);
> > +        if (drmmode_crtc->reset_fb_id &&
> > +            drmmode->fb_id == drmmode_crtc->rotate_fb_id)
> > +            drmmode->fb_id = 0;
> > +        drmmode_crtc->reset_fb_id = FALSE;
> >          drmmode_crtc->rotate_fb_id = 0;
> >  
> >          drmmode_bo_destroy(drmmode, &drmmode_crtc->rotate_bo);
> > diff --git a/hw/xfree86/drivers/modesetting/drmmode_display.h
> > b/hw/xfree86/drivers/modesetting/drmmode_display.h
> > --- a/hw/xfree86/drivers/modesetting/drmmode_display.h
> > +++ b/hw/xfree86/drivers/modesetting/drmmode_display.h
> > @@ -98,6 +98,7 @@ typedef struct {
> >  
> >      drmmode_bo rotate_bo;
> >      unsigned rotate_fb_id;
> > +    Bool reset_fb_id;
> >  
> >      PixmapPtr prime_pixmap;
> >      PixmapPtr prime_pixmap_back;
> -- 
> Cheers,
> 	Lyude Paul