[PATCH] modesetting: Update fb_id from shadow allocate and destroy if not set

[Lyude Paul]

yes if you wouldn't mind, after that I'll add it to the modesetting pull

On Fri, 2018-06-22 at 00:28 -0700, Tony Lindgren wrote:
> * Lyude Paul <lyude at redhat.com> [180621 22:55]:
> > Hey, was a patch updated re: Keith's comments ever posted for this? Was
> > going
> > to review this, but I can't find any versions of this patch other than
> > this
> > one
> 
> You got the second version already, that's patchworks:
> 
> https://patchwork.freedesktop.org/patch/203834/
> 
> Looks like it's missing the description, do you want
> me to resend it with the description below?
> 
> Regards,
> 
> Tony
> 
> > On Sat, 2018-02-03 at 10:34 -0800, Tony Lindgren wrote:
> > > Looks like drmModeDirtyFB() stopped working at some point and we now
> > > call it with fb_id of zero for rotated displays. This will stop displays
> > > relying on DRM_IOCTL_MODE_DIRTYFB ioctl to display anything.
> > > 
> > > This regression probably with commit 3dcd591fa9b7 ("modesetting: Add
> > > support for using RandR shadow buffers") that inroduced rotate_fb_id.
> > > 
> > > Let's fix this issue by copying rotate_fb_id to fb_id if zero and then
> > > reset it back to zero after we're done with the rotation.
> > > 
> > > After the fix we then call drmModeDirtyFB() we then have the
> > > following fb_id changes:
> > > 
> > > $ startx
> > > drmmode_get_default_bpp: fb_id: 51
> > > drmmode_get_default_bpp: cleared fb_id
> > > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > > drmmode_set_mode_major: drmmode->fb_id: 52
> > > dispatch_dirty: ms->drmmode.fb_id: 52 fb_id: 52
> > > ...
> > > 
> > > $ xrandr --output DSI-1 --rotate right
> > > drmmode_xf86crtc_resize: cleared old_fb_id
> > > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > > drmmode_shadow_allocate: drmmode_crtc->rotate_fb_id: 50
> > > dispatch_dirty: ms->drmmode.fb_id: 50 fb_id: 50
> > > ...
> > > 
> > > $ xrandr --output DSI-1 --rotate normal
> > > drmmode_shadow_destroy: cleared drmmode_crtc->rotate_fb_id
> > > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > > dispatch_dirty: ms->drmmode.fb_id: 0 fb_id: 0
> > > drmmode_set_mode_major: drmmode->fb_id: 51
> > > dispatch_dirty: ms->drmmode.fb_id: 51 fb_id: 51
> > > ...
> > > 
> > > Cc: Hans De Goede <hdegoede at redhat.com>
> > > Cc: Jason Ekstrand <jason.ekstrand at intel.com>
> > > Cc: Laurent Pinchart <laurent.pinchart at ideasonboard.com>
> > > Cc: Sebastian Reichel <sebastian.reichel at collabora.co.uk>
> > > Cc: Tomi Valkeinen <tomi.valkeinen at ti.com>
> > > Signed-off-by: Tony Lindgren <tony at atomide.com>
> > > ---
> > >  hw/xfree86/drivers/modesetting/drmmode_display.c | 10 ++++++++++
> > >  hw/xfree86/drivers/modesetting/drmmode_display.h |  1 +
> > >  2 files changed, 11 insertions(+)
> > > 
> > > diff --git a/hw/xfree86/drivers/modesetting/drmmode_display.c
> > > b/hw/xfree86/drivers/modesetting/drmmode_display.c
> > > --- a/hw/xfree86/drivers/modesetting/drmmode_display.c
> > > +++ b/hw/xfree86/drivers/modesetting/drmmode_display.c
> > > @@ -997,6 +997,12 @@ drmmode_shadow_allocate(xf86CrtcPtr crtc, int
> > > width,
> > > int height)
> > >          return NULL;
> > >      }
> > >  
> > > +    /* Must have proper drmmode->fb_id for drmModeDirtyFB() */
> > > +    if (!drmmode->fb_id) {
> > > +        drmmode->fb_id = drmmode_crtc->rotate_fb_id;
> > > +        drmmode_crtc->reset_fb_id = TRUE;
> > > +    }
> > > +
> > >  #ifdef GLAMOR_HAS_GBM
> > >      if (drmmode->gbm)
> > >          return drmmode_crtc->rotate_bo.gbm;
> > > @@ -1084,6 +1090,10 @@ drmmode_shadow_destroy(xf86CrtcPtr crtc,
> > > PixmapPtr
> > > rotate_pixmap, void *data)
> > >  
> > >      if (data) {
> > >          drmModeRmFB(drmmode->fd, drmmode_crtc->rotate_fb_id);
> > > +        if (drmmode_crtc->reset_fb_id &&
> > > +            drmmode->fb_id == drmmode_crtc->rotate_fb_id)
> > > +            drmmode->fb_id = 0;
> > > +        drmmode_crtc->reset_fb_id = FALSE;
> > >          drmmode_crtc->rotate_fb_id = 0;
> > >  
> > >          drmmode_bo_destroy(drmmode, &drmmode_crtc->rotate_bo);
> > > diff --git a/hw/xfree86/drivers/modesetting/drmmode_display.h
> > > b/hw/xfree86/drivers/modesetting/drmmode_display.h
> > > --- a/hw/xfree86/drivers/modesetting/drmmode_display.h
> > > +++ b/hw/xfree86/drivers/modesetting/drmmode_display.h
> > > @@ -98,6 +98,7 @@ typedef struct {
> > >  
> > >      drmmode_bo rotate_bo;
> > >      unsigned rotate_fb_id;
> > > +    Bool reset_fb_id;
> > >  
> > >      PixmapPtr prime_pixmap;
> > >      PixmapPtr prime_pixmap_back;
> > 
> > -- 
> > Cheers,
> > 	Lyude Paul
-- 
Cheers,
	Lyude Paul