Modesetting can't switch back from a software to a hardware cursor (patch to follow)

[Michael Thayer]

Hello Emil,

On 04.03.2016 19:52, Emil Velikov wrote:
> Hi Michael,
>
> On 4 March 2016 at 14:26, Michael Thayer <michael.thayer at oracle.com> wrote:
>> On 02.03.2016 18:43, Michael Thayer wrote:
>>>
>>> At present if modesetting ever fails to set a hardware cursor it
>>> switches back to a software one and stays that way until it is unloaded.
>>>    The following patch should fix that.  I say "should" because I had
>>> difficulties testing it - the cursor simply disappeared when it should
>>> have been rendering in software, though the debugger showed that
>>> pixman_image_composite() was getting called whenever the cursor moved,
>>> and my kernel driver was getting dirty rectangle information.  My
>>> feeling is that the patch is correct and something else is broken.  I
>>> have not investigated in depth in case some one else immediately has an
>>> idea.
>>
>>
>> My apologies for the noise.  Without going into detail, the failure to show
>> the software cursor was due to the unclean way in which we (VirtualBox)
>> handle 3D acceleration, and nothing to do with the X server or modesetting.
>> I tested my patch again, taking this into account, and it worked as
>> expected.
>>
> Note that I'm not the person who wrote any of that code, although I do
> wonder... Why do we have to attempt/probe for hardware cursor
> everything time ? If the first invocation has failed it is reasonable
> to expect that all/most sequential ones will also fail.

Thanks for taking a look.  In VirtualBox and possibly other 
virtualisation environments, the user may enable or disable mouse 
integration with the host system.  Handling this involves enabling (for 
integration) and disabling (for no integration) hardware cursor support. 
  So rendering a cursor in hardware can fail on one invocation and 
succeed for the same cursor on the next.

> Is the failed call to drmModeSetCursor/drmModeSetCursor2 an
> intentional behaviour, it suspiciously sound like a bug in the drm
> module (some along the line) to me. Which DRM module is that ?

Do you mean the -EINVAL?  That is used to mean that drmModeSetCursor2 is 
not supported, but can also mean that a particular cursor cannot be 
rendered in hardware.  I suppose you could call that a bug if that is 
what you were referring to.

> Regards,
> Emil

Regards,

Michael
-- 
Michael Thayer | VirtualBox engineer
ORACLE Deutschland B.V. & Co. KG | Werkstr. 24 | D-71384 Weinstadt

ORACLE Deutschland B.V. & Co. KG
Hauptverwaltung: Riesstraße 25, D-80992 München
Registergericht: Amtsgericht München, HRA 95603

Komplementärin: ORACLE Deutschland Verwaltung B.V.
Hertogswetering 163/167, 3543 AS Utrecht, Niederlande
Handelsregister der Handelskammer Midden-Nederland, Nr. 30143697
Geschäftsführer: Alexander van der Ven, Jan Schultheiss, Val Maher