Proposal for RandR version 1.6, Leases and EDID-based output grabs

[Pekka Paalanen]

On Thu, 04 May 2017 11:02:44 -0700
Keith Packard <keithp at keithp.com> wrote:

> Pekka Paalanen <ppaalanen at gmail.com> writes:
> 
> > That means you need an explicit key to denote HMDs. More below.  
> 
> I don't think so. Presumably the VR system will have a known list of
> HMDs it works with, and so it will probably have a list of EDID
> information; I expect that's where the data for monitors-to-ignore will
> come from in many cases. After all, the goal is to prevent the desktop
> From shifting onto a display only to get kicked off as the VR app takes
> over the HMD.

Hi,

I disagree on the details, more below.

> > That way the desktop would extend there automatically, but it would
> > also be advertised as a HMD to clients. If it's not listed at all,
> > would it be possible to advertise it as a HMD and DRM-leased out etc.?  
> 
> I think all we'd do is offer a new RandR request which listed "all"
> monitors, including those hidden from the desktop, and expect that the
> monitor-specific applications would use that to detect the presence of
> their magic outputs and use them. For an HMD, you'll also have other
> devices connected, so we just need some way of positively associating
> the various input devices with the specific monitor (in case there is
> more than one).

Such a RandR request is something I would not like to have to replicate
on Wayland. The display server contains the policy, it should not just
expose everything up for grabs. This is a fundamental difference
between X11 and Wayland architectures, and I think the output
information database should support both ways.

> > At least for Wayland, I'd like to not advertise "everything" as
> > possible HMDs but only those that really are.  
> 
> I don't think the window system needs to know that the displays are HMD,
> only that it shouldn't present them as part of the desktop. Everything
> else about them can be left outside of this spec.

I disagree. Wayland will likely have a special protocol extension
exclusively for advertising HMDs, so the display server will need to
know which ones are HMDs and which one are not, regardless whether the
desktop is extended there or not. This extension could also offer the
DRM leasing capabilities.

I gave the PSVR as an example of hardware where one both can
meaningfully extend the desktop there and offer it as a HMD. When that
output is taken into HMD use, the display will automatically stop
putting the desktop there - but only temporarily. All windows that were
on that extended output should still be there when the VR
app/compositor finishes. This is window management policy, so the
window manager must know what is going on.

(That is similar to the fullscreening with video mode changing
mechanism, where the video mode requested by an application is
temporary, and normality will be restored automatically when e.g. the
window disappears by client crash or the user switches to another
window.)

Having the window manager know what is a HMD and which client is active
on it will let it make better management decisions and even allow
switching between VR apps, or temporarily switching from VR app to the
desktop when supported, and back.

> > I also read some discussions about 3D TV modes, but couldn't really get
> > a conclusion if they are supposed to render like if you were in a 3D
> > cinema theatre or not. Apparently that was enabled in a firmware
> > upgrade and maybe had some extra requirements.  
> 
> Hard to imagine this being relevant for the Linux desktop at least...

Depends on how a video player will deliver the 3D movie content. Can it
be done as a normal GUI app providing stereo buffers, or does one need
to turn the video player into a VR app and reimplement the cinematic
mode in software. The latter option works for any HMD, but the former
option needs less power from the computer as head tracking etc. is
outsourced to the "HMD" itself.

> Thanks again; I'm sensing that a simple 'ignore this monitor for the
> desktop' might suffice for now, but that we'll end up wanting something
> more complicated in the future. I think the key is to try and avoid
> making that harder by what we do now, but I don't think we should be
> trying to implement a larger solution too soon.


Yes, that I agree with.

Ultimately I would envision an output device database describing
what kind of a device it is (a normal monitor, a video projector, a
HMD, a TV, ...) and then the software that implements the desktop or UI
policy will decide how that will be used. Some policy examples:

- A projector: do not extend desktop UI, but have the output normally
  available, so one can put regular windows there (presentation
  software). Turned on by default.

- A HMD: Do not extend desktop, do not expose to normal apps, keep it
  off until special request.

- A HMD with cinematic mode support: Extend desktop, turn on by
  default, allow special HMD requests.

- A TV: Try to disable overscan or try to compensate for it by default.

Also, a normal monitor and maybe a TV might have physical image size
relevant, while for a projector and a HMD the physical size is mostly
irrelevant.

So, how can we start simple while avoiding the need to break everything
if we later want to extend support for all the fancy stuff...

I would suggest to have a "output device type" attribute in the
database, and support only one value for now: "HMD". Then all display
servers can encode the policy to hide all HMDs by default.
Implementationwise it is no different from your idea, but separating
device description from usage policy would be a good thing in my
opinion. You can still document suggested policies in the spec if you
wish, only the wording will be more of a recommendation than a
requirement.


Thanks,
pq
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <https://lists.x.org/archives/xorg-devel/attachments/20170505/9442250b/attachment.sig>