XInput 2: the big picture, relationship to toolkits

[Peter Hutterer]

On Thu, Aug 14, 2008 at 01:14:43PM -0400, Owen Taylor wrote:
> > This is what floating slave devices are for. You float the device, grab it and
> > map the device coordinate range (in the application) to the desired area. I
> > believe this should cover these use case sufficiently?
> 
> If this use case exists (a definite question) then I think you want
> events delivered through the master pointer for this as well ... client
> side drawn pointers are really painful for applications, and you want
> normal event delivery to subwindows. However, I think we can say that
> this case isn't significant. If it comes up, it could be handled by
> watching the window position and adjusting the configuration of the
> slave device.

The use-case for floating SDs is the GIMP, where I may want the tablet to map
to the canvas without doing anything else on the screen. This would of course
require the app to draw the cursor.

The other use-case is where you want your tablet and your mouse controlling
the same cursor. In this case, you can grab the MD and process in the usual
way. ConfineTo restrains the cursor but doing so will also lose precision
since the server relocates events outside ConfineTo.

There is no mechanism for having MDs map to a window/area (yet). 
It might be worthy to know that if you grab an attached SD, you float the
device for the duration of the grab which is a good solution if you want the
tablet to control the mouse unless being used on the canvas.

As a side note, floating SDs have their own sprites in the server, it is
merely the rendering that is suppressed.
 
> Again, what I'm talking about how the slave device coordinate ranges
> are mapped onto pointer positions. This is something that has to be
> configured in the server. You could make preserving the aspect ratio
> part of the specification for that. Say you specify, for device:
> 
>  Root window rectangle
>  Preserve aspect boolean
> 
> But you could also push it off to the configuring client. If the setting
> is a list of:
> 
>  Screen #
>  Device rectangle
>  Root window rectangle

Oh, actually, this would be useful for the scaling code in the server
(dix/getevents.c:rescaleValuatorAxis()). We know the axis ranges for the
device and the screen, so we could do this transparently.

4 settings:
off - no scaling
all - current scaling x - width, y - height
x   - scale x - with, scale y with same aspect.
y   - scale y - height, scale x with same aspect.

The latter two may leave areas of the screen or device unused. e.g. if a 4:3
tablet is x-mapped to 16:10 screen, the lower area of the tablet is basically
dead.
I think this is a trivial enough change, but needs exposure through
configuration APIs.

> > I just checked the code and it looks as if the valuators reported are the
> > device coordinates - for absolute devices. This should give you subpixel
> > ranges, especially if you float the device and map it to arbitrary areas. Note
> > that these valuators are always in absolute device coordinates and not
> > relative to any window. 
> 
> The current code, as I read GetPointerEvents(), seems to kill the
> subpixel positioning since it converts to screen coordinates then
> converts back to device positioning. 

yes, you're right, those innocent *v0 = x; lines I missed yesterday.
I'm trying to remember why we did that, something with screen crossing. This
code needs cleanup...

> But beyond that, asking clients to do the the conversion between device
> valuator coordinates and root window coordinates seems unnecessarily
> annoying ... the client has to be aware of all the details about how device
> window coordinates map to screen coordinates and exactly replicate the
> algorithm that the server uses.




> > Relative devices however are clipped to screen coordinates. I guess the main
> > problem here is simply a problem with XInput's axis specification. An axis can
> > be absolute or relative, but not both. If it is relative, it will not get
> > scaled by the server and is initialised (usually) with a range of -1/-1 or
> > 0/-1.
> >
> > If such a device reports a relative axis movement by +10, it is difficult to
> > scale that into subpixel values without knowing where a pixel ends and a
> > subpixel starts. We'd have to look at device resolution + screen resolution to
> > get any decent approximations. That said, I don't feel qualified yet to
> > comment too much on that, having spent too little time on it. Simon, maybe you
> > want to chime in here?
> 
> Subpixel positioning only makes sense when an axis is slaved to the X/Y
> position of the pointer. Otherwise, the axis is just a number. 
> 
> When that slaving does occur, I don't see a problem with getting
> subpixel relative motions ... the server has to know how it is
> converting valuator motions into changes to the pointer position ...
> that algorithm should work as well for non-integral pixels as integral
> pixels. (You obviously have to track a subpixel position, this can be
> done with dev->last.valuators[0].) 

well, right now the server basically does x += dx, that's it. Simon was
talking (private mail to me) about using acceleration remainders as subpixel
information.

> The problem in this area I do see is that each *device* has to be either
> relative or absolute. So if you have a relative device that both
> controls the pointer and has other interesting relative axes (the scroll
> wheel, say), then how do you generate master events that have both:

*sigh* You can't right now. XI 1 doesn't allow for mixed axes.
 
> I suppose, as part of the convention saying that v[0]/v[1] for master
> devices are fixed-point screen coordinates, we could say they are always
> absolute, no matter what the 'model' is from the ValuatorClass. Ugly,
> but seems workable.

I was thinking along the lines of including all changed valuators in both abs
and relative in each event. You just use whatever you feel is appropriate
(based on the flags in the DeviceInfo struct which tell you which axis is
abs/rel).

Obviously relative valuators 3 and above cannot be included in the absolute
valuators since they cannot be scaled meaningfully by the server.

> Ah, missed the newSlave field. I don't think you really save having to
> ListInputDevices and track changes since the two pieces that
> are most important ... type and name ... are not provided in the 
> DeviceClassesChanged, but doing that tracking is certainly feasible.

Well, you need to list devices once, at startup and then after each
DevicePresence event. For DCCE, you can then get type/name from the internal
table you've prepared earlier. Doing a ListInputDevices after each DCCE is not
good, you really want that information before. Moreover, by the time your
request is processed, the MD may have switched several times already and the
reply would not include the data you wanted when you issued the request.

> > > One possible fix here would be to add another type of 
> > > device information class .. ExtendedValuatorInfo, say that
> > > contains extra information per axis.
> > 
> > I was toying with the idea of axis labelling through device properties.
> > If we can come up with a sort-of standard for label names (x, y, z, pressure,
> > tilt, etc.) that might be the simplest and most flexible approach.
> > 
> > The biggest advantage here: in the end the server doesn't care for anything
> > but x/y, so let the driver label the axes once to reflect their physical
> > meaning. Then let the clients argue about how to interpret each axis.
> > 
> > I have not yet found the time to sit down and actually think it through so
> > there may be flaws in that plan.
> 
> Leaving some issues I have with the details of the device property
> interface aside, I think it's as workable to put extended valuator
> information there as it is in the ClassInfo structures. But it's going
> to be far simpler if the device properties are supplied by the driver
> rather than being argued about by clients :-). Otherwise, you'll need
> some vastly more complicated solution with .fdi files and hal/DeviceKit
> just to get things labeled properly.

Sure. The idea was that driver labels axis as "X, Y, Pressure, Tilt, 42,
Bananas" and the clients are expected to know what that means and how to
interpret it for their specific use-case. There's probably some sorting-out to
do with (run-time) axis mapping changes, but that should be straightforward.

> > See above, DCCE does it :)
> 
> Can you determine the initial condition?

Hmm, no. When you run XListInputDevices you see the MD as it is now, but you
don't know which was the lastSlave. The server doesn't store this info ATM,
but it'd be trivial to add.

Cheers,
  Peter