Fixed point matrix representation considered harmful

[Keith Packard]

I'm sitting here trying to do some simple projective presentations and
I'm running up against our fixed point matrix representation. Using
16.16 to store a projective transformation turns out to be a really bad
idea.

The essential problem is that we take a homogeneous 2D coordinate V
(x,y,1) and multiply it by our transformation matrix M to compute
V' (x',y',w'). Assuming that our matrix coordinates each have .5 ULP of
error. Consider a simple keystone correction matrix:

	| 1.1   0.05   -50 |
	| 0     1.1      0 |
	| 0     0.0001   1 |

Set x and y to 1024. Now the computation of w' is about to
cause a serious problem. The value 0.0001 ends up represented as
0x00000007 in fixed point, which is off by about 7%. Multiply that by
1024 and add it to 1 and our 'w' value ends up off by about .6%. This
error propagates into the computation of the normalized x'' and y''
values.

So, seemingly simple transforms can end up with large computational
errors. I can't see any good way around this except to represent
matrices as floating point numbers. Even 32-bit floats would serve here
-- the problem is just the lack of precision in representing the small
perspective correction factors.

I propose that we add the ability to transmit matrices across the wire
in floating point form, and that we store them internally in the server
in floating point form.

The first change is backwards compatible -- we won't remove the existing
fixed point matrix transmission function, we'll just add a floating
point one. The second change breaks all drivers that use transforms. We
can preserve ABI by leaving the existing fixed point matrix in place and
adding a floating point matrix a the end of the picture object.

-- 
keith.packard at intel.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.x.org/archives/xorg/attachments/20080318/07cee82d/attachment.pgp>