Fixed point matrix representation considered harmful

[Keith Packard]

On Wed, 2008-03-19 at 07:20 -0500, Pat Kane wrote:

> That sounds like a good idea, here are a couple of quick questions:
> 
>     1)  are you talking about IEEE 754 floating point for internal storage?

I don't care about internal storage, only what appears on the wire. Of
course, on the wire, we'll only encode in IEEE format.

>     2)  how do we deal with floating point exceptions?

We could guard against them by screening the matrix when set.
Remembering that the only thing we do with the matrix is run coordinates
through it, we can prevent overflow by range checking the values, and we
can prevent divide-by-zero by finding the zero for the last row and
making sure it's not within the dimension of the associated picture.

We should do this even for the current fixed point matrix though.

-- 
keith.packard at intel.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL: <http://lists.x.org/archives/xorg/attachments/20080319/a6b919e4/attachment.pgp>