Re: DirectX and Lua

[Dave Dodge <dododge@...>]

On Mon, Mar 13, 2006 at 11:32:59AM -0800, Mark Manyen wrote:

> Does anyone have some ideas on how to make the floating point
> problems between DX and Lua go away that is more constructive than
> use OpenGL?

Is it perhaps the known problem where Direct3D quietly messes with the
floating point unit in ways that affect other code:

    http://www.virtualdub.org/blog/pivot/entry.php?id=53

    Initializing Direct3D with default settings causes the precision
    bits in the floating-point control word to be reset such that FPU
    calculations always occur with 24-bit precision (single precision
    / float). This is much more serious as it causes roundoff errors
    to become much larger, and it means that double-precision math can
    no longer represent all valid values of a 32-bit int. For this
    reason, if you invoke Direct3D within an application that may not
    be expecting it, such as within a video filter, you should set the
    D3DCREATE_FPU_PRESERVE flag when creating the device.

Microsoft apparently has their reasons for doing this in Direct3D:

    http://discuss.microsoft.com/SCRIPTS/WA-MSD.EXE?A2=ind0504b&L=directxdev&D=1&P=4524&F=P

    The D3DCREATE_FPU_PRESERVE flag exists for those applications who
    want to set their own FP control word and/or stay with
    double-precision. You still must be very careful when setting it
    to non-standard rounding or enabling exceptions. For games, you
    typically don't want the performance hit of having the FP unit
    working in double-precision.

    [...]

    Using this flag with the FPU in double-precision mode will reduce
    Direct3D performance.

                                                  -Dave Dodge