Re: LUA_NUMBER -> Integer conversion bug

[Mike Pall <mikelu-0603@...>]

Hi,

Sebastian Rohde wrote:
> I can see your point but I don't think that lua handles this problem
> correctly. As soon as lua does "conversion tricks" that assume the
> coprocessor to be in a specific state lua has to take care about being in
> that state AND recovering the old state afterwards. This is what you expect
> from DirectX and what DirectX is does using D3DCREATE_FPU_PRESERVE. 

No. The "standard" FPU state is very well defined by various
documents. The most notable are the x86 ABIs for POSIX and
Windows. The former specifies fpucw = 0x37f, the latter specifies
fpucw = 0x27f. They differ only in the default precision setting
(extended vs. double).

Microsoft choose to deviate from this with DirectX by leaving the
FPU state set to "float" precision. Blame them, not Lua. In fact
D3DCREATE_FPU_PRESERVE really should've been the default setting.
Even back then when it mattered (on a Pentium I or II without GPU
acceleration). If you choose to live dangerously then you should
get the option, but not just by accident.

> Everything that does an implicit assumption concerning the FPU state (which
> I did not find noted in any doc yet?) has to be considered as a bug, imho.

This is not just about "conversion tricks". Every compiler
generates tons of code which relies on the implicit assumptions
from the ABI. Many math functions will produce invalid results if
the precision or the rounding mode is not the default. Changing
the default FPU state is a really bad idea.

Anyway, you would've gotten wrong results even with earlier Lua
versions, except under different circumstances. Try this:

$ lua -e 'local y=987654321; print(y, y+0)'
987654321	987654321

$ lua -l setfpu -e 'local y=987654321; print(y, y+0)'
987654321	987654336

Here "setfpu" is a library which sets the FPU state to float
precision, just like DirectX without D3DCREATE_FPU_PRESERVE. IMHO
finding out about this problem earlier with Lua 5.1 is a bonus.

Bye,
     Mike