[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: lua internals debugging hints? : lua_number2integer() not working
- From: Mike Pall <mikelu-0510@...>
- Date: Fri, 21 Oct 2005 18:28:22 +0200
Hi,
David Burgess wrote:
> I presume the number2int that fails is the arithmetic version not
> the inline assembler. Or do both fail?
Right. Because adding the magic constant 2^52 + 2^51 cuts off
small integers when single-precision (24 bits) is selected.
The inline assembler variant would still work, but I guess it
was too problematic with different compilers.
And this doesn't solve other failures caused by DirectX messing
with the FPCW (signalling and denormals handling).
I guess even something trivial like:
for i=1,1e9 do end
won't terminate due to the precision loss (I haven't tried
though).
Bye,
Mike