Re: Possible bug in MS_ASMTRICK? (Lua 5.2)

[Ross Bencina <rossb-lists@...>]

On 22/04/2013 6:26 AM, [ex] wrote:
> The problem I suspect is that the code under the MS_ASMTRICK is not
> correct (llimits.h:204):
> this block:
>      __asm {__asm fld n   __asm fistp i}
>
> that is used in:
>      #define lua_number2unsigned(i,n)  \
>           {__int64 l; __asm {__asm fld n   __asm fistp l} i = (unsigned
> int)l;}
>
> it's not guaranteed to floor a number, it depends on the FPU rounding mode.
> as explained here: http://stereopsis.com/FPU.html

It is true that the behavior of fistp depends on the current rounding mode.

However I think you will find that the default rounding mode is usually 
fixed by the platform ABI (or even the C language spec?) so the behavior 
should at least be predictable.

Don't try adding .5 to fix this either, since round to nearest on 1.5 
and 2.5 etc rounds in opposite directions.


> I suspect this is a bug because the slow alternative is not giving me
> the same result, but I can be missing something.

If you have an SSE capable processor I think there is a fast alternative 
using SSE.

Some discussion here:
http://www.musicdsp.org/showone.php?id=246

Ross.