Re: problem with Visual Studio

[Peter Cawley <lua@...>]

Changing L to LL has no effect in this case. Also interesting is that
in my test matrix of {VS2010, VS2012, VS2013} x {x86, x64}, only
VS2010 x86 exhibits the wrong behaviour. Said wrong behaviour seems to
be caused by a call to the CRT's _ftol2 routine, which nominally
converts from floating point to integer - I would initially guess that
either _ftol2 is at fault for not handling large numbers properly, or
the compiler is at fault for generating code to call the wrong
conversion routine.

On Tue, Apr 15, 2014 at 10:22 PM, Coroutines <coroutines@gmail.com> wrote:
> On Tue, Apr 15, 2014 at 2:13 PM, Roberto Ierusalimschy
> <roberto@inf.puc-rio.br> wrote:
>> Can someone with more experience with MS check this?
>>
>> Consider the next program:
>>
>> /*---------------------------------------------------*/
>> #include <stdio.h>
>> #include <windows.h>
>>
>> int main (void) {
>>   unsigned __int64 x = 0xFFFFFFFFFFFFFFFFL;
>>   double d = (double)x * 0.9;
>>   unsigned __int64 y = (unsigned __int64)d;
>>   printf("%I64x %I64x  %g\n", x, y, d);
>>   return 0;
>> }
>> /*---------------------------------------------------*/
>>
>> 'x' is the maximum value allowed for its type, 2^64-1. Then it is
>> converted to double, and rounded to 2^64. The product with 0.9 gives
>> a number "much" smaller than 2^64 (~1.6e19 versus ~1.8e19) which,
>> when rounded, clearly fits in 'y'. However, when I run this program
>> in Visual Studio 2010, it prints 'y' as 8000000000000000, signaling
>> an overflow.
>>
>> (I need this casting, so that math.random(math.mininteger, math.maxinteger)
>> can give its full range of values.)
>>
>> Thanks in advance,
>>
>> -- Roberto
>>
>>
>
> I'm just taking a wild guess here, but don't you need long long for
> that 0xFFFFF... literal?  "0xFFFFFFFFFFFFFFFFLL" ?
>
> I thought long int on Window x64 was still 32 bits (don't you need
> long long for 64 bits?), I was looking for the wikipedia article on
> data models for different operating systems and I couldn't find it :(
> Maybe I'm wrong and this is standard per x86_64.
>