[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: why 0.1+0.2-0.3~=0
- From: Dirk Laurie <dirk.laurie@...>
- Date: Tue, 13 Mar 2012 10:56:05 +0200
>> in my program and on the website http://www.lua.org/cgi-bin/demo
>>
>>
>> tmp=0.1+0.2-0.3
>> print(tmp)
>>
>> gives
>>
>> 5.5511151231258e-17
>>
>
Because 0.1 is not exactly 1/10, 0.2 is not exactly 2/10,
0.3 is not exactly 3/10. Those "not exactly" terms
sometimes cancel out but usually do not. You can
can see it if you print out more than 16 digits.
> print(string.format("%.17f",0.1))
0.10000000000000001
> print(string.format("%.17f",0.2))
0.20000000000000001
> print(string.format("%.17f",0.3))
0.29999999999999999
