Re: Floating point

[Doug Rogers <rogers@...>]

Andrew Sidwell wrote:

> I'm wondering if anyone knows of a way to make sure a value is only to
> two decimal points?  I only ask because the obvious solution using
> (math.ceil(n * 100) == (n * 100)) doesn't work:
> > print(math.floor(3466) == math.floor(34.66 * 100))
> false
> It would also be quite nice to know why this doesn't work.

Ben's right - it's not easy to tell because floating point can only 
represent a fixed number of multiples of (possibly negative) powers of 
two. So things like 400.5 (801 * 2^-1) and 73.0625 (1169 * 2^-4), can be 
represented exactly, but not 51.2 (256 * 5^-1).

You can make the statement above return true by rounding instead of 
flooring:

> math.round = function(x) return math.floor(x + 0.5) end
> print(math.floor(3466) == math.round(34.66 * 100))
true

You can also determine if a value is within a particular epsilon of a 
multiple of 0.01:

> function ismul(x,base,epsilon)
>>   epsilon = epsilon or 0.0001
>>   return math.abs(x/base - math.floor(x/base)) <= epsilon
>> end
> print(ismul(45.02, 0.01))
true
> print(ismul(45.019, 0.01))
false
> print(ismul(45.020005, 0.01))
false
> print(ismul(45.0200005, 0.01))
true

Doug

--
--__-__-____------_--_-_-_-___-___-____-_--_-___--____
Doug Rogers - ICI - V:703.893.2007x220 www.innocon.com
-_-_--_------____-_-_-___-_--___-_-___-_-_---_--_-__-_