lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index]

In Pallene we try from "%.6g" to "%.17g" and pick the smallest one that round-trips to the original number. (We can't use %a because our output is C, not Lua). We need special cases for +∞ and -∞. It doesn't work for NaNs. We also care whether the number ends up as a float or integer literal, so we check if %g gave it a decimal digit or exponent.
https://github.com/pallene-lang/pallene/blob/b1c0c87b749a3a9e4ebfd500a1a08af53492b8fc/src/pallene/C.lua#L56 

-- Hugo

Em 20/06/2023 08:18, Thijs Schreijer escreveu:

Hey list,

I’m running into the following issue (minimal example of a problem in a JSONschema validator):

local maximum = 9007199254740991
local code = "return function(val) return (val <= "..tostring(maximum)..") end"
local func = load(code)()
print(code)
print(func(maximum))
print(func(maximum+1))

Results in:
return function(val) return (val <= 9.007199254741e+15) end
true
true

The second “true” is wrong, since it actually is 1 greater than the maximum.
Because of the exponential notation used in the generated code, precision is lost, causing this mistake.

Trying again, using a float formatting argument;

local maximum = 9007199254740991
local code = "return function(val) return (val <= "..("%.0f"):format(maximum)..") end"
local func = load(code)()
print(code)
print(func(maximum))
print(func(maximum+1))

Results in:
return function(val) return (val <= 9007199254740991) end
true
false

Works. (I also tried “%d” instead of “%.0f”, but that rolled-over to a negative number)

Now all of this is probably highly system/compile-time-config dependent. I get that.
But keeping in mind that “value” that is passed in is represented in the system as a proper value, how can I format the number in the code such that does not loose any precision in it’s journey from; number => string => lua-parser => number ?

Not just for ints but also floats? (And independent of Lua version, above was on PuC-Rio Lua 5.1)

Any help is greatly appreciated.

regards
Thijs