Re: Formatting numbers without precision loss

[Lars Müller <appgurulars@...>]

You misunderstood the issue: The given number can be represented
precisely by a 64-bit float. In fact all integers up to and including
2^53 can be represented precisely by 64-bit floats. The issue is just
that conversion to string rounds "too much" by default, losing
precision. Thus the round-trip tonumber(tostring(x)) == x fails.

On 20.06.23 13:53, bil til wrote:
> > I’m running into the following issue (minimal example of a problem in a JSONschema validator):
> >
> > local maximum = 9007199254740991
> Isn't this very obvious that something like this will happen at larger
> float numbers? This is typical restriction for float.
>
> Check e. g. Online-converter Float -> hex, then you should see the
> problem more clearly (you do not define very well in your post, which
> bit resolution you use for your floating point calcualations, this
> depends strongly on your machine...).
>
> Due to this, for "counting values" like time or large encoder values,
> it is often advisable to use int's.