[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: Formatting numbers without precision loss
- From: Thijs Schreijer <thijs@...>
- Date: Tue, 20 Jun 2023 14:00:12 +0200
> On 20 Jun 2023, at 13:53, bil til <biltil52@gmail.com> wrote:
>
>> I’m running into the following issue (minimal example of a problem in a JSONschema validator):
>>
>> local maximum = 9007199254740991
> Isn't this very obvious that something like this will happen at larger
> float numbers? This is typical restriction for float.
>
> Check e. g. Online-converter Float -> hex, then you should see the
> problem more clearly (you do not define very well in your post, which
> bit resolution you use for your floating point calcualations, this
> depends strongly on your machine...).
>
> Due to this, for "counting values" like time or large encoder values,
> it is often advisable to use int's.
Yes, but keep in mind, the original number was already present in the same machine!
The question is not about infinite precision, just about reproducing the exact same value I previously had in the same system.
“%a” seems to solve that, if available.
Thijs