Re: [patch] fix luai_num* null states

[Roberto Ierusalimschy <roberto@...>]

> On Tue, Mar 4, 2014 at 3:34 PM, Roberto Ierusalimschy
> <roberto@inf.puc-rio.br> wrote:
> >> > I had another small issue on luaV_numtointeger(). I had an 'integer
> >> > overflow' on the line 81 of lvm.c. I solved as follows:
> >> >
> >> > -  if (cast_num(MIN_INTEGER) <= n && n < (MAX_INTEGER + cast_num(1))) {
> >> > +  if (cast_num(MIN_INTEGER) <= n && (n - cast_num(1)) < MAX_INTEGER) {
> >>
> >> This is tricky... In my machine, this change does not pass the standard
> >> tests for vanilla Lua (64-bit floats and integers). (I am trying to figure
> >> out why, but when I add some printfs to the code, it works again; weird...)
> >
> > With the new code, the machine does all computations for
> > (n - cast_num(1)) < MAX_INTEGER in registers, with more precision than
> > 64 bits. We do need the "overflow" here to get the right result.
> > (Think when 'n' is 2.0^63, which does not fit into a 64-bit integer.)
> 
> Sorry,  it really sounds tricky.. I don't get it yet =(.. in that
> case, we should have a float comparison, don't we? Thus, the '<'
> expression would return 0, the 'if' test would fail and the function
> would return 0, which should be the right result for n = 2.0^63. Is
> that wrong? I must be missing something.

MAX_INTEGER is stored as a 64-bit float constant, so it is rounded
to 2^63. Then, with the computation done with 80 bits, we have
(2^63 - 1 < 2^63) done precisely, resulting in true. (In the original
code, the whole (MAX_INTEGER + cast_num(1)) is stored as a 64-bit constant,
correctly rounded to 2^63. The computation now is 2^63 < 2^63, which is
false.)


> BTW, these tests are publicly available for Lua 5.3?

I think they are not. We are about to release a new work verion in the
following weeks; we will make the tests available then.

-- Roberto