Re: problem with string.format %d and very large integers

[Norman Ramsey <nr@...>]

 > > What are your thoughts about using int64?  Not ANSI?  Not performant?
 > 
 > Both. Moreover, it does not solve all cases. For instance, fseek is
 > limited to long in C.

Ouch!  OK, thanks for the clarification.

 > As pointed out in the case of fseek, the big problem is the interface
 > with C. And documenting everything would be messy; the limits are not
 > even the same for every function (e.g., int x long x size_t).

Agreed.

 > Although it was me who came with the "widespread" argument, I guess it
 > is not that widespread in pragmatic terms. As Dirk pointed out, it is
 > rare (or impossible, in Lua) to have too large structures, so in real
 > programs seldom (or never) a real index will be larger than 2^31 or
 > whatever the limit. It may be more useful to collect real scenarios
 > where this problem actually happens (e.g., format, which applies to
 > generic numbers, not only indices or sizes) and try to solve them
 > (e.g., with error messages).

I like the sound of this plan.  Part of it is that Lua is intended as
a safe language.  And at many of these boundaries, the cost of error
checking is going to be small compared with the cost of an operation
like sprintf() or fseek().   I'm hoping that the relevant tests might
look like this:

  if ((LUA_NUMBER)(int) n != n)
    lua_error(L, "numeric overflow in string.format(... %d ...)");

or

  if ((LUA_NUMBER)(long) n != n)
    lua_error(L, "numeric overflow in file:fseek(...)");

On modern hardware with good branch prediction these tests will have
good instruction-level parallelism, plus the cost of the test is small
compared with the cost of the operation the test protects.  And the
test is likely to be done entirely in the floating-point unit.  

I would be totally happy with a failure mode that calls error()
instead of silently using only the least significant N bits.


Norman