Re: Question about table arrays

[Mike Pall <mikelu-0601@...>]

Hi,

Rici Lake wrote:
> This would appear to restrict the array part of a table to 64 million 
> elements even on ILP64 architectures.

I asked this back then, when I sent my 64 bit patches:
  http://lua-users.org/lists/lua-l/2004-11/msg00032.html

I never got a reponse, but the define was quietly changed from
24 to 26. :-)

> So why 26?

floor(log(INT_MAX/sizeof(TValue))/log(2)) (for a 16 byte TValue).
Ok, so this limit only applies to 32 bit platforms. But there are
several places in the source which index the array part with an
int. This is a 32 bit type on LP64 (most POSIX) and LLP64 (Win64)
platforms.

I guess when you approach these limits (i.e. a single array with
a size of more than 1 GB), you might want to consider alternate
approaches (e.g. userdata). I had to do this because it's no fun
when the GC sweeps such a table ever so often (traversetable() is
atomic even in Lua 5.1).

[Alas, a non-atomic traversal doesn't help. One really needs to
completely avoid the traversal, because D-cache pollution kills
the performance. I'm not sure generational GC would help.]

> On the other hand, if the Lua instance were compiled with lua_Number as 
> a (single-precision) float, then the array part could grow to a larger 
> size than could be represented as an integral lua_Number, which seems 
> like it ought to trigger an error of some sort.

This is not a problem because you can only create number objects
for numbers which are representable with lua_Number. So even if
the array grows larger (with a sparse tail), you just can't set
or get these elements because you are lacking the proper key.

Not having a representation for contiguous integral numbers with
a precision of more than 24 bits may lead to problems with loops,
though. But I don't think there is any part in the Lua core or
standard libraries which fails in this case (all loops are done
with int32 cast to lua_Number).


BTW: Speaking of types and sizes ... anyone noticed sizeof(char)
sprinkled throughout the source? Funny enough, but this doesn't
accomplish anything. sizeof(char) is _always_ 1 (that's how
sizeof quantities are defined). One realls wants to check
CHAR_BIT (but the Lua core doesn't). I have some doubts that Lua
works on systems where this is not 8. So one might as well remove
the superfluous code.

Bye,
     Mike