A warning about struct/pack modules and suggestion for public userdata.

[Tom N Harris <telliamed@...>]

Anyone using struct[1] or pack[2] modules should be aware that neither 
of them check for stack overflow before unpacking. This is a problem if 
you will be unpacking more than 20 values (or whatever LUA_MINSTACK is) 
or especially if the unpack string can come from someplace else.

    s = string.rep('b',10240)
    unpack(s,s)
    -- core dumped

The obvious fix is to pre-scan the format string and call lua_checkstack 
appropriately. Of course I've done this in the struct module I'm using 
for Larc[3].

And as I write this, I'm getting an idea for a different way to access 
packed data using tables. You compile a description of the data, which 
looks something like a C struct definition but with slightly more logic 
(a Pascal string could be "uint8_t len; char[len] name;") When attached 
to a string it can act like a table to unpack fields individually. But 
at the moment I don't want to be distracted from Larc. Maybe some other 
time or if anyone else wants to use the idea.

Part of my struct library deals with 64-bit integers. I'm sure I'm not 
the only person doing it and it seems silly for every instance of a 
library to create incompatible userdata that are all essentially the 
same thing. Since most metatables are registered already, I think it 
would helpful if common formats were published so anyone who wants an 
8-byte block of memory for storing a 64-bit integer in native word order 
will know that it's called "large integer" (Though a better name should 
probably be used.) Same with a complex number, or matrix, for example.

How should common userdata be constructed? Something simple like a large 
integer can just be done by hand. But perhaps there's some types that 
need more control over the creation? It may be worthwhile to use a 
metamethod __new that allocates the userdata. But should __new also be 
responsible for initializing? I think it's simpler to leave that to the 
application and __new simple returns a userdata ready to be written to.

There's also the matter of who is responsible for the metatable. It 
could be first-come where a module that implements a type should do 
nothing if the metatable already exists. Alternately, you could be free 
to replace the metatable with your own. Either way, the user should be 
able to load modules in any order.

To do this we neeed a list of metatables names, guidelines for 
describing userdata, and recommended best practices for libraries. 
Anything else?

[1] http://www.inf.puc-rio.br/~roberto/struct/
[2] http://www.tecgraf.puc-rio.br/~lhf/ftp/lua/#lpack
[3] http://code.google.com/p/lua-larc/source/browse/trunk/struct.c

--
- tom
telliamed@whoopdedo.org