Re: Lua Chunk Type

[nobody <nobody+lua-list@...>]

>> What you can do (if you don't care about upvalues - these are not 
>> stored), is to save the compiled function via lua_dump[1] and then 
>> load it again later.  That will reduce the time needed to load it 
>> again (no parsing of Lua code) and may be useful across several 
>> runs of the program (or if you want to send the same function to 
>> several states). It's generally not a good idea to do this during
>> a single run.

> My desire is to use the same chunk with multiple states, so it
> sounds like your suggestion of using lua_dump makes sense.  I do have
> a few questions (I'm still a novice on the API side of Lua):

The first question to ask yourself:  Does the (small-ish) speed-up
matter?  Are you creating a dozen or so states? (don't bother, unless
the Lua chunks are HUGE) Or thousands of states? (then it's probably a
good idea)

> 1) How do upvalues play a part with lua_dump?

It shouldn't matter in your case, but here's an explanation anyway:

  local x = 1
  return function( )
    local y = x
    x = x + 1
    return x*y
  end

Here, x is an upvalue.  It's a local variable sitting outside the
function.  (And it keeps its value across invocations.)  The way things
work, there's no general safe way to save its value when dumping the
function returned from this chunk, so it simply isn't stored at all and
initialized to 'nil' when you load the dumped function.  Upvalues could
be shared between functions, and that property would be lost.  Upvalues
could also refer to "complex" data types, including computed contents
etc. - a pretty ugly example:

  STACK = { }
  local stack = STACK
  local insert, remove = table.insert, table.remove
  local actions = {
    push = function( v )  insert( stack, v )  end,
    pop  = function( )  return remove( stack )  end,
  }
  return function( cmd, ... )
    local action = actions[cmd]
    if action then  return action( ... )  end
    error "invalid command"
  end

(There's no sane way to dump & restore the upvalues of the returned
function without knowing what should happen.  If you're just beginning
Lua, it may take a while until you understand all that happens in
there... Fun question:  What is an upvalue of what function(s) and
what's a plain local variable?  (You can check with debug.getupvalue -
it also tells you the names.)  Keep in mind that when you load() this
code, you have to execute the result to get the function that it
returns.  Otherwise, you'd be poking at the file's main chunk and not
the contained functions.  And that bit is what should make this
irrelevant in your case:)

If you're just loading a file (or string) and then dump that, it has no
external upvalues.  (Well, it has exactly one - _ENV - but Lua knows how
to handle that one and will initialize it properly for you.)

> 2) I see that lua_dump requires a Lua writer.  Is this something I 
> have to create myself?

Yes/No - depending on how you do it.

No - if you can just call 'string.dump'.  (You can use that from C
(after opening the Lua libraries) via getglobal "string", getfield
"dump", pushing/moving your function to the top of the stack, and then
lua_call-ing with one argument and one return value.  It's also easy to
wrap that up in a function.)

Yes - if you want to do it from C without using the Lua-side stdlib (but
it's still quite simple).  You can look at lstrlib.c for a writer that
creates a string (using the auxlib's luaL_Buffer).  You can also look at
luac.c in the Lua sources for a writer that just dumps things into a file.

> 3) How do I call the state with the dumped chunk?

Did you mean: How do I turn the dumped function representation back into
a function?

Simple: You load it just like Lua code, e.g. by luaL_loadbuffer (not
luaL_loadstring, because of embedded \0 bytes), and then you have the
function on the Lua stack and can proceed as you did before.

> 4) Is there an example of this you can point me to?

In general: The Lua stdlib is quite readable and contains many
interesting examples.  Just look at any l*lib.c files in the source.  In
particular, lbaselib.c contains load, loadfile, ... as luaB_load,
luaB_loadfile, ... and lstrlib.c contains string.dump as str_dump (with
the 'writer' just above it).

In particular on dump&load:  I couldn't find a good one focusing on this
in the Lua wiki, so here's a mediocre example that opens 16 states and
recreates a function on each:

/* ----- begin example ----- */
/* This example code hereby placed in the public domain,    *
 * no attribution required, no warranty, etc. blahblah...   *
 *  -- nobody, on Cfn 72, 3182 YOLD                         */
#include <lua.h>
#include <lualib.h>
#include <lauxlib.h>

/* should probably be an external file, an auto-generated header, ... */
static const char *code =
  " x = (x or 0) + 1"
  " print( ('Hi, #%d here on call #%d.'):format( id, x ) )";

#define N 16

int main( void ) {
    int i;
    const char * dump;
    size_t dump_len;
    /* open a bunch of states */
    lua_State *L;
    lua_State *Ls[N];
    for (i = 0; i < N; i++) {
        if (NULL == (Ls[i] = luaL_newstate( ))) {
            return 1; /* crappy & short exit */
        }
        luaL_openlibs( Ls[i] );
        lua_pushinteger( Ls[i], i );
        lua_setglobal( Ls[i], "id" );
    }
    L = Ls[0]; /* our "master" */
    /* load function on first state & dump it */
    luaL_loadstring( L, code );    /* f:code */
    lua_getglobal( L, "string" );  /* f:code t:string */
    lua_getfield( L, -1, "dump" ); /* f:code t:string f:str_dump */
    lua_remove( L, -2 );           /* f:code f:str_dump */
    lua_insert( L, -2 );           /* f:str_dump f:code */
    lua_call( L, 1, 1 );           /* s:code_dumped */
    dump = lua_tolstring( L, 1, &dump_len );
    /* now you could malloc a buffer and copy the string out, or else
     * you have to leave it on the stack until you're done with it */
    /* replicate the function everywhere */
    for (i = 0; i < N; i++) {
        luaL_loadbuffer( Ls[i], dump, dump_len, "code" );
        lua_pushvalue( Ls[i], -1 );
        lua_setglobal( Ls[i], "greet" );
    }
    /* ok - get rid of the string */
    dump = NULL;
    lua_remove( L, 1 );
    /* test runs */
    for (i = 0; i < N; i++) {
        lua_getglobal( Ls[i], "greet" );
        lua_call( Ls[i], 0, 0 );
    }
    for (i = N-1; i >= 0; i--) {
        lua_getglobal( Ls[i], "greet" );
        lua_call( Ls[i], 0, 0 );
        lua_close( Ls[i] );
    }
    return 0;
}
/* ----- end example ----- */

Things to note:

 *  It's been a while since I last used Lua from the C side (and it's
    also been a while since I last used C), so there may be bugs. The
    thing compiled with -Wall -Wextra, Lua API checks enabled and then
    some and nothing complained, so any remaining bugs should be
    relatively well-hidden.

 *  I'm storing things in the Lua state's global environments, which
    you may want to avoid if you're not "alone" in those states.

 *  Any error would bring this thing down the hard way.  Better to run
    a "protected main" (like lua.c does), then you can decide to
    (fixup|pray)&retry | cleanup&exit | ...

 *  It's also a good idea to pcall into Lua functions.  The checks etc.
    would make this example even longer, so I left that out.  Again,
    see lua.c - it abstracts away the details for its purposes and does
    a lot of other fancy stuff that's worth reading.  (Same for the Lua
    stdlib, which is entirely implemented in terms of the API.)

 *  I aliased my "book-keeping" state L to Ls[0], using it for several
    purposes.  If you have really many states, having an extra state
    just for book-keeping is a good idea.  (It separates concerns, and
    you might not even need to open the libs on it.  A Lua state is a
    nice & flexible data structure and can be used just as that.)

 *  That you now have the example above doesn't mean you have to use it.
    In most situations, the speedup isn't worth the increase in
    complexity.  In this case, simply doing luaL_loadstring on each of
    the states would get rid of ~40% of the code, the slow-down is not
    noticeable here.  (Heck, for this example of running short code just
    twice on a meager 16 states I could get away with luaL_dostring and
    you wouldn't notice.)

> It might be helpful to add that I am currently developing on an 
> embedded system with no operating system.  This will eventually run 
> on a PC run Windows and/or Linux.

Are you _severely_ memory-constrained or is the processor _really_ slow?
 If no, don't bother optimizing too much (especially if you're headed
for "normal" computers.)