Re: Lua persistence library?

[Chuck Messenger <cmessenger@...>]

From: Luiz Henrique de Figueiredo <lhf@tecgraf.puc-rio.br>

> >lua_save() would write the existing VM state to buf
> >lua_restore() would regenerate the Lua VM state from the
> >same buffer.
>
> Saving VM state is not the same as saving a lua_State, or is it?
> In 4.1, you wil be able to have several "threads" sharing the same global
> environment, and so the two things will be different.
>
> Perhaps you want to save VM state simply to run someting else for a while and
> then switch back to the old state. If that's the case, then 4.1 is for you!
>
> Do you really want to save the state of the VM, ie, what function is
> executing and where it is and so on? Or do you simply want to save the global
> environment, ie, global variables and their values? In the second case, the
> easiest way would be to output Lua code that restores it when loaded. There
> are examples for this in lua/test. However, even this is not sufficient 
> because
> C functions cannot be saved (perhaps saving the names of the global variables
> that have C functions would be sufficient for some applications).
>
> In summary, saving state is in principle an attractive and useful feature,
> but it is ill-defined and probably too costly if done in full generality.
> The solutions I'd like to see for it would use precompiled chunks because
> the format is already defined and the code for loading them is already in 
> place.
> In other words, lua_restore would be simply lua_dofile.


I only have in mind saving the state in the "outermost" level.
That is, I don't want to save the state during a C function callback,
but only outside of a lua_dobuffer().  That is, for example:

L1 = lua_open();
lua_dobuffer(L1,...)
lua_dobuffer(L1,...)
....
lua_save(L1, S1);
....
L2 = lua_restore(S1);
lua_dobuffer(L2,...)
...
L3 = lua_restore(S1);
lua_dobuffer(L3,...)
...


Like that.

There are, admittedly, some sticky problems to solve with
binary values which interface between Lua and C.  Just doing
a cursory examination, I see two types of binary values Lua
gives to C:

    tags            -- as generated by lua_newtag()
    references      -- as generated by lua_ref()

These same values would be restored by lua_import().  It
would be up to the C environment to remember what these values
were, so they could be restored in the C environment after
lua_import().  So these are no problem.

I see two types of binary values which C gives to Lua:

    user objects    -- as passed to lua_pushusertag()
    C functions     -- as passed to lua_pushcfunction()

Lua must give symbolic names to these binary values when
exporting the state, then restore appropriate (different)
binary values when importing it.  How can we accomplish this?

I suggest that a lua_extern_t structure be used:

struct {
    char *name;
    void *addr;
} lua_extern_t;

int lua_export(lua_State *L, char *buf, int len,
     lua_extern_t *extern, int nextern);

lua_State *lua_import(char *buf, int len,
     lua_extern_t *extern, int nextern);

(I changed the proposed names from lua_save() and lua_restore() to
lua_export() and lua_import()).

Before calling lua_export, we must construct a vector of
lua_extern_t's -- one for each C-defined external object.
We can safely, I think treat function addresses the same
as user data for this purpose (just to simplify the
interface a bit).  When lua_export() finds an external C
reference (i.e. either a C function or user data address),
it looks up the address in extern[].  It saves the
corresponding name (all the names in extern[] must be
unique) in the export buffer.  If it doesn't find the address
in extern[], then lua_export() fails.

On the other side, we must pass lua_import() the same
extern[] vector, but this time, filled in with the new
addresses (for the new C environment).  lua_import()
checks that we've passed the same length extern[], with
the same names, as we gave lua_export().  It uses the
corresponding extern[].addr field as the new C address
(for the C function or user data).

The entire Lua state would be saved, including the
lock status of objects, tag methods, etc.

So, how's that?  Then, the whole Lua state can be saved
and restored -- ideally, cross-platform.

Why do I want this?  Basically, in my app, I will have
an "evolving" state, which needs to be saved to disk,
and passed around to different computers.  An app will
load a Lua state, modify it some (with some user input),
then save the state to pass to another person/app, who
will restore the state, modify it, pass it along, etc.

Other people might find this functionality useful,
too.


    - Chuck