Re: Feature request: more integration of Lua and C memory management

[Mark Hamburg <mhamburg@...>]

I've just sketched out (but haven't tested) a small set of routines that
supports hybrid objects combining userdata and a Lua table:

void initHybridSupport( lua_State *L );
    /* Initializes the hybrid object support system. (Mostly just
    a few tables.) */

void *makeHybrid( lua_State *L, size_t udsize, lua_CFunction gc );
    /* Creates a hybrid userdata/table object. Leaves the table on the top
    of the stack and returns a pointer to the userdata. udsize is the size
    of the userdata. gc is the GC function for the userdata. */

void *checkHybridUserdata( lua_State *L, int index );
    /* Get the userdata for the hybrid table at the given index. Throws
    on failure. */

void pushHybridTable( lua_State *L, void *ud );
    /* Push the table portion of a hybrid based on the userdata portion.
    Throws if the table doesn't exist. */

void retainHybid( lua_State *L, void *ud );
    /* Arranges for the table portion of a hybrid to be retained. This
    does not implement reference counting and any number of retains can
    be balanced by a single release. Throws if the hybrid lacks a
    table. */

void releaseHybrid( lua_State *L, void *ud );
    /* Counter one or more calls to retainHybrid. */

If you want to attach a metatable to the hybrid -- e.g., to support methods
-- you do so via the table portion. Any C methods, can then use
checkHybridUserdata to get their data.

If you want reference counting, you can implement in the userdata and call
retainHybrid on a 0-to-1 transition and releaseHybrid on a 1-to-0
transition.

I haven't tested the code, but it's pretty simple. The chief easy extension
to Lua that would benefit it is a version of raw get that takes a light
userdata as a parameter to avoid one push and possibly to optimize some of
the table lookups. That wouldn't be worth doing until it was demonstrated to
be a significant portion of the runtime.

Here's a rundown on the structure.

UD-to-Table table: A weak-valued table mapping the userdata values as light
userdata to their table counterparts.

Table-to-UD table: A weak-keyed table mapping tables to userdata values.
This is how checkHybridUserdata works. Alternatively, with a bit less safety
one can just use an entry in the table itself.

Retain-table: A table mapping light userdata versions of the userdatas to
their tables.

Metatable-cache: A table mapping GC functions (as light userdata) to
metatables. One could make this weak-valued if data types were likely to
disappear. Note that this cheats by converting a function pointer to a void*
which isn't strictly legal C. One would need a separate key otherwise.

The routines are pretty much straight line code to do the things needed to
maintain these tables. Several pieces could be reused in a broader system
that supported retaining userdata and pushing userdata from the pointers.
The basic costs look to be:

* A hybrid object incurs two table entries one in the UD-toTable table and
one in the table-to-UD table.

* Pushing a hybrid object consists of a light userdata keyed registry lookup
followed by a registry lookup on the table on the stack followed by a check
followed by a remove.

* Running a method for a hybrid object consists of a lookup in the table, a
raw lookup in the metatable to get the __index entry, and a lookup in the
__index table (assuming it's a table and not a function). A C method may
then want to get the userdata which costs a light userdata keyed registry
lookup, a data copy within the stack, a rawget, a conversion to a userdata
pointer, and popping two entries.

This does not make values private to the userdata. To do that, one has to
resort to proxy tables and per object metatables.

Mark