Re: C binding strategy

[Martijn van Buul <martijn.van.buul@...>]

On 8/14/2012 7:26 AM, Francisco Tolmasky wrote:
> There are a few "small" object types that I need to occasionally
> pass over to the C side of things. For example, lets say a Point
> class (containing x and y). These points get used internally by the
> C code to do graphics stuff, etc (or say Quaternions, etc.). The
> point is, I'm referring to simple objects that you might use a struct
> for, would rarely have a pointer for in C or a non-trivial lifetime
> like an "Image" class or something. I've devised two strategies:
>
> 1. Create a traditional userdata-style object that I expose to Lua.

This is what we did. I added Lua to an existing project, which already 
had a way of passing data (based on a reference-counted boost::any, but 
modified considerably to speed things up). The idea was that data could 
travel freely from Lua to C++ and back, without it being copied more 
often than strictly neccesary. Together with some template- and macro 
glue this makes for bindings that are relatively easy to write (and 
read). For example, the C function handling the __add metamethod of our 
Point-class just reads

  int Add(lua_State *L)
  {
    // Can only add two CEPoints.

    // ELuaBindings::ToCEPoint() will raise luaL_error if the
    // arguments are of the wrong type.

    const CEPoint *a = ELuaBindings::ToCEPoint(L, 1);
    const CEPoint *b = ELuaBindings::ToCEPoint(L, 2);

    CEPoint *result = ELuaBindings::PushNewCEPoint(L);
    *result = *a + *b;
    return 1;
  }


The overhead in terms of "amount of boilerplate code" is limited, the 
actual *runtime* overhead is acceptable, but what causes some concern is 
memory usage. The userdata only contains a refcounted pointer, 
irregardless of the size of the data pointed at by that pointer. Simply 
put, the garbage collection fails to realise that there is a difference 
between a "Point" and an "Image", and only "sees" the memory burden of 
the pointer. It simply doesn't realise that the memory footprint of a 
userdata may be bigger than the size of the userdata itself, and when 
left to its own devices will quickly cause memory to run out - even 
though the memory 'consumed' by the lua state will remain low.

So far, the only way we found around this is to manually call the 
garbage collection on a (very) frequent basis. However, this causes a 
significant additional runtime; it is not uncommon to see the garbage 
collection take more time than the actual payload.

Kind regards,

Martijn