Re: Lua 5.1

[Mark Hamburg <mhamburg@...>]

The mention of the 5.1 feature list inspires me to list out the places where
I've patched Lua in my current project in case its helpful to the broader
community:

* Added:

LUA_API void lua_pushuserdata (lua_State *L, void *udata) {
  if( !udata ) {
    lua_pushnil( L );
    return;
  } else {
   Udata *u = ((Udata *) udata) - 1;
    if( u->uv.tt != LUA_TUSERDATA ) {
      lua_pushliteral( L, "value was not a userdata" );
      lua_error( L );
      return;
    }
    lua_lock(L);
    luaC_checkGC(L);
    setuvalue(L->top, u);
    api_incr_top(L);
    lua_unlock(L);
  }
}

This allows one to take a value returned by lua_newuserdata and use it to
push a heavy userdata on the stack. It isn't safe since the object might
have been collected, but it doesn't seem inherently less safe than some
other things you can do from C. It avoids the need to have a table of refs
and a ref value inside the object and it makes the push a bit faster.


* Heavy userdata had some space in it's header given packing limitations, so
I added an lu_byte tag field. This makes typechecking heavy userdata a lot
faster (for up to 256 types) particularly if you can't guarantee shared
metatables because of a need to store per userdata links to other Lua
objects. To support this, I added the following APIs:

LUA_API int lua_setuserdatatag (lua_State *L, int idx, unsigned char tag);
    /* Returns 1 if successful and 0 if not a heavy userdata. */
    
LUA_API int lua_getuserdatatag (lua_State *L, int idx);
    /* Returns -1 if not a heavy userdata or returns the tag value. */


* I added an API for replacing luaD_throw and luaD_rawrunprotected. This
could also be handled by using macros for the names though having an API
allows for Lua being built in a separate library from the overrides (not
critical to what I'm doing but potentially interesting).


* I added a __methods entry to the metatable. This is much like Peter
Shook's __self patch on the Lua user's wiki.

The __methods table is essentially only accessible via the obj:method syntax
(though see below). The purpose to this is that it allows one to write
methods for userdata and know that the first parameter in the call is of the
appropriate type because it won't have been miscalled with obj.method and
can have been hidden via a protected metatable.

If the object has a __methods metatable entry, I don't even try the object
or the __index metamethod. The does mean that you can't override a method by
putting a field in a table, but it reflects the most common case and results
in faster execution. I thought about cascading through the __method table to
the object and then the __index function but this seemed like a weird
inversion of the object and the metatable, so it seemed cleaner to just
define it as not accessing the object.

obj:method(...) now essentially translates to:

    getmethod( obj, methodName)( obj, ... );

Where getmethod is defined as:

    function getmethod( obj, methodName )

        assert( type( methodName ) == "string" );

        local mt = getmetatable( obj );

        if mt and mt.__methods then
            if type( mt.__methods ) == "function" then
                return mt.__methods( obj, methodName )
            else
                return mt.__methods[ methodName ]
            end
        else
            return obj[ methodName ]
        end

    end

This is implemented in C to avoid protected metatable issues.
    
I also added Lua calls:

hasmethod( obj, methodName ) -- returns a boolean indicating whether or not
the object has the indicated method.

callmethod( obj, methodName, ... ) -- for cases where the obj:methodName
syntax can't be used

pcallmethod( obj, methodName, ... ) -- for completeness though one could
write:

    pcall( callmethod, obj, methodName, ... )

I'm thinking of adding something like:

methodclosure( obj, methodname ) -- returns a function that invokes the
method


* I modified the syntax error code to return a table containing the line
number and the error message together with an appropriate __tostring
metatable entry. This makes it easier to go to the error line within an
editor.


* I added support for a __write metatable entry that will always be invoked
even if the key is present in the table. This avoids the use of a proxy
table and the problems that causes for things like table iteration. I'm now
thinking that a better solution would be to introduce mode options for
tables that would allow them to be protected for reading and/or writing by
forcing access to such protected tables through the existing __index and
__newindex metamethods. I haven't looked into whether this can be made as
efficient as the current __write metatable entry approach which treats this
as a "fast" metatable entry.


* I added a shallowcopy function to luaxlib. For primitive values, it just
returns the value it was supplied. For tables, it copies the entries in the
table into a new table and copies the metatable. For heavy userdata, it
currently returns nothing but I could see adding a __shallowcopy
metatamethod to handle this. I found this particularly useful for copying
metatables since all metatable access goes through the raw accessor APIs. I
currently expose this as part of the table library and have it only support
tables, but I'm not entirely happy with that choice.

[ This can be written without modifying the Lua sources. It just proves to
be pretty useful. ]


I hope this list is helpful. As you can see, one of the big thrusts has been
toward making userdata as efficient as possible. This is because I'm trying
to build an object system in which almost everything runs through Lua and
it's important to keep things as cheap as possible to avoid temptations to
write stuff outside the system.

Mark

P.S. Do any of these changes make the language no longer Lua? I don't think
any of them break compatibility with existing Lua programs from either a
syntactic or semantic standpoint.