Re: [ Userdata ] __index aware of original lookup value

Dear Miss Tuma,

This looks like exactly what I wish vanilla lua would do[1]. Unfortunately, I don't have control over the lua distro (I'm making a general purpose library that works across lua versions and with luajit for users other than myself). I would hope that one day, this becomes part of core Lua (I would make the petition, but I'm new here and I honestly have no idea who controls such a thing/who I need to talk to).

I would imagine the best way to add it would be for lua to essentially provide a function `(lua_)lookup_value` (name pending a better bikeshed than in the deep of night) that pushes the lookup value on the stack / returns the value that originally failed lookup. Adding a third argument might disturb older code who perform switch statements based on the arity (int argcount = lua_gettop(L)) that lua provides the metamethod.

[1] That looks like a super interesting project, though! Have you also considered things like implementing a new garbage collector for `ljx` too, since the one the original lua (and luajit) have implemented is essentially one of the earlier and older models?

On Sat, Feb 27, 2016 at 1:05 AM, Karel Tuma <kat@lua.cz> wrote:

Excerpts from JeanHeyd Meneide's message of 2016-02-17 07:38:37 +0100:
> cue long plea for efficient single inheritance in lua

Long ago I've encountered this too. Note that this is a general problem, not
limited to your userdata approach - it happens anytime you need to do single
inheritance without __index wrappers.

You have only two options

1) Have function() handler for each __index handler in the chain, who will track
the original value for you (slow!)

2) Patch lua to pass this value as third argument

I did the 2) for LuaJIT [1] (should apply to all versions of luajit cleanly).

Here is how it works:

pintsize:~/ljx/src$ cat test.lua
a={}
b={}
setmetatable(a,{__index=function(t,k,t2) print(t,k,t2,t==a,t==b,t2==a,t2==b)end})
setmetatable(b,{__index=a})
test=b.x
pintsize:~/ljx/src$ lua5.1 test.lua
table: 000000000089bbf0 x nil true false false false
pintsize:~/ljx/src$ ./luajit.exe test.lua
table: 0x008eb0c0 x table: 0x008ead70 true false false true
pintsize:~/ljx/src$

Note that modifying original rio lua is probably not worth it, as the speedup will
be only minor, unless you happen to have very long __index chain.

On LuaJIT it can make huge difference though.

[1] https://github.com/katlogic/ljx/commit/22b7bab96031d6473c5611b24219011701bb9972

> I hope this message finds you well.
>
> I am developing a lua <-> C++ binding (like the several other that
> exist), and I've made significant headway into getting a very performant
> and easy to use wrapper. My final challenge in increasing the performance
> comes from trying to make basic "variable" get/sets on userdata made by C
> code work.
>
> The Question: Is there a way to get the original object that a lookup
> was performed on in a cascading series of `__index` queries? Right now, lua
> hands you the current object in the `__index` cascade that lookup failed on
> and the key name it is trying to find. This is all well and good and based
> on the specification, but my problem is that I want what the original
> lookup failed on (e.g. the userdata that started it all, in my case).
>
> This tree might help explain what my dilemma is (pseudo code):
>
> myuserdata -- userdata
> myuserdata.mt -- metatable
> myuserdata.mt.__index -- table
> myuserdata.mt.__index.mt -- second metatable
> mysuserdata.mt.__index.mt.__index -- function that performs variable lookup
>
> When I do
> myuserdata.x -- cascades down to second function, but the provided "lookup
> failed" argument is userdata.mt.__index, the table, not myuserdata. How do
> I get to 'userdata' ?
>
> Details: I used to have it working by overriding the `__index`
> metamethod of the userdata and then connecting a C function to it. The C
> function had a list of all the functions and variables for that type, and
> dispatched accordingly. Because the `__index` lookup failed on the
> userdata's metatable (I only had 1 metatable on the userdata), I would get
> the userdata as the "lookup failed on this" object. Calling my C function
> to get either a variable (instance.a) or a function (instance:bark) had
> worked because I could establish the proper context with the userdata and
> go forward. Unfortunately, this approach was about ~2x slow thanks to
> having to route requests through the C function, taking a significant
> performance hit to not just looking up member variables but calling member
> functions from lua that showed heavily in all benchmarks.
>
> I made things faster by changing the userdata's metatable to a table
> instead of a C function. The table had all of the non-metamethods that I
> wanted to provide for the userdata I had created in C. This worked, but I
> still needed something to hold the variables that would get found
> eventually by an `__index` call. I did not mind if variable getting/setting
> was slow, as most people using the framework use member functions and
> expect those to be performant.
>
> So, I made a metatable and then put all of the variables in a C
> function that I linked on that metatable's `__index` method. I think set
> this secondary, variable-lookup metatable to the I then linked this The
> problem is now that I get the table as the "lookup failed on this" object,
> meaning I cannot get the userdata and appropriately substitute the
> "get/set" syntax style.
>
> NOTE: I noticed that in Lua 5.1, 5.2, and 5.3, the Lua C API "leaves"
> the original userdata below the bottom of the stack (in my specific case, 2
> stack values underneath it). The lua API doesn't stop me from digging past
> API-specified beginning of the stack, so a dirty hack I'm compelled to do
> is to simply do a stack get at -4 (to result in -2 and get the userdata),
> but I'm obviously relying on Undefined Behavior if I'm doing that to work,
> even if it's present in all of the versions of the API I care about.