lua-users home
lua-l archive

[Date Prev][Date Next][Thread Prev][Thread Next] [Date Index] [Thread Index] 

Lisa,
I think what you are trying to do is semantically incoherent, so it is not surprising that it is difficult. It's semantically incoherent because it confuses values with references to values ("boxes").
A local in Lua is just like a locally declared variable in C. Consider the following, which I think is a mirror of what you're trying to do: 

typedef struct IntList {
  struct IntList *next;
  int val;
} IntList;

IntList *set_and_advance (IntList *ints, int newval) {
  int i = ints->val;
  i = newval;
  return ints->next;
}
Obviously, this isn't going to work :) The local variable 'i' is different from ints->val and the assignment does not make it the same. If I wanted, for whatever reason, to use that style, I would have to make 'i' a pointer and explicitly dereference it: 

IntList *set_and_advance (IntList *ints, int newval) {
  int *i = &ints->val;
  *i = newval;
  return ints->next;
}
Lua, however, does not have a pointer type. So there is no obvious way of translating this into Lua. On the other hand, it is not a very common C idiom either.
C++, of course, provides the semantically odd reference type. Perhaps that's what you were thinking of: 

IntList *set_and_advance (IntList *ints, int newval) {
  int& i = &ints->val;
  i = newval;
  return ints->next;
}
References are odd beasts, though, since they are a weird mixture of runtime data and compiler sugaring. For example, you cannot assign one reference to another reference: 

int& i = &ints->val;
int& j = i;  /* ILLEGAL */
int& j = &i; /* Tell the compiler *not* to dereference i */

contrast with:

int* i = &ints->val;
int* j = i;  /* Correct */
int* j = &i; /* Wrong-o */
Of course, the natural way of writing this function (whether in C or in Lua) avoids all this mucking about with references and pointers: 

IntList *set_and_advance (IntList *ints, int newval) {
  ints->val = newval;
  return ints->next;
}
So what's actually going on in this version of set_and_advance? A deceptively simple question.
I would say, the structure 'ints' is being told to alter its 'val' field. But it seems that a lot of programmers, particularly those who program in C++, have the intuition that the '=' operator is being directed at the ints->val object itself. And, indeed, C++ allows '=' to be overridden, and not '->val='.
That's not really the full story, though. In C++ the '=' message is not being sent to the value of ints->val (that is, it is not being interpreted by the value 42, say). Rather, it is being sent to the (invisible) box containing the value 42. And that box is part of 'ints'. So what is actually going on is that 'ints' is being asked to provide the box containing the value, and then the '=' message is being sent to that box.
The fact that 'ints' is not otherwise consulted in the transaction is often a weakness. For example, it precludes 'ints' from applying any sort of data-coherency checks. Consequently, it is quite common to use a member function to change structure data members, and many people would say that allowing direct access to a data member is bad C++ style. This leads to code which looks like: 

  aList->setval(aList->getval() + 1);

rather than the arguably more readable:

  ++aList->val;

It's all what you're used to, I suppose.
In any event, in Lua table-assignment is semantically a message to the table (and that includes assignment to global variables, since global variables are just syntactic sugar for operations on the environment table). That's rather different from local assignment, although you might think of local assignment as a message to the Lua stack, after a compile-time translation from local name to stack index. 

So, to get back to your glue code.
It's quite straightforward to create a proxy object which represents a C structure (or C++ object); the proxy object can interpret get and set messages as it sees fit. One simple way of doing this is to create two tables of getter and setter methods for each class, and have the __index/__newindex metamethods look up the key in the associated getter/setter table for the class. In this model, the getter method for an integer member would probably convert the appropriate C numeric type into a Lua number, and the setter would attempt to convert the Lua number back into the C numeric type. (Note that round-tripping an int through a double does not lose data; only 64-bit integer types -- or single-precision floating point -- cause problems.)
Since the environment table is just a table, a similar technique can be used to dynamically add "global bindings" to C objects; an example can be found at <http://lua-users.org/wiki/BoundScalarGlobalsOne>. That example does not export functions to dynamically add mappings, but it should be clear how to do that. (It's also written in Lua, rather than C, but I hope it's clear how it could be written in C, as well.) If you do dynamically map globals, you have to think through what the expected behaviour would be if the global is already in use, but otherwise the implementation is pretty simple; also, that example tries to not override existing metamethods implemented on the environment table, but that is probably an unnecessary complication.
Finally, it is fairly simple to patch Lua to provide for a sort of reference/pointer ("boxed") datatype, but the fact that the Lua type system operates entirely at runtime makes it tricky to do without an explicit derefence operator, similar to the C '*' operator. It depends on defining a 'mutate box' primitive, which might be written ':=' or '<-'; the semantics of 'foo <- val' would be fairly similar to the C expression '*foo = val'. A small patch to implement the mutate operator can be found on the <http://lua-users.org/wiki/LuaPowerPatches> page. 

Hope that was at least interesting,

R.


On 10-Aug-05, at 10:11 AM, Lisa Parratt wrote:


Hi,
I've been developing a number of libraries for Lua (currently on 5.1w6) for the company I work for (this unfortunately means I can't go into too much detail about them or their uses).
My current main project is a glue code system for allowing Lua scripts to access C variables and functions. Rather than taking the approach to tolua, etc. of directly binding these elements into the Lua namespace, it instead takes a more dynamic object oriented approach.
Proxy objects (tables at their core) intercept attempts to get and set values, look up indices, look up structure/union members and call functions. These appear to the user to be native Lua data types, but instead apply operations to the underlying C equivalent. These proxies are generated on the fly through the use of the __index metamethod.
Unfortunately, I've had to add some metamethods to give them the truly native feel: 

__type: Overrides the return value of the Lua function type().
__set: Overrides an attempt to set the value of a Lua value. This is used to make assignment set the value of the underlying C value, rather than replacing the proxy object with the new value. See below for some caveats. __tonumber: Overrides an attempt to convert a Lua value to a number. This is invoked by the C function lua_tonumber(), among other methods. This, along with some modifications to the VM arithmetic operations, allows proxies to be used in mathematical operations as though they were native Lua numbers. __tostring: Overrides an attempt to convert a Lua value to a number. This is invoked by the C function lua_tostring(), among other methods. This allows proxies to be used as though they were native Lua strings. __ueq: Untype checked equals comparison. The normal comparison metamethods do not allow mixed types, preventing numbers from being compared against proxies. This metamethod is invoked when an attempt is made to compare different types. 
__ult: Untype checked less than comparison.
__ule: Untype checked less than or equals comparison.
__not: Overrides the not operator. A proxy is really a table, meaning that "not proxy" will always return false. This allows the result to be dependent on the underlying value of the proxy.
Can anybody suggest any ways of providing similar functionality that appears seamless to the end user without having to make these modifications? 

There are a couple of troubling issues:
Occasionally, intermittent errors such as "attempt to compare number with boolean" occur. I'm currently putting these down to subtle stack corruption issues, but they squirm away from beneath me when I try to instrument my code. Does anyone have any hints for debugging such issues?
Locals and the __set metamethod - essentially, for every VM cycle, the interpreter has to check whether RA represents a local. If it does, then it will check if the __set metamethod needs to be used. If this check is not done, then everything breaks horribly because the VM attempts to reuse a register and mistakenly triggers the __set metamethod. Currently, I'm walking through the call info stack to determine which registers are locals and which aren't at the start of each cycle, but this is horribly inefficient. I've tried adding a second stack of flags which indicates which registers are locals and which aren't, and this is maintained when the stack is resized, when a Lua function is called, when a Lua function returns, and when a Lua function tail returns. However, this doesn't work - the flags do not mirror the results of the call info stack walk. I suspect this may be related to upvalues and other similar issues. Can anybody shed any light on this issue?
Personally, I don't like having to have made these changes - they make upgrading to new versions of Lua more difficult and slow Lua down - but needs must.
Any assistance people can provide to help me remove my custom metamethods and restore the performance and reliability of the VM would be appreciated. If you need any clarifications, don't hesitate to ask, I'm well aware that I can blabber on a bit :) 

Cheers!
--
Lisa
http://www.thecommune.org.uk/~lisa/