Rici tries to explain. Was Re: Arguments by reference?

[Rici Lake <lua@...>]

On 1-Sep-05, at 3:37 PM, Aaron Brown wrote:

> It may seem as though people are making a big deal out of a
> subtle terminological quibble, but I'd like to add something
> from personal experience that may explain this.
>
> When I first learned Lua, I had the rule of thumb in my head
> that Lua passes by value, except for tables, which are
> passed by reference.  This caused me to think incorrectly
> about my code.

Well put (and the rest of it, too). (I also liked Boyko's quote from 
Lewis Carroll, which showed up while I was writing this.)

Let me try this once more, for what it's worth. If it seems confusing, 
feel free to ignore it.

It's very easy to start thinking about data at a very low level. (This 
is encouraged by C.) So you get in your head a model where an integer 
and a table with 27,348,312 entries in it could not possibly fit in the 
same "space". Obviously, the table is "much bigger".

Furthermore, we easily fall into the trap of thinking about "variables" 
as locations, or containers.

The combination of these two preconceptions makes the idea that:

   local a = 3

and

   local a = giantTable

*must be* different kinds of operations. Obviously, the giantTable 
can't "fit" into the local "a". (Or if it could, we'd only be able to 
use three locals in our whole program.)

So let's throw that all out the window, and stop thinking about how the 
language is *implemented*. Let's trust the implementers to worry about 
the low-level stuff, and get it right. We can just think about the 
semantics, and how to write beautiful programs with that.

Instead, think about objects floating around in some cybernetic soup. 
We can't really get a grip on an object -- it's all slimey from the 
soup -- but we can say "that object over there, the little light green 
one, let's call it 'a'"

And that's exactly what Lua is presenting you with.

  local a = obj

means "for the duration of this scope, let's call that object, the 
light green one with a big smile, let's call it 'a'". That's referred 
to "binding": we've "bound" the syntactic name 'a' to some object.

If we subsequently say

  a = 3

we have not "put 3 in a". What we've done is change what we mean by 
'a'. Now it's bound to a different object, a steel-grey one with three 
dots on top, say.

When a function is called, it is "given" a list of objects (the 
arguments). Inside the function, the parameters are bound in turn to 
each object. The binding is purely syntactic: the object has not been 
"put into" the parameter, nor has it somehow acquired a "name".

Now, let's look at the table itself. The table is not a container, 
either. We don't really "put things" into a table. What we do is 
construct a mapping between objects. The table associates a "value" 
object with each "key" object. So we can ask a table what object it 
associates with a given key.

Tables are mutable; so we can change the association:

  t[foo] = giantTable

This means, the next time t is asked what (the value currently bound 
to) 'foo' is, it should say (the value currently bound to) 
'giantTable'.

Binding is not a copying operation; it is simply the association of a 
syntactic name with an object floating around in the object soup.

Some objects are permanently bound to special names. (We call these 
names "constants".) For example, the name 42 is always bound to some 
object of type "number". Every use of 42 in our program refers to the 
same object (because the binding is permanent). If we say:

  t[42] = "forty-two"

what we've done is told (the value currently bound to) t that it should 
now associate (the object permanently bound to) 42 with (the object 
permanently bound to) (the string) "forty-two".

Both numbers and strings are immutable in Lua. That is, you cannot 
change a character in "forty-two" any more than you can make 42 into 43 
by frobbing the last bit. If an object is immutable, then there *could 
be* multiple copies of it floating around in the soup, because these 
various copies are indistinguishable from each other. But whether or 
not Lua happens to make a copy of an immutable object is not relevant. 
On the other hand, tables are mutable, so Lua cannot make a copy of a 
table without you finding out about it. Choosing to copy an immutable 
object is a legitimate low-level implementation efficiency, but you 
should not think about it when you are writing programs. You will 
probably do better to believe that objects are never copied, whether 
they are numbers or giantTables.

In Lua, there are two important syntactic constructs which are neither 
constants nor names: function and table *constructors*.

  {a = 7, b = giantTable}

is not a constant. It is executable code which creates a new table, in 
which the string "a" is associated with the number 7, and the string 
"b" is associated with the (value currently bound to) giantTable.

Similarly,

  function(x, y) return x + y end

is not a constant. It is executable code which creates a new function 
object.

And, thanks to syntactic sugar (and I emphasize the word syntactic), 
the statement:

  local function foo(x, y) return x + y end

is precisely equivalent to:

  local foo = nil
  foo = function(x, y) return x + y end

That is, it is an executable statement which constructs a new function 
object, combined with a syntactic binding of the name 'foo' (within the 
block scope).

Now, a function object, once created, carries bindings around with it. 
So if we do this:

function foomaker()
  local tab = {a = 7, b = giantTable}
  local function foo(k, v)
    local rv = tab[k]
    tab[k] = v
    return rv
  end
  return foo
end

x = foomaker()

x is now bound to a function, newly-created by foomaker, and that 
function carries around a binding to a table which was also 
newly-created by foomaker. Every time x is called, the function object 
is executed with k and v bound to the supplied arguments, and the same 
binding 'tab' as it was created with.

Lua, unlike some languages, allows outer-scope bindings like 'tab' to 
be rebound. In effect, every time that foomaker is called, it creates a 
new *binding*. We don't really see the effects of that in this example, 
because tab is never rebound. So let's look at an example where that 
actually happens (this is the classic example, which you'll also find 
in PiL):

function accountmaker()
  local balance = 0
  local function accept(some_more)
    balance = balance + some_more
    return balance
  end
  return accept
end

Every time accountmaker() is called it creates a new binding 
('balance') and a new function ('accept') which carries this binding 
around with it. (Note that the fact that 'balance' happens to be 
initially bound to a number in this example, while 'tab' was initially 
bound to a table in the previous example, is completely irrelevant to 
what's going on.)

So:

account1 = accountmaker()
account2 = accountmaker()

= account1(10)
= account2(35)
= account1(7)

Try to predict what this will do before you try the code out.

Hope that helps someone.

R.