rethinking method calls with __mcall metamethod rather than __index/__call

[David Manura <dm.lua@...>]

A few Lua operations are implemented in terms of more primitive
operations.  ">" and ">=" are implemented in terms of "<" and "<="
respectively [8].  "<=" may be implemented in terms of "<" if a __le
metamethod is not given.  Moreover, method calls, a:b(c), are
implemented in terms of indexing and calling: local a = a; a["b"](a,
c).

There is a cost to that design.

Consider a "set" object with set operations:

  -- Simple Set ADT
  local set_mt = {}
  set_mt.__index = set_mt
  function set_mt:union(set2)
    for k in pairs(set2) do print(k) self[k] = true end
  end
  function set(t)
    local self = setmetatable({}, set_mt)
    for _,v in ipairs(t) do self[v] = true end
    return self
  end

  -- Example
  local s = set{'a', 'b', 'c'}
  s:union(set{'c', 'd'})
  assert(s['a'] and s['d'])
  assert(not s['union']) --> fails
  assert(not s['__index']) --> fails

As seen, if a set "s" has a method "union", this implies that
s['union'] is true.  For such reason, Penlight [1] avoids using the
index operator for membership tests in its set and map ADTs.  Also, if
we use the common technique of storing methods in the metatable
("set_mt.__index = set_mt" above), then s['__index'] is not nil
either, a subtle potential bug or security hole.

That also has implications in the __pairs/__ipairs discussions [7].  Given

  local s = set{'a', 'b', 'c'}
  for k,v in pairs(s) print(k, v) end

would we want to design it to print a, b, and c?  or would we want it
to print union and __index?  After all, all these values (k) satisfy
the condition that s[k] ~= nil, so it would seem consistent that pairs
should print all of them.  Most likely, we only want to print a, b,
and c.  However, in some cases we may want to iterate over method
names (reflection).  The important point is that perhaps it would be
more consistent for s[k] == nil for method names k and we could
provide some other way to iterate method names.

Going further, consider a proxy object that forwards method calls:

  -- Proxy
  local mt = {}
  function mt.__index(_, k)
    return function(self, ...)
      local priv = self[1]
      return priv[k](priv, ...)
    end
  end
  function proxy(o)
    return setmetatable({o}, mt)
  end

  -- Example
  local s = proxy("test")
  assert(s:sub(2,3) == "es")
  assert(s.sub(s,2,3) == "es")

Splitting the method call into index and call operations results in
the inefficiency of a temporary closure being created per each method
call (granted, e.g., these might be cached).  Yet the above
implementation is still too simplistic:

  local s = proxy(math)
  assert(s.pi == math.pi) --> fails
  assert(s.sqrt(4) == 2) --> fails

If an object has both methods and fields, then the __index will need
to test or guess whether priv[k] is a method that should be wrapped or
a value that should be returned as is:

Such complications occurred in MethodChainingWrapper [2].

Consider also the potential for error in colon v.s. dot syntaxes for
method/function calls [5-6]:

  o:f() --> correct
  o.f() --> bad.  function is called, but in the wrong way, resulting
in f likely failing some way

If these were separate operations, then we could allow o.f to be nil,
and the above will fail earlier with cleaner errors (o.f is nil).

I agree with the thinking that o:f() and o.f are two separate
concepts.  One is message passing.  The other is indexing.  But Lua
forces the former to be defined in terms of the latter.

An alternative, proposed for consideration, is to provide a new
__mcall metamethod for method calls (a.k.a. message passing).  If
__mcall is not provided, Lua would revert to the old behavior of
consulting __index and __call instead.  The proxy example above would
be reimplemented more cleanly as follows:

  -- Proxy
  local mt = {}
  function mt:__mcall(k, ...)
    local priv = self[1]
    return priv:[k](...)   --[A]
  end
  function mt:__index(k)
    local priv = self[1]
    return priv[k]
  end
  function proxy(o)
    return setmetatable({o}, mt)
  end

  -- Example
  local s = proxy("test")
  assert(s:sub(2,3) == "es")
  local f = s:sub; assert(f(2,3) == "es")   -- [B]

Note that the above makes use of two proposed extensions: (A) using a
variable method name in the method call as proposed in [3] and (B)
using colon closure construction notation as proposed in [4].

[1] http://penlight.luaforge.net/index.html#T10
[2] http://lua-users.org/wiki/MethodChainingWrapper
[3] http://lua-users.org/lists/lua-l/2009-05/msg00001.html
[4] http://lua-users.org/lists/lua-l/2009-01/msg00606.html
[5] http://lua-users.org/wiki/ColonForMethodCall
[6] http://lua-users.org/lists/lua-l/2003-08/msg00248.html
[7] http://lua-users.org/wiki/GeneralizedPairsAndIpairs
[8] http://www.lua.org/manual/5.1/manual.html#2.5.2