Proposal for a standard way of defining custom operators in Lua

[Leo Romanoff <romixlev@...>]

Hi,

I continue my experiments with Lua and I get more and more excited about the language and its implementations. It is so small and so powerful! 

One of the aspects that I like most about Lua is its runtime efficiency. Lua VM is already pretty fast. And LuaJIT is really one of the best JITs around. Which makes me think that among other things, Lua can be a very interesting target platform for other (mostly dynamic) languages. One could create very efficient implementations of other languages by simply compiling them into Lua. But this ides is not new and was explored by quite some people.

When it comes to using Lua as a target "low-level" language, I miss certain features that would make it a more efficient platform for such a role. Those features could also benefit Lua even when it is used a primary language. I have quite some questions and ideas about it, but I don't want to put too much information into a single message. Therefore, I'd like to start with a simple question.  

This time I'd like to ask a question about custom user-defined operators.

As a preparation, I've read this page (as well as mailing list archives, etc) about this topic:
http://lua-users.org/wiki/CustomOperators

While I can see that a few operators can be modeled to some extent by existing Lua syntax, I also feel that there is a lack of a generic solution for it by means of a pure Lua. Of course, using Metalua or some macro systems for Lua one could probably achieve the goal, but it sounds like an overkill for me.

Therefore I was thinking about the following idea:
- Introduce in Lua a standard mechanism to define a new operator. This definition could look like this:
  lang.defineop('symbolic form', 'metatable method name', precedence, associativity, mode)

( This API is a bit inspired by Prolog-way of defining operators. For an example, see http://www.learnprolognow.org/lpnpage.php?pagetype=html&pageid=lpn-htmlse40 )

I used 'lang' as a new namespace for language-specific extensions here. It could be anything (e.g. debug, etc). 

The semantic of defineop arguments is as follows:
- Symbolic form defines how the operator looks like, e.g. '<<<' , '%%' , '<=>' 
- Metatable method name defines a method that should be invoked from the metatable when this operator is used. The value could be e.g. '__produce', '__send', etc
- Precedence would define operator precedence compared to other operators
- Associativity would define if it is left-associative, right-associative, etc
- Mode would define if it is prefix, suffix or infix. It may also specify that it is an unary operator.

All standard Lua operators could/should be predefined using this approach (or at least pretend they are defined this way).
Any (new) operators defined by means of 'defineop' would directly influence how Lua lexer works. In case of conflcts, e.g. '<<' vs '<', a certain policy could be used to pick the right operator (e.g. a longest match approach, putting explicit space to separate symbols, etc).
  
Once defineop introduced new operators, user-scripts or code parsed afterwards is allowed to use those operators. Lua lexer/parser would parse it properly, create an a proper internal representation (e.g. map the symbolic name to a special metatable method name) and later VM will try to dispatch via usual metatables using a provided special method name as a key. If no such special method is defined in a metatable of a given object/table, it should result in an error.

Benefits of this approach for defining new custom operators:
- It is very generic and powerful
- It makes it easier to define DSLs (domain-specific languages) and custom syntax using Lua
- Follows Lua ides of using metatables for dispatching operators to their implementations
- Allows for definition of ANY symbolic operators (as long as they represent a character sequence without spaces in between)
- Introduces rather small changes to the lexer (At least I think that required changes would be rather small)
- Does not require any changes in the Lua VM
- It does not change the semantics of existing Lua programs (unless someone wishes to explicitly redefine standard operators)

What do you think about this proposal?
- Does it sound technically feasible with a relatively low implementation effort?
- Would it be useful as a built-in feature of Lua?
- Have you already experienced situations/projects where you wished you would like to have such a feature?
- Are there any strong arguments against having this feature as a built-in feature in Lua?

Looking forward to your feedback!

Thanks,
   Leo