Syntax trees, macros, and reflexive Lua evolution

["Dr. Rich Artym" <rartym@...>]

The never-ending suggestions for surface syntax improvements and Mark
Hamburg's mention back in September of Lua macros and his reference to
that great essay at

             http://www.paulgraham.com/avg.html

made me think ... we may be going about Lua evolution in a very poor way.

Lua may not generate a syntax tree explicitly as Roberto said, but it does
have a formal grammar and therefore each program certainly has an implicit
syntax tree.  In principle, there is no reason why this syntax tree can't
be generated for each chunk and manipulated by Lua, and the resulting new
tree compiled up again, either directly, or messily by first converting it
back into Lua source.  Thus, Lua macros needn't lag too far behind LISP's.

In many ways, Lua is doing itself no favours by making so many of its
decisions on the basis of syntax.  It really doesn't matter, that's not
where the power lies, and you can't please everyone in that area anyway.
Perhaps we need less specific syntax and more inherent, generic power.

Wholesale attachment to a 1-dimensional textual representation is rather
bizzare, when you think about it, and my guess is that it will disappear
entirely once programming attains some sort of maturity.  Lua evolution
could certainly benefit from regaining some of the power that was lost to
the world of computing when LISP was pushed out of the way into a niche.

Let's differentiate however between a simple direct tokenized program and
a normal form syntax tree which contains no decorative nor framing items.
It certainly would be interesting to reexamine Lua from that perspective.

And who knows, maybe thinking about the relationship between this syntax
tree and the VM could become the real focus of evolutionary change, with
syntax becoming a side issue which requires consent only between yourself
and your favourite macro set. :-)  And that's perfectly distributable too.

Rich Artym.
-- 
Existing media are so disconnected from reality that our policy debates
spin around a fantasy world in which the future looks far too much like
the past.   http://www.zyvex.com/nanotech/MITtecRvwSmlWrld/article.html