Re: Lua plans to add meta-programming support?

[Axel Kittenberger <axkibe@...>]

A proper equivalent the C-preprocessors "#line" capabilty would solve
or "outsource" all this problems including personal tastes on commas
and what so ever, as it would allow you to write you own preprocessing
language without losing to debugability. I'd prefer something that
would even allow to tag tokens rather than just lines. I guess this is
a rather leightweight patch to lua*, so when I get some time and
leisure for that, I'd come up with a prototype.

*(contrary for example to immutables which would be pretty hefty :-)

On Sat, Mar 10, 2012 at 6:22 PM, Xavier Wang <weasley.wx@gmail.com> wrote:
> I also think of to use lpeg to do this. but I found lpeg has it's limit:
>  - as you said, we need some trick to preserving the line information, etc.
>  - and a killer issue is, lpeg parse the *whole* code, not the code
> chunk, we can't offer a lua_Reader to lpeg!
>  - using lpeg to reimplement a lua lexer may introduce bugs, and can
> hardly produce the same error message as Lua.
>
> that's why I think of a general purpose token filter to lua itself, it
> doesn't have such issue. we can make metalua/luaMacro/lc top of it.
> using it, we can split the compile time and run time in Lua, that open
> the door of lua meta-programming. we can easily implement the macro in
> Lua, what Roberto was thought.
>
> maybe a discuss about what a token filter will do and how to provider
> this service is necessary. e.g. maybe a token filter will produce the
> ast of Lua code, and serve it to metalua.
>
> 2012/3/10 steve donovan <steve.j.donovan@gmail.com>:
>> On Sat, Mar 10, 2012 at 8:08 AM, Xavier Wang <weasley.wx@gmail.com> wrote:
>>> Now we are discussing some semantics proposals in Lua. I believe that
>>> the main solution is makes Lua easy to meta-programming. i.e., add
>>> some hooks to llex, makes Lua can callbacks on lexing).
>>
>> We did have the token-filter patch by lhf, which did pretty much what
>> you suggested. I based the first version of LuaMacro on top of it, but
>> requiring a patched Lua is irritating, so the current version uses
>> LPeg to do the tokenizing [1]  (The tricky part is preserving original
>> line information so that error messages and error tracebacks make
>> sense)
>>
>> These are lexically-scoped macros, so they don't contaminate the whole
>> global namespace.  For instance, consider replacing mytable[#+1] with
>> mytable[#mytable+1] which was suggested recently.  That is, you can
>> write code like this:
>>
>>  require_ 'rawhash'
>>
>>  Tab mytable, another
>>
>>  t = {1,3}
>>
>>  -- Here # is short for #mytable
>>  mytable[#+1] = 1
>>  mytable[#+1] = 2
>>
>>  -- without indexing, behaves just like a table reference
>>  assert(type(mytable)=='table')
>>
>>  -- it is still possible to use #t explicitly
>>  assert(mytable [#]==mytable[#t])
>>
>>  assert(mytable[#-1] == mytable[1])
>>
>> Here 'Tab' is a macro which generates locally-scoped macros, and the
>> implementation is here [2]
>>
>> Also works interactively, where you have access to other things like
>> lambda short-forms:
>>
>> C:\Users\steve\lua\LuaMacro\tests> luam -lrawhash -i
>> Lua 5.1.4  Copyright (C) 1994-2008 Lua.org, PUC-Rio
>> Lua Macro 2.3.0 Copyright (C) 2007-2011 Steve Donovan
>>> Tab mytable
>>> mytable = {10,20,30}
>>> = mytable[#]
>> 30
>>> f = \x(2*x)
>>> = f(2)
>> 4
>>
>> There are limitations - this is smart substitution but it is still on
>> the lexical level. I cannot implement the most commonly suggested |x|
>> 2*x syntax for short lambdas because it's hard to know where the
>> expression ends without parsing it.  Some of the freedom of
>> token-filtering is also lost, so I can't show you a filter that
>> replaces newlines in tables with commas (I'm thinking of an elegant
>> yet efficient way of doing this)
>>
>> steve d.
>>
>> [1] https://github.com/stevedonovan/LuaMacro
>> [2] https://github.com/stevedonovan/LuaMacro/blob/master/tests/rawhash.lua
>>
>