Re: Status of token filters?

["Mark Meijer" <meijer78@...>]

I took a quick look at your LuaSub site. I think I like it. Not in the
least because filters are written in plain Lua, and you can use
different (sets of) filters for different source files, which is
something I thought about suggesting and citing advantages for in my
first post ;-)

Like you said, the downside is having the extra lexer stage in front
of the regular one. But if something like this won't be an official
part of the Lua core, then it's probably worth it to keep it as
cleanly separated from the core as possible (with the option of the
token reader patch for cases that need speed... nice touch, although I
haven't been able to find it yet :P).

Does this mean that LuaSub could theoretically be made into a
runtime-loadable module? Or does it necessarily need to "kick in"
before any Lua code can be parsed and executed?

Btw I'm also guessing it should work equally well with Lua 5.1.3 as it
does with Lua 5.1.2, as-is.

It must be difficult to get something like this to work in a way
that's relatively easy to use. I haven't read everything in detail
about how LuaSub does this, and so it looks a bit daunting at first
glance, but that is to be expected. What I understand so far from the
approach taken, though, well... Like I said, I think I like it :-)

- Mark


On 03/02/2008, Asko Kauppi <askok@dnainternet.net> wrote:
>
> luaSub does the front end thing by creating a lua_Reader that feeds
> the resulting code (after tokenizing & syntax mods) to liblua,
> internally. It's just an extended 'lua' application, with all the
> normal command line flags working as usual. The downside is that
> there's essentially a second, unneeded tokenizing round (the usual
> Lua round) but it really isn't much of a speed issue, either.
>
> I guess it all comes to how and where one wishes to use syntax mods.
> Personally, I think the entry step for them should be relatively low,
> and they should be usable commonplace, by common man. This year will
> certainly tell.
>
> -asko
>
>
> Mark Meijer kirjoitti 3.2.2008 kello 0:29:
>
> > On 02/02/2008, Asko Kauppi <askok@dnainternet.net> wrote:
> >> As Luiz mentions with the token filter patch, it's mainly _intended_
> >> to be used with 'luac', not liblua itself.
> >>
> >> Having worked my head off with luaSub, I think token filters won't
> >> even be needed in Lua core, at all. Along the lines of Luiz's
> >> recommendation, token filtering will be an added layer more in the
> >> realm of the syntax front end.
> >>
> >> Good news:      no changes to Lua required
> >> Bad news:       there are multiple solutions with differing way of
> >> defining
> >> syntax mods (this is not necessarily bad, imho)
> >
> > Yeah I noticed Luiz writing that somewhere. Isn't the "syntax front
> > end" of luac the same realm as the "syntax front end" of the Lua,
> > though? Changes would be required to it, but there would then be two
> > variations of it (one the runtime core, and one the pre-compiler
> > tool). And in order to use the token-filter enabled version, one would
> > be required to precompile all their stuff and have the runtime invoke
> > the resulting bytecode.
> >
> > Precompile config files created/edited by your users because you
> > wanted them to have a less intimidating/more intuitive syntax? I don't
> > really see any advantage, nor how it mitigates any of the possible
> > problems associated with making it officially part of Lua in the first
> > place.
> >
> >
> >> Maturity-wise, the original patch's main shortcoming is in not giving
> >> syntax awareness. Both Metalua and luaSub do provide that.
> >>
> >> If you wish to test-drive luaSub for your own particular itch, I'll
> >> be happy to take in people for a test drive. It _is_ approaching
> >> beta.
> >
> > Sounds nice. Can't guarantee when I'd have time to indulge myself with
> > it, though.
> >
> >
> > -Mark
>
>