Re: Will Lua 6.0 have LPeg?

[William Ahern <william@...>]

On Tue, Mar 04, 2014 at 09:05:46AM -0600, Andrew Starks wrote:
> On Tue, Mar 4, 2014 at 8:08 AM, Richard Hundt <richardhundt@gmail.com> wrote:
> > On 3/4/14 9:00 AM, Dirk Laurie wrote:
> >> Is LPeg useful enough to justify how very much larger
> >> Lua's C source code will become if it is included?
> >>
> >> IMHO, if the gain will merely be that the ordinary user
> >> would no longer need to build/download it separately
> >> (which LuaRocks can do for you) and require it (which
> >> takes one line of code), the answer is NO.
> >>
> >> On the other hand, if the Lua implementation itself exploits
> >> it for its lexer/parser/VM encoder and its string library,
> >> thus demonstrating how useful it can be while at the same
> >> time providing some highly instructive well-written LPeg
> >> code, the answer is YES!
> >>
> > Shameless plug, but you can try it already [1]. It should give you an
> > idea of how it might be integrated. Feedback welcome.
> >
> > Nyanga statically links with LPeg, and although the grammar syntax is
> > very similar to the 're.lua' module, it doesn't pass a string to
> > 're.lua', but produces an LPeg grammar directly using the API.
> >
> > [1] https://github.com/richardhundt/nyanga#grammars
> >
> 
> 
> LPeg is much easier for me to use than Lua's pattern's. That is most
> likely because I do not have any prior experience with PCRE. Unlike
> Lua patterns, I can read an LPeg grammar out loud.
> 
> I'm wondering:
> 
> How would Lua evolve with a first class "pattern" type? That is,
> merely adding "pattern" as a type would have an impact, because you
> could check for it as an argument and respond differently. There might
> be other definitions of "first class" that I'm blind to and that might
> be interesting, as well.

LPeg has a type routine. Once could easily write

local function ispat()
  return type(x) == "userdata"
     and package.loaded["lpeg"]
     and package.loaded["lpeg"].type(x) == "pattern"
end

I've never had occasion to do that, even though I use LPeg regularly.

> Could Lua's patterns be implemented in LPeg? If you removed the
> existing pattern code, then added LPeg + Lua patterns, then what is
> the size/performance difference?

The beauty of Lua's pattern code is that it can easily and quickly execute
the pattern matching logic without compiling the pattern or even using any
dynamic memory. Once you add in stuff like alternations, it just makes more
sense to compile it into a bytecode, because now you're doing stuff like
if/else (branching) and goto (jumping backward).

To get anything near the same speed with PEGs, or even regular regexes,
you'd at a minimum need to memoize the pattern string after compiling into
an intermediate form.

Lua's pattern matching code is all about simplicity--from usage down to the
implementation. That's a strength, not a weakness, IMO.