Re: [release] parser-gen

[Peter Melnichenko <mpeterval@...>]

On Wed, Aug 30, 2017 at 6:07 PM, Benas Vaitkevičius <vsbenas@gmail.com> wrote:
> Dear Lua community,
>
> I am glad to announce parser-gen, a parser generator that I created together
> with LabLua this summer. The code can be found on GitHub:
> https://github.com/vsbenas/parser-gen
>
> The tool extends the LPeg(Label) module to provide these features:
> 1) generate parsers based on a PEG grammar description (similar to ANTLR and
> re);
> 2) allow easier error description and reporting;
> 3) automatically handle space characters, comments or any other patterns set
> by the user;
> 4) build ASTs automatically based on the rule name;
> 5) generate recovery grammars automatically, build partial ASTs for inputs
> with errors;
> 6) generate error labels for LL(1) grammars.
>
> All these features should make the description of parsers easier, faster and
> more concise. Any of these features can be disabled if necessary (for
> example, to build a custom AST).
>
> The grammars used for this tool are described using a PEG-like syntax, that
> is identical to the one provided by the "re" module, with some extensions
> used in the "relabel" module and "ANTLR" grammars. See a working example
> parser here:
> https://github.com/vsbenas/parser-gen#example-tiny-parser
>
> For a more advanced example check out the Lua parser:
> https://github.com/vsbenas/parser-gen/blob/master/parsers/lua-parser.lua
>
> The package can be installed using luarocks:
> $ luarocks install parser-gen
>
> This is a beta release, please report any bugs when found.

This looks interesting, thanks!

I've tried the Lua parser on some real code.
Testing on files in src/luacheck/ in
https://github.com/mpeterv/luacheck at commit ca21257:

analyze.lua, linearize.lua, parser.lua: errors with `too many
captures` in lpeglabel.match call.
format.lua: it seems to be confused by `{[[...]]}` construction.

    Syntax error #1: expected an expression after '[' for the table
key at line 332(col 18)
    Syntax error #2: expected '=' after the table key at line 332(col 58)
    Syntax error #3: expected an expression after '=' at line 333(col 4)
    Syntax error #4: expected '}' to close the table constructor at
line 333(col 4)

Testing on files from Luarocks show some other types of errors. But
these can be fixed.
The `too many captures` error is more troubling as it appears for most
larger files.
Additionally, the memory consumption of the parser seemed very high on
these larger files
(hint: turn off swap before testing this if you have it enabled normally).
Not sure if this is due to limitations of parser-gen or lpeglabel or
perhaps some bad case in grammar.

-- Best regards,
-- Peter Melnichenko