Re: speed of Lua/LPeg parsers

[Duncan Cross <duncan.cross@...>]

On Mon, Sep 13, 2010 at 11:20 AM, Miles Bader <miles@gnu.org> wrote:
> I always knew that lpeg was decently quick, but I had a nice
> experience this weekend that really drove the point home.
>
> I use Lua as a frontend-language for my raytracer.  For testing
> purposes, I thought it would be nice to write a parser for the input
> language of another raytracer, "PBRT", so I could compare the results
> of rendering exactly the same scene.
>
> PBRT is written in C++, and uses a yacc/lex parser for its input
> files.  I wrote my parser using LPeg and Lua, taking no special
> measures for speed, and doing lots of error checking and other
> post-processing in Lua.
>
> As it turns out, however, my LPeg/Lua parser is not only decently
> fast, but actually seems much _faster_ than PBRT's yacc/lex parser.
> For some fairly large and complex input files (millions of lines), my
> LPeg/Lua frontend can read them in less than half the time that PBRT
> takes.  I got even more speedup later by linking with luajit.
>
> Anyway, it's just an anecdote, and I suppose PBRT's parser might be
> doing something really stupid that slows it down a lot, but I'm quite
> impressed...
>
> -miles
>
> --
> Cat is power.  Cat is peace.
>
>

This reminds me of something I was wondering about: could a special
version of LPeg be created that takes direct advantage of LuaJIT
acceleration, so that the "bytecode" of the PEG patterns itself gets
JITtered? Does LuaJIT support (even conceptually, if not at this
precise point in the beta) external C modules that have
JIT-specialised functions, like the functions in the standard Lua
modules are? If so, would it be possible for a single dynamically
linked library to contain both the standard and JIT versions, or would
they need to be separate? Or is this all a blind alley, and the only
realistic way would be a port of LPeg to Lua itself?

-Duncan