Re: [ANN] LPeg 0.11

[Roberto Ierusalimschy <roberto@...>]

> Note that the "inside LPeg" loop is:
> 
>   1. faster (I guess it avoids various LPeg invocation setup costs)
> 
>   2. Uses ten times as much memory (the "maxresident" value)... ><
> 
> It's (2) that's the real problem, of course, as my real-world tests
> ended up exceeding system memory for inputs that worked OK with an
> outside-of-LPeg loop.  As I also mentioned, however, doing everything
> in LPeg would also make non-trivial grammars much easier to handle.

I think Patrick explained the problem quite well. Lua cannot discard
captures before running them, and it cannot run them until it knows
for sure that the pattern will not fail. (The exception is run-time
captures.)


> Right, which is why I'm suggesting an explicit LPeg directive that
> says "this is OK, I know what I'm doing."  The onus is then on the
> user to make sure that they deal with the effects of this.

LPeg already has a construction for that: it is called run-time
captures. This is exactly what a run-time capture does.

A run-time capture is too expensive if you use it a lot (e.g., one
call for each char). But if you can manage to insert it only in "upper"
loops of your grammar, the cost could be quite reasonable. For instance,
try the following pattern in your little experiment:

FILE = m.Cmt(CHAR * CHAR^-200, function () return true end)^0

-- Roberto