Re: [ANN] LPeg 0.11

[Miles Bader <miles@...>]

Patrick Donnelly <batrick@batbytes.com> writes:
> On Mon, Apr 1, 2013 at 6:21 PM, Miles Bader <miles@gnu.org> wrote:
>> Roberto Ierusalimschy <roberto@inf.puc-rio.br> writes:
>>> I am not sure I understood what you want. If you do not want to
>>> keep any captures from ONE_STATEMENT, why does it have captures in
>>> the first place?
>>
>> Any results from ONE_STATEMENT are already recorded via
>> side-effects, so any captures inside it have already been used, and
>> I want to discard them for the "next iteration" (of ONE_STATEMENT).
>
> LPeg can't know those captures can be discarded and so can't "flush"
> the captures. The reason for this is quite simple and you brought it
> up: side-effects. Consider (automagically) flushing a pattern's
> captures which calls capture functions with side-effects. However,
> the enclosing pattern *does not match* and so those side-effects
> should never have happened.

Right, which is why I'm suggesting an explicit LPeg directive that
says "this is OK, I know what I'm doing."  The onus is then on the
user to make sure that they deal with the effects of this.

Typically this would mean that any error handling resulting from a
failed top-level match would discard previously recorded results, or
that it doesn't matter (e.g. when the program just errors out and
terminates as a result).

[My memory of Prolog is pretty fuzzy, but I think this sort of
corresponds to the Prolog "cut" operator.]

>> Trying this in real life leads to memory explosion, whereas just
>> using an explicit loop outside of LPeg doesn't, so clearly
>> something's being kept around inside LPeg between iterations.
>
> You talk about memory explosion, do you mean LPeg stack overflow?

No, I mean it uses so much heap space that my (4GB) system starts to
thrash itself to death... :]

The exact same grammar with the loop moved "outside of LPeg" is fine,
so the memory issues are not due to actual recorded results.

For very simple top-level grammars, e.g. those that simply cover the
input with the repetition of single pattern, doing the loop outside of
LPeg is reasonably easy, but for anything even a just a little more
complex, it becomes a pain, and sort of ugly to boot.

Hmm, I suppose I should try to come up with a succinct example that
illustrates this.

Thanks,

-miles

-- 
Suburbia: where they tear out the trees and then name streets after them.