Re: lpeg and streams

[Roberto Ierusalimschy <roberto@...>]

> > > I would like to be able to use lpeg to parse a stream of incoming
> > > data.  I would also like to be able to use it for parsing huge
> > > files.  In both cases, I won't have the entire input in memory.
> > > [...]
> >
> > The way LPEG works (and most parsers in general), it seems that all old
> > data must be available, though it would seem LPEG 'could' work in a way
> > that permitted more data to be fed in later... still requiring that the old
> > data be maintained.

It is not necessary to keep *all* old data available, only what is
needed for backtracking. Of course, if you write a "bad" pattern it will
need all data, but "good" patterns should be able to work with small
pieces. (This is what Dave suggested, but LPEG would do the higher-level
loop automatically for you.)

-- Roberto