Re: Lpeg and malformed input / Lpeg and subjects that do not fit into memory

["Mark Meijer" <meijer78@...>]

2008/9/18 Roberto Ierusalimschy <roberto@inf.puc-rio.br>:
>>[...]
>> Is that possible with current version of Lpeg or is another way of
>> solving this planned in a near future?
>
> I don't see any problem in doing that, but you have to manage the buffer
> yourself. That is, when matching the initial part of the buffer, use
> a position capture to tell how far the match went. Then read another
> piece, concatenate it with the unhandled part of the previous buffer,
> and repeat.

This is just a suggestion, it may have been considered already,
perhaps the implementation of LPeg doesn't easily allow for this, I
don't know. But maybe a nice idea might be to allow the user (of LPeg)
to specify an optional callback function, which is called whenever
LPeg wants to read more data. The callback then returns the next piece
of data, or nil on end of data/stream/file/whatever. The current
implementation could be kept as the default fast path, for when no
callback is specified. I think this may be a bit more flexible than
the above solution, and doesn't require any changes to the
grammar/captures to handle it.