Re: slow str:gmatch(), str:match() pattern

[Dirk Laurie <dirk.laurie@...>]

2016-02-03 15:18 GMT+02:00 Egil Hjelmeland <privat@egil-hjelmeland.no>:
>     line = (' '):rep(50000):match'([^\n]*)\n'
>
>
> takes > 20 seconds on my PC. Adding a start-anchor:
>
>     line = (' '):rep(50000):match'^([^\n]*)\n'
>
>
> is instant. Just getting the position of newline:
>
>     pos = (' '):rep(50000):match'\n()'
>
>
> is also instant. Tested on lua 5.[123]
>
> Actually I was using a gmatch() loop to parse out lines from blocks of data
> read from a socket, so adding the start-anchor was not an option.
> So I ended up refactoring the whole thing using str:find() and str:sub().
>
> It seems  str:match'([^\n]*)\n' tries really hard to find a matching
> substring, even though it should be clear after first scan that is not
> possible.
>
> Could that be optimized without adding too much complexity or affecting
> performance of other pattern matching cases?

For the application you have, just add a question mark to the pattern.

for line in buffer:gmatch'([^\n]*)\n?' do

This will still match all the empty lines.