Re: out of memory

[Enrico Colombini <erix@...>]

At 17.13 04/10/02 -0300, Roberto Ierusalimschy <roberto@inf.puc-rio.br> wrote:
>Anyway, that code can really gives an "out of memory" error. This is not
>a bug, but a limitation of the way Lua manipulates strings. (Java has
>similar problems.)

I'd like to use Lua for text adventures, but I soon found out that fast
string manipulation (especially repeated concatenation) is not the
language's stronger point.
Also, I haven't found out a practical way to write long strings (~5 kchars)
in a script, so that they could be properly read and edited by people using
an 80-char/line editor. Using a .. line concatenation to write a long
string as multiple source lines raises the above problem.

By the way, why is Basic so efficient in string handling and concatenation?
I don't remember encountering problems even when creating very large
strings char-by-char. It uses reference counting, if I remember correctly.
Maybe a moderate amount of over-allocation for strings could at least make
most concatenations a lot more efficient. It could be proportional to the
current string's length, ranging from 0 extra chars (for 1..3 chars
strings) to a fixed (customizable) limit, say 1 kB, for larger strings.

>Everytime you do `s = s.."x"' you create a new string. So, when s has
>size 50,000, after that statement you have two strings, the original one
>with 50,000 elements and a new one with 50,001 elements. After another
>loop you have three strings, and so on. Even if you do not run out of
>memory, you will use huge amounts of memory and of CPU time.

Why out of memory? Shouldn't the garbage collector get rid of the old strings?
And (asking out of ignorance) couldn't reference counting be useful for
strings, e.g. for getting rid of temporary strings like those created in
the above loop? After all, strings do not contain references to other
objects, so there is no risk of cross-referencing.

  Enrico