Implemented fast string append and a new call a = string.append(a,b)

["Paul Sheer" <paulsheer@...>]

Hi there,

Considering LUA's poor string concatination performance (see

   http://www.lua.org/pil/11.6.html

), I have implemented a new API call string.append() to facilitate
fast string concatination. Patch against lua-5.1.1 is here:

   http://www.protonet.co.za/fast-string-append.html

Could someone help me out and check in the patch my use of

    setobjs2s(L, L->top - 1, L->base);
    setnilvalue(L->base);

Is this correct?

(Sorry, no time to update to 5.1.4.) The patch works as follows:

When string.append() is called, it returns a special "flex" string that
has more data allocated to it than required for its own length
(the amount of data allocated is the length rounded up to the
nearest power of 2.)

When string.append() is called on a flex string, it just fills in more of
the buffer and adjusts the length. Its hash value does not change.

In other words, string.append() may or may not return the first
argument back to you, hence can not be used like:

     x = string.append(y, z)

but only like:

     x = string.append(x, z)

So this is quite nasty, but does solve a problem for me.

A better implementation would be

     string.append(x, z)

Which appends to the string in-place, finding the pointer in the
hash table, and reallocating it. I was just a bit lame to understand
the GC/hashtable implications.

I tested the loop:

-- 
local n = 1
local s = "hellothere"
while n < 50000 do
    s = string.append(s, "a")
    n = n + 1
end
-- 

FYI with this patch, the performance of the above loop is faster by a
factor of 200.

It's faster by a factor of 30 than addString of section 11.6 of PIL

I hope this will encourage someone to do this properly at some
point.

Myself, I think fast string concatination is *extremely* important
for any language.

Lua is great -

Kind regards

-paul