Re: GC Optimisation question for Lua4

[jjensen@...]

>> Either way, the more the merrier.  While I'm hopeful the 5.1
>> incremental collector will work, the current stress tests I've run
>> with it don't hold a candle to the speed and ease of use of luarc.
>
> That sounds strange. A quick run on heapsort or ack (from the shootout)
> shows Lua 5.1 faster than luarc.  (Of course, those tests are unreal
> because they generate no garbage, but a test that spend all its time
> generating and collecting garbage is unreal, too. It was easy to create
> a test where Lua 5.1 ran in 1/3 the time of luarc, as it is easy to
> create tests where luarc runs much faster than Lua 5.1.) Anyway, it
> seems that Lua 5.1 at least can "hold a candle" to the speed of luarc.

I generate a lot of per game frame garbage (now that I have a ref counted
version of Lua).  I would expect luarc to be slower than Lua 5.0.2 for
cases where no garbage is generated (due to the added reference counting
overhead).

But this test isn't entirely unreal... it is a compounded version of what
I see every frame of my game:

http://lua-users.org/lists/lua-l/2004-03/msg00393.html

The common case for me is that I work with a large Lua state (lots of
data), and I have script that generates lots of per-frame garbage.  None
of the per frame table data is cyclic (very much on purpose).  So, when a
string is created and handed to Lua for a quick function call, it is
destroyed as soon as it is removed from the stack.  Actually, that's what
happens with closures, tables, and userdata, too, which is very
convenient; it's like C++'s deterministic finalization.  And best of all,
only the minimal amount of memory is consumed, as there is no garbage to
actually collect.  When running in tight memory environments, this is a
boon.  (Contrast this with my days in Lua 4.1 with Amped 1 where the
script code was altered to perform NO memory allocations... that was hard
to work with, because no advanced Lua features could be used.)

> About "easy of use", what is difficult in 5.1 incremental? (Or, what
> is simpler to use in luarc, apart that we have to break cycles
> by explicitly calling the GC?)

When referring to the benchmarks in the above list posting, the only way
to bring down the Lua 5.1w0 (and Lua 5.0.2) memory was to call the garbage
collection routine either every pass through the inner loop or every pass
through the outer loop.  The timings for doing this in the inner loop
skyrocket.  The timings in the outer loop are much better, but more memory
is used.  luarc, for all non-cyclic cases, never requires the garbage
collector to be run.  That's what makes it harder to use the 5.1
incremental collector.  I have to convince my script writers to
strategically place calls to the garbage collector throughout the script
code in order to keep memory down.

I've seen the benefits of a reference counted Lua, and I also know of the
benefits of incremental and generational collectors.  A thought... the
author of the language Squirrel does a really interested thing with his
code.  Two different styles of garbage collection are #ifdef'ed into the
code.  One is reference counted with mark/sweep, and the other is purely
reference counted.  Memory is saved for the second one by making the
assumption that there will never be cycles, and if there are, you delete
the object yourself.  There is actually a keyword called 'delete' in
Squirrel that does this.  I've considered adding this as an extension to
luarc for those cases where tables are cyclic.  It's manual intervention,
but it is far faster than the mark/sweep collector on large datasets.

Again, I'm not downplaying Lua 5.1's incremental collector.  I plan on
using it in the future.  But I'm glad luarc exists, because I can use it
as a benchmark against the incremental collector to offer advice on how
the incremental collector can be better improved for the common cases I
see it in.  I may even make a version of Lua 5.1 with reference counting
support and the backing incremental collector (which will be far faster
than the mark/sweep collector).  We'll see.  I'm waiting for the next Lua
5.1 build before investigating that.

Josh