RE: Thinking about Lua memory management

["Jack Palevich" <jpalevic@...>]

I am glad to hear so many suggestions about Lua memory management! Unfortunately I am on vacation right now, and I don't have a good enough Internet connection to reply to each suggestion individually.

I am quite impressed by the quality and brevity of the current Lua code. So I hesitate to suggest ways in which it could be improved. (At least before doing extensive research to back up my suggestions.) Garbage collection may be an area where different application domains prefer different garbage collection algorithms.

One interesting thing I discovered is that Lua's current gc performance seems to be highly dependent upon the quality of the malloc / free heap implementation. I tested the same Lua program on two different platforms with roughly the same performance, and one platform was 10 times slower at garbage collection than the other. Although I have not yet run a performance tool to see exactly what's going on, I suspect the difference may be due to the difference between the two platform's "free" routines.

As for generational garbage collection, that's certainly worth trying. (It seems to be popular in modern Java JITs.) Unfortunately for games it may not be sufficient, because the worst-case behavior of generational garbage collection is just as bad as single-generation garbage collection. (And game players are nearly as unhappy with a stutter that happens once a minute as a stutter that happens once a second.)

One of the attractive features of reference counting is that it allows the Lua application programmer to control the generation of garbage -- if they avoid creating circular references, they will never have to collect garbage. Even if they do create circular references, if they manually break the circular references, they can still avoid having to collect garbage.

I agree that the cascading deletions problem means that reference counting isn't really real-time. But since the programmer controls when the cascade happens, they may be able to hide the cascade from the user.