Change in GC behaviour -- Ever-increasing memory usage

[Uli Schlachter <psychon@...>]

Hi there,

I came across a supposed leak[1,2] with some C code. I think I distilled
the behaviour that causes problems into the attached Lua program and
would now like to get some input on "which code is to blame".

I noticed that the problem occurs with Lua 5.3, but does not occur with
Lua 5.1 or 5.2. Thus, I could do a Git bisect based on the GitHub mirror
of the Lua source code[3]. The result is that commit [4] introduced the
problem.

My main question now is: What to do about this?


Some more information about the program: The library in question
provides bindings to GObject-based C libraries. To support callbacks, it
has to luaL_ref() the callback function and it will then luaL_unref()
later from the __gc metamethod of some of the involved object.

This is just what the attached program does: In a tight loop, it creates
a table, luaL_ref()s it and arranges for a call to luaL_unref() later.
Apparently the code can now create references faster than they are released.

I hope this is enough information to help me. Thanks for any hints and
suggestions that you might have!

Cheers,
Uli

[1]: https://github.com/awesomeWM/awesome/issues/1490
[2]: https://github.com/pavouk/lgi/issues/157
[3]: https://github.com/lua/lua
[4]:
https://github.com/lua/lua/commit/0d745ed04c93e907e9f2bd8c21ce1ca27bba9b6a
-- 
Bruce Schneier can read and understand Perl programs.

local registry = {}
local refnil = -1

-- Lua-based reimplementations of luaL_ref and luaL_unref. Yes, this makes no
-- sense to have in Lua, but helps with the following experiments.
local function luaL_ref(t, obj)
	if not obj then
		return refnil
	end
	local next_free = t[0]
	local ref
	if next_free then
		ref = next_free
		t[0] = t[next_free]
	else
		ref = #t + 1
	end
	t[ref] = obj
	return ref
end
local function luaL_unref(t, ref)
	if ref >= 0 then
		-- Push the free reference into the freelist
		t[0], t[ref] = ref, t[0]
	end
end

-- Some helper functions
local guard_mt = {
	__gc = function(self)
		self.callback(self.arg1, self.arg2)
	end
}
local function make_guard(callback, arg1, arg2)
	setmetatable({callback = callback, arg1 = arg1, arg2 = arg2}, guard_mt)
end

-- Now simulate what LGI is doing when running the "leaky program" (https://github.com/pavouk/lgi/issues/157)
local iter = 0
local stop = 1000
local max = 0
for i=1, stop do
	--print(iter, collectgarbage("count"), #registry, registry[0])
	max = math.max(max, collectgarbage("count"))
	if max > 50000 then
		error("too much memory used after " .. tostring(i/stop*100))
	end
	for i = 1, 1000 do
		local ref = luaL_ref(registry, {})
		make_guard(luaL_unref, registry, ref)
		iter = iter + 1
	end
end