...
It seems that's related with the local table defined inside the function, but I don't know if that's the expected behavior or how to avoid it (not using jemalloc is not an option). Btw, this is also happening in Lua 5.3, but not in LuaJIT (I guess because LuaJIT uses its own allocator).
Any explanation for this behavior?
Thanks a lot! :)
You may try to turn off the tcache of jemalloc. I guess tcache may use huge memory since you create 20k threads.
Hi,
Turning off tcache is not an option (this code runs as a Varnish module -i.e. shared library-, which uses jemalloc), but using your hint I tested two approaches:
1. Using lua_newstate() int oder to provide a custom allocator implementation based on jemalloc's mallocx() and rallocx(), using a single specific arena (i.e. MALLOCX_ARENA()).
2. Calling 'thread.tcache.flush' after every script execution.
Both work in the toy C program, but only second one works when benchmarking the real implementation. I don't know why, but I guess that's because of jemalloc or Varnish Cache internals I still need to investigate.
Thank you very much!
Best, |