Re: LuaTask and memory allocation

[Rici Lake <lua@...>]

On 19-Jun-07, at 4:21 PM, Paul Ducklin wrote:

> Using LuaTask 1.6.0 statically linked into
> Lua 5.1.2. (A few other packages are built-in,
> namely the LuaSocket core, lfs, lrandom, bitlib
> and struct.) This is on Linux.
>
> Starting Lua, mem use (as shown by top) is about
> 1m resident, 2.2m allocated (virtual). Seems quite
> modest to me.
>
> Every time I call task.create() (even if the tasks
> do nothing but sleep immediately) mem usage goes
> up by about 70k resident, but a whopping 8m
> of virtual memory. So with 250 tasks I am _using_
> just 17m but have 2g allocated! Perhaps unsurprisingly,
> at 257 tasks Lua segfaults...
>
> Why 2m for Lua but an additional 8m of virtual memory
> for each created task, esp. when the task is only
> _using_ a modest 70k?
>
> I was rather hoping to handle a couple of hundred
> proxied sockets inside a simple Lua program... whilst
> there is plenty of physical memory to support this I
> am, absurdly, running out of address space :-)
>
> What's going on?

Each task requires a C stack; the default stack allocation
on linux is quite large. There might be a way to reduce it;
I'm not al tanto with LuaTask. Other operating systems may
(by default) make a smaller C stack, but since C does not
check for stack overflow, that can be dangerous.

You might want to try using plain old Lua coroutines,
which do not require a C stack but which do have some
limitations as a result.