RE: helper threads layer

["Vijay Aswadhati" <wyseman@...>]

On Thursday, March 09, 2006 9:16 AM, Javier Guerra wrote:

> hi everybody

> finally, i think i have it working! it's a library of helper
> threads, meant to make it easy to write nonblocking libraries in
> C.

This may be the beginning of something new and exciting!!

> to be a bit more specific: in C, you split funcions in three:
> 
> static int xxx_prepare (lua_State *L, void **udata)
> static int xxx_work (void *udata)
> static int xxx_finish (lua_State *L, void *udata)

> and register it with a macro add_helperfunc (L, &ops)   (where ops
> is a struct with those three C functions).  after that, you get a
> Lua function in the stack.

> from Lua, you call that function, it calls the xxx_prepare()
> funcion with any given parameters and returns a 'task'.

> before that, you would have created a couple (or more) FIFO queues
> with helper.newqueue(), and one or more threads with
> helper.newthread().  each thread has one input and one output
> queue (several threads can share a queue).

> when you add the task to the queue (with queue:addtask (task)), a
> thread will pick it up, and execute xxx_work() with the same udata
> built by the xxx_prepare().  after xxx_work() returns, the thread
> pushes the task in the output queue.

> back in Lua, you can do a queue:wait(), it blocks until there's a
> task in the queue (put there by a thread).  then you call
> helper.finish(task) to execute xxx_finish().  xxx_finish() should
> dispose any memory used by the task (in udata), and return any
> value to the Lua caller.

> now, we could write non-blocking versions of most IO libraries,
> and build Lua wrappers that use this primitives to get a coroutine
> scheduler.

> i'll post it later in LuaForge; any thoughts about the
> architecture?

A few concrete examples would be of great help. I am particularly
interested in how one would go about creating a singleton timer
instance using this package and how the main dispatch loop would
look like in the "C" or "C++" world.

I have a C++ network layer that maintains its own send and receive
queues that is pumped by threads managed within this layer. It
dispatches remote messages received to various message handlers
which are now in C++; I would like to add the capability to add lua
based message handlers. 

But I am having difficulty in conceptualizing the main loop that
pumps the network layer, the timer facility and the Lua engine. That
is my problem of course, but any code to illustrate how a main loop
with threads in the picture would probably help me in rearchitecting
my current project to use Lua.

On a frivolous note, I would probably name this package as
'concurrency' instead of 'helper'.



-- 
Javier