Re: wrapping multithreaded C code in a library

[Javier Guerra Giraldez <javier@...>]

On Sat, Oct 8, 2011 at 1:55 PM, Thijs Schreijer <thijs@thijsschreijer.nl> wrote:
>> if you're not set on your design of multiple threads driving the Lua
>> code, you might consider my Helper Threads Toolkit, where a single
>> coroutine-driven Lua code drives several C tasks running on several
>> threads [1].
> Reviewed it, looks like a possible solution for dispatching work to some C routines. But how would this handle callbacks from external events? Eg. The thread doing the callback was not initiated from Lua, hence there is no corresponding lua thread that can be resumed. Can this be handled to?

two ways pop in my head:

A) simplistic but might get hairy: set a thread blocked waiting for
the external event.   a long-blocked thread has very little overhead
beyond the stack frame allocation, so it might be cheaper than it
sounds.

B) have to read some code:  check how the current code creates an
event object and how it pushes into the queue; then replicate the
3-steps (prepare, work, finish) without spawning a thread, and in your
callback push the event in the queue.  the queues are there to allow
the helper thread to signal without touching the Lua state, so it
should work easily from your callback.


>> if you already decided on several threads interacting with a LuaSocket
>> loop, i'd say that your idea is workable; except that I'd replace the
>> TCP packet with an event on another kind of socket.  In POSIX systems,
>> you can create a socketpair, or a pipe.  On Linux, there's and
>> eventfd, BSD have a similar thing.  All these are file-like objects,
>> handled by a filedescriptor, and you can wait on signals there with
>> the classic select() call, the same used by LuaSocket.
> I'm on windows, so didn't know this. As deployment will probably be linux this might be a good pick. Pitty about losing platform independence though.

I'm sure windows have similar things, but i don't know how they're
called.  the only IPC mechanism i've heard about is NamedPipes, but i
think they're nothing like files, so select() wouldn't fire on them.
OTOH, i think there's a generic 'wait' primitive that works on many
different objects and could be used instead of select().

still, you lose platform independence (on the other way).

note that HTT is based in pthreads, so it won't work out of the box on
windows either.  fortunately, it uses relatively few pthreads
features, and nothing too esoteric; so win-pthreads (or other thin
pthreads-like layer) should help.

-- 
Javier