[Date Prev][Date Next][Thread Prev][Thread Next]
[Date Index]
[Thread Index]
- Subject: Re: Coroutines and blocking IO
- From: Jon Smirl <jonsmirl@...>
- Date: Fri, 27 Jan 2006 00:39:04 -0500
On 1/26/06, Javier Guerra <javier@guerrag.com> wrote:
> On Thursday 26 January 2006 7:12 pm, Diego Nehab wrote:
> > The tough part is not to get these things to work, or even to make them
> > efficient. The problem is to come up with a simple, consistent and
> > portable interface that will make most people happy.
>
> ok, what about something like this:
>
> a C library that makes it easy to run a C function in a background thread.
> this background function wouldn't be allowed to use the lualib API, so it
> can't mess up with the Lua state. each background thread is associated with
> a userdata object with the thread's state, allowing the Lua code to check
> when it's done. besides, when the background function finishes, it's
> associated object goes into a queue, so it's easy to just ask for the next
> finished process.
>
> so, if i want to write a new library and i want it to be nonblocking, i just
> write the Lua-callable functions to prepare and spawn these background
> processes. inside them, it makes blocking calls, but it doesn't matter.
>
> the resultant Lua API have non-blocking calls and a centralized 'finished'
> queue.
Check out libevent - http://www.monkey.org/~provos/libevent/
The main program needs to end up in event_dispatch(). Down inside
event_dispatch() it is doing a blocking select/poll/etc. Your
secondary threads need to generate signals to wakeup the main thread
and get it out of the blocking event_dispatch().
There are three main categories:
1) async net IO using events like libevent
2) sendfile - a special case of event driven IO but critical for a web server
3) threads that wrap blocking IO
In general the event driven web servers are faster than the threaded
ones since they eliminate the overhead of threading. Sending messages
between threads when IO completes involves a lot of overhead
especially since the thread may not be running on the same CPU. The
rule of thumb is use events if possible, then fall back to IO threads.
Zeus and Lighttpd are both event driven, but Zeus spawns a copy of
itself onto each CPU in the system and Lighttpd doesn't support that.
That's why Zeus is the fastest server out there.
>
> the key point is that the background C functions (part of the IO library, not
> the thread library) MUST NOT use the lualib API. what do you think Diego,
> would that be doable without messing with the Lua core? (the main downside
> of LuaThreads)
>
> > The next version of LuaSocket will probably support IOCP on Windows. If
> > it does, it will be in association with a built-in coroutine dispatcher,
> > similar to the one in Copas. I will avoid using threading as much as
> > possible.
>
> maybe if the threading done once... it could be done well enough
>
> > It is unlikely that I will rewrite Lua's standard IO library to work
> > with the same dispatcher. But I promise to think about it. I will be
>
> if we come up with an easy to use C framework, i volunteer to write file IO.
> in fact, it would be almost trivial, and probably the first test of the
> framework
>
> > bothering Mike Pall and David Burgess a lot with this subject in a
> > couple months.
>
> lets shake the table a little...
>
> --
> Javier
>
>
>
--
Jon Smirl
jonsmirl@gmail.com