Re: expanding on luaproc ideas

[Coda Highland <chighland@...>]

On Fri, Apr 27, 2012 at 10:35 AM, John Belmonte <john@neggie.net> wrote:
> I recently discovered the nice "Exploring Lua for Concurrent
> Programming" paper.  (Yes, I'm a little behind.)  It briefly mentions
> expanding the implementation outward to remote processes.  I'm
> actually wondering about expanding the implementation inward, to
> coroutines.  In other words, could there be a concurrency abstraction
> across pure coroutines (cooperative multitasking within a Lua state),
> preemptive multitasking between Lua states, and remote processes
> (whether another OS process on local machine or remote via network)?
>
> Why would that be interesting?  My assumption is that coroutines are
> significantly lighter weight than Lua states.  With a concurrency
> abstraction encompassing coroutines, the application designer doesn't
> have to decide about pure coroutines vs. real threads vs. distributed
> computing up front--  the same software building blocks will work as
> efficiently as possible in each of the modes.  As the system scales
> you'd think about locality of messaging-- first level would be
> coroutines within a Lua state, next Lua states within an OS process,
> then OS processes within a machine (could help isolation, not
> performance), and finally machines within a network.
>
> I haven't thought much yet about whether an implementation is feasible
> or how the luaproc API would change.

Certainly such an implementation would be FEASIBLE, but it wouldn't be USEFUL.

The main reason that adding coroutines as an abstraction here isn't
beneficial is because it provides no benefit. While it is true that
coroutines have less overhead than a second full Lua state, coroutines
have more overhead than simply writing the algorithm serially.
Coroutines do not provide parallelism; they do not accomplish
multiprogramming in any real sense. They can't take advantage of
additional resources such as additional cores, additional CPUs, or
additional machines.

/s/ Adam