Re: co-routines [was Re: Unicode]

[Edgar Toernig <froese@...>]

Wow.  First thanks very much for the detailed answer.

Roberto Ierusalimschy wrote:
> 
> > Could you be a little bit more specific?  Just some functions to save/set
> > L->stack & co and a registration for lock-callbacks on table-set/get?
> > Or something more sophisticated?  What about the GC?
> 
> We broke the lua_State in two structs; the lua_State itself has the
> thread-specific data, and it points to the other structure with the
> "global" data. There is only one new function (let us call it lua_newstack,
> we don't know the final name): the call lua_newstack(L, stacksize) returns
> a new lua_State sharing the global state with L. Actually, lua_open is now
> a macro lua_newstack(NULL, stacksize).

Sounds familiar :-)  I can't remember why I didn't follow that way but
at least one reason was, that it takes to GC passes to collect all data
from the threads (first to collect the userdata that holds the state and
another one for the objects used by that thread).  And it's slower.  But
not that obtrusive like the implementation in Sol-41 and much more ver-
satile.

I tried that in Sol-42 (same location).  I noticed two points:  you need
something to pass data from one state to another.  Your solution with
lua_ref(L1)/lua_getref(L2) is IMHO not correct.  You may raise an error
in the wrong state.  I made a sol_receivevalue(L, otherL, otherIndex)
to get objects from another (suspended) state.  That may only raise
errors in the current state.

Another point is that I have globals for each state.  At the moment
(with the standard API) it seems impossible to change globals for
other states without a new API function.

Btw, does it makes sense to have state-local hooks or/and a state-local
registry?

> About the locks: we considered the whole core of Lua as one single critical
> region (it is too complicated and too expensive to lock every single
> internal operation that can have impact in other threads). So all API
> functions have a pair of macros LUA_LOCK/LUA_UNLOCK around them. On the
> other side, whenever Lua calls C, in unlocks its core. So, unless you are
> in a closed loop completely inside Lua, you have plenty of opportunites to
> switch treads.

Sounds like the "Big Kernel Lock" in first Linux SMP kernels.  The problem
was that it scaled very bad.  Basically you limit the use of Lua to one
state at a time.  I guess, people not bothered by that could just as
easy use coroutines ;-)

Don't you think that a handful of locks could work?  Especially the stack
does not need a lock and I guess that's the most used location.
Hmm... even one lock for only L->gstate (my pointer to the global state)
could be possible/better?  (I have no pthread enabled system to test
any of this.)

I once hear about Windows, that dlls do this kind of locking by default
to protect them against other threads (only heard, not sure).  You wouldn't
even need the LUA_LOCK/UNLOCK for a Lua.dll.  I guess Windows-guys would
like your simple locking ;-)

Ciao and thanks again, ET.