Re: co-routines [was Re: Unicode]

[Roberto Ierusalimschy <roberto@...>]

> I can't remember why I didn't follow that way but at least one reason was, 
> that it takes to GC passes to collect all data from the threads (first to 
> collect the userdata that holds the state and another one for the objects 
> used by that thread). And it's slower. 

The only difference in the GC is that it has to traverse multiple stacks 
instead of only one. It is quite simple, and as efficient as it was (when 
there is only one stack). Notice that, in my implementation, a thread (or 
stack) is not a new type, and is not subjected to GC: The external code 
must destroy (close) it explicitily. (Of course, you can pack it inside a 
userdata, and set a GC tag method for that.) 


> I noticed two points: you need something to pass data from one state to 
> another. Your solution with lua_ref(L1)/lua_getref(L2) is IMHO not correct. 
> You may raise an error in the wrong state. 

You have to work really hard to get an error calling lua_getref. But I 
agree that it is not "polite" to do calls over a foreigner state. A 
"passvalue" function is already in my "to think" list (although I guess 
that for that case we need a "sendvalue" instead of a "receive" one). 


> Another point is that I have globals for each state.  At the moment
> (with the standard API) it seems impossible to change globals for
> other states without a new API function.

Each thread has its own "globals". lua_setglobals operate over the
given thread, so you can change globals as you like. (Again, you may
need to call that function over the "wrong" thread, but again you have
to work hard to get an error over lua_setglobals.)


> Sounds like the "Big Kernel Lock" in first Linux SMP kernels.  The problem
> was that it scaled very bad.  Basically you limit the use of Lua to one
> state at a time.

I don't think this is really a "Big Kernel Lock". Every time Lua calls C it 
unlocks, so even a loop like «while 1 do a=sin(i) end» will be preemptive. 
(OK, you cannot do busy waiting, like «while not a do end») And, as I said, 
you can register an empty call hook, so that every function call that Lua 
makes it also unlocks; or you can even register an empty line hook, so that 
no program can keep the lock for too long (with line hooks, you can even do 
busy waiting). The hooks are per/thread, so each thread can have different 
tradeoffs bettween preemptiveness and efficiency. 


> I guess, people not bothered by that could just as easy use coroutines ;-)

I also like coroutines, And, using a call hook as a dispatcher, we can get 
almost "real threads" with coroutines. But many people prefer real
trheads. But coroutines block the whole program in I/O operations (yes, you 
may use select, but not always, and not for output). Moreover (again, I 
agree this is sad) most OS offer an "official" implementation for threads, 
but not for coroutines.


> Don't you think that a handful of locks could work?  Especially the stack
> does not need a lock and I guess that's the most used location.

Even the stack needs lock, because another thread may be doing garbage 
collection. And the more locks you have, the bigger the chance of subtle 
errors; and the performance gets worst too (lock/unlock are not that 
cheap).

-- Roberto