Re: Lua and preemptive scheduling for coroutines

[Jose Luis Hidalgo <joseluis.hidalgo@...>]

On Sun, Mar 31, 2013 at 3:45 PM, Leo Romanoff <romixlev@yahoo.com> wrote:
> BTW, async is more or less coroutine.create + coroutine.resume() + some book keeping? Or is it more than that?

It is just the create, and then the manager stores the coroutine as
ready to update on next loop. It could be a create+resume but it was
easier this way.

> Another question: You do not support preemptive multtasking, do you? I.e. your async threads need to use wait_for or channel:read to yield, right?

Right now it is not preemptive, everything in SLB3 is designed to be
as fast as possible while keeping the code easy and clear to follow,
as far as I know using hooks could be much more slower. But, I can try
add preemptive support if someone really depends on it :)

>
> One thing I'd like to have is the ability to provide a Lua code implementing parts of the logic that
> - decides which thread should be executed next

Ok, that's something that can be added easily... right now you can
replace the update function, which the default implementation is :

    // This can be replaced to implement a proper scheduler if needed. It
    // can also be used to filter which active coroutines are updated.
    // By default executes all active coroutines available.
    virtual void update_implementation() {
      while(update_pullNextActiveCoroutine()) {
        update_resumeCoroutine();
      }
    }


> - defines new waitfor-like or async-like functions

That's something you can do right out of the box:

    // -- Initialization of the manager ------------------------------------

    // Method called by init once the lua_state is set. You can override this
    // to define your own waitfor and async functions.
    virtual void init_implementation() {
      lua_State *L = lua_state();
      pushLua_waitFor(L, 0);
      lua_setglobal(L, "waitFor");
      pushLua_async(L);
      lua_setglobal(L, "async");
    }

pushLua_waitFor, and pushLua_async allows you to push multiple
versions of wait/async functions, in the case of waitfor you can also
push a ID (0 in the example above) to help you identify which
wait-like function was called.

> - etc
>
> I.e. I'd like to be able to "derive" from your CoroutineManager in Lua. Since it is not possible directly the idea is to have a class LuaBasedCoroutineManager which allows me to provide Lua code for different overridable methods. Then each such method could look like:
>
> void LuaBasedCoroutineManager::methodX(args) {
>     if(luaCodeForX != null) {
>         // execute this lua code with provided args
>     } else {
>         // invoke default implementation (e.g. from the base class)
>     }
> }

I bet it's possible to derive from the current CoroutineManager, and
implement a LuaBasedCoroutineManager by properly calling lua functions
:) Right now, the only thing that it is not yet implemented is the
preemptive multitasking.

>
> If something like this would be possible, I could write my Lua apps without any need for writing a single line of C/C++ code. I only need to load your library. No makefiles, no need for C compilers, etc. No need for script developers to know C.

Well... there's a problem, your lua main loop should call the
coroutinemanager::update, that's what make this library tick :)

>
> Very nice!  The channels example is very interesting.
>
> While looking at your APIs and examples I couldn't understand if your wait_for constructs could wait for a specific event or events? Or are they simply a replacement for yield, which result in waiting for a resume call. I think being able to wait for a certain event from a set of alternatives or for all events from a certain set would be very powerful.

It is, and in fact you can wait for whatever you want, the API it offers is

    // Signals a coroutine that was waitingForSignal, to continue its
    // execution on next update. It returns true if the coroutine was indeed in
    // that state and was successfully moved to active.
    bool signalCoroutine(const Thread);

    // Aborts the given coroutine, it will return true if the coroutine was
    // indeed a valid coroutine. False if you called this over a coroutine that
    // no longer existed.
    bool abortCoroutine(const Thread);


Basically a wait-for function stores the Thread (lua_State *|copiable)
somewhere... when the event or events are met, it is just a matter of
signaling the coroutine, or abort it, and on the next update cycle it
will be continued. You can also push values to the coroutine stack so
they will be there as a result of the wait:

    // Pushes some value to the thread stack, once the coroutine is signaled the
    // result of waitFor will receive all pushed values as a result.
    template<class T>
    void pushResultToCoroutineStack(Thread thread, T value) {


Ok... That one depends on SLB3 ;)


>
>> The coroutinemanager is part of SLB3 (Simple Lua Binder, to wrap C++
>> objects to lua), but it can be used alone. I haven't documented
>> properly SLB3 yet, neither previous versions of SLB... but the code is
>> used in our game engine on a daily basis, in fact the coroutine
>> manager has been backported from the game engine to SLB .
>
> OK. I briefly looked at the code and it seems to be small and self-contained enough to be used on its own.

the "pushResultToCoroutineStack" I believe is the only method that
uses SLB3 features...

> It is a bit pity that the most interesting comments on this thread arrived after I have spent time on creating a working solution ;-) If you guys have had commented before, I would probably use one of your solutions, instead of reinventing my own from scratch. But on the other hand I learned a lot about Lua VM and coroutines.

Sorry, I read the list messages from time to time...


Regards,
   JLH