Coroutines vs. Callbacks, was Re: Lua on Reddit again

[Tim Mensch <tim-lua-l@...>]

On 1/31/2011 7:07 AM, Peter Cawley wrote:
> On Mon, Jan 31, 2011 at 1:45 PM, Alexander Gladysh <agladysh@gmail.com> wrote:
>> On Mon, Jan 31, 2011 at 16:26, Peter Cawley <lua@corsix.org> wrote:
>>> On Mon, Jan 31, 2011 at 1:10 PM, Steve Litt <slitt@troubleshooters.com> wrote:
>>>> Speaking about what might replace it -- could something with callback
>>>> functions replace it? They're powerful as all getout, but until now appeared
>>>> to be the sole territory of toolmakers, with application programmers simply
>>>> writing and passing small callbacks. Could that be because in other languages,
>>>> callbacks are just too syntactically difficult?
>>
>>> node.js has had a lot of publicity recently, and its modus operandi is
>>> to have callbacks everywhere as to avoid blocking.
>>
>> Side note: You ought to use coroutines in Lua for this.
> 
> That was my first thought too, but doing so in the naive way means
> that while waiting for a I/O event to complete, you need to remember
> the entire coroutine including all of its stack, which is a lot more
> expensive than remembering a single closure.

The casual game programming SDK "Playground" uses Lua coroutines to
achieve convenient UI sequencing, so you can say:

a = DoModal("some/dialog.lua")
if a=='ok' then
  -- do one thing, e.g., play the game
else
  -- do another thing, e.g., select a new player and then play the game
end

The dialog.lua file would describe the layout of a dialog box, for
example, with actions attached to various buttons that could then spawn
other dialog boxes or start a level of the game or what have you. You
could stack as many of these dialogs as you'd like, and that did happen
once when a bug created an "Are you sure you want to exit?" dialog every
time you hit the big red X in the corner of the window, but I digress...

I came up with the design, so I take full responsibility, but here are
the downside consequences of this approach that I've learned after
supporting developers in the creation of 100+ games based on Playground:

1. Only about one in five developers really understood it without having
it explained to them in person. To the rest it seemed like black magic,
and they would avoid doing anything complex, if they did anything at
all, in Lua.

2. When developers did use it, they would eventually get to the point
where they would call a C/C++ function, which would then call back into
Lua and attempt a DoModal(), triggering the dreaded "C/Lua" boundary
error, which most developers again failed to really understand.

3. Even when you get past #s 1 and 2, if you've got a bug somewhere in
your Lua code, any error AT ALL in the UI thread means that the entire
UI comes crashing down, because the current window context and UI loop
is tangled up with the Lua stack in your coroutine. I couldn't pcall()
everything because I needed to be able to yield() from within any function.

Maybe my design was just poor, and there's a way to do what I was trying
to do without causing the above three problems. But I spent a lot of
time thinking about it, and at the end decided to discard the coroutine
approach.

In my new approach, the "stack" of window contexts lives in a Lua table,
where each entry is a set of callbacks. If a single callback dies, then
just that callback fails, and other aspects of the UI continue to
function normally.

Ideally a game will ship with no bugs. But IMHO a bug where a single
button doesn't do what it should is better than one where the entire
game stops working, and during development your game feels MUCH more
robust if hitting the wrong button at the wrong time doesn't actually
appear to crash it.

Coroutines certainly work well for some things -- and queuing up a
complex load sequence might be one of those things -- but my point is
that it's also possible to overuse them in ways that make things less
rather than more robust.

Tim