Re: coroutine.resume() crash

[Sean Conner <sean@...>]

It was thus said that the Great John Dunn once stated:
> > From: Gé Weijers <ge@weijers.org> 
> > You used the word 'asynchronous' in your first email. I hope you're not
> > trying to (for instance) call into the Lua interpreter from a signal
> > handler or another thread, because that's going to royally mess up
> > things and just end in tears😀. Lua is quite single-threaded by design.
> >
> 
> It's all on a single thread. In my case I'm using libevent as a message
> loop to drive not only timers but also asynchronous network IO across
> multiple independent Lua states. 

 By "independent Lua states" do you mean such states created by
coroute.create() (from within Lua) or by lua_newthread() (from C), or is is
completely unrelated states created by lua_newstate()/luaL_newstate()?  Lua
coroutines (or Lua threads) are separate structures, but they do share a lot
of state internally.  Lua states created by lua_newstate()/luaL_newstate()
are isolated from each other.

> I'm also not driving the event loop from
> within Lua - each Lua script is run once on load and after that it's all
> driven by various event handlers calling back into Lua when things happen
> - very similar to node.js model. I just get a callback from libevent when
> something happens ( ie a timer has expired, data is available on a socket,
> etc ) - I get context related to the timer ( in my case a pointer to a c++
> timer object ) which is why I was storing the lua_State to use to call
> back into Lua when this happens.

  I did something similar 12 years ago---the main code was in C, Lua scripts
were called when events happened.  Three years ago I implemented the main
driver in Lua [1] (without reference to the C version) and both are
remarkably similar in how they work.  The main driver is eventloop(), meant
to be run from the main coroutine.  This code will handle time events, as
well as network events [2].  When it does run a coroutine to handle an
event, the coroutine is run until it either finishes, or a "blocking" type
call is made.  

> What seems to work is - 
> 
> 1.  when Lua starts I store the 'base' state using lua_setfield(L1,
>     LUA_REGISTRYINDEX, .. ). Call this L1.
> 2. Timer.New() creates a new userdata - this can happen both in the main
>    Lua state or a coroutine - call this L2. L2 is stored in a c++ data
>    structure associated with the userdata

  I handle timeouts differently.  Time events are queued in a binary heap,
and the main loop checks the current time, and any coroutines that have
"timed out" (or requested to sleep for a period of time) are then scheduled
to run [3].  For reference, this code has been tested quite well.  It runs
my gopher server [4], my Gemini server [5], and a few services at work (one
of which processes millions of SIP messages per day).

  In my code, a function is associated with the event (as can be seen on
line 193 of [1]).  This function is responsible for finding or resuming the
proper coroutine to run.  You can see this starting with line 98 of [6].  

> 3. user assigns function to timer.EventHandler. This is via a metatable so
>    I get a callback from Lua with some lua_State Lx. I store a reference to
>    the function using this passed in state via luaL_ref
> 4. coroutine yields
> 5. at some point later libevent calls a callback on my c++ instance
> 6. this instance uses the stored L2 in the object to call lua_getfield(L2,
>    LUA_REGISTRYINDEX, ..) to retrieve the L1
> 7. I look up the function via lua_rawgeti and call it using L1 as my
>    lua_State
> 
> If coroutines aren't being use this still works - it just means L1 == L2.
> 
> While this works it's not clear to me this is safe and there are a couple
> of places where I have questions. For example, when I store(3) and
> retrieve(7) the function does it matter which lua_State is being used?
> Currently I'm using L2 to store and L1 to retrieve in my test and it's
> working - do all coroutines share the same LUA_REGISTRY which makes this
> possible? Also is there a better way then my scheme of stashing the base
> lua_State? I didn’t see a way to determine the base from an arbitrary
> lua_State in the c API.

  I can't shake the feeling that your timer events abstraction isn't quite
right, but I can't quite say why.

  -spc

[1]	https://github.com/spc476/lua-conmanorg/blob/master/lua/nfl.lua

[2]	The code supports systems that don't serve up time events via
	select()/poll()/epoll()/kqueue() (depends upon OS).

[3]	Starting with line 166 of [1].  The binary heap implementation
	starts at line 58.

[4]	Server: gopher://gopher.conman.org/
	Code: https://github.com/spc476/port70

[5]	Server: gemini://gemini.conman.org/
	Code: https://github.com/spc476/GLV-1.12556

[6]	https://github.com/spc476/lua-conmanorg/blob/master/lua/nfl/tcp.lua