Re: Treating threads as separate states for separate scripts

[Max E <max@...>]

On Sun, 2011-09-04 at 21:44 -0800, Max E wrote:
> Hello list,
> <SNIP>
> Thanks preemptively,
> -Max E.

All right, after reading the Lua source code some more, I figured it all
out myself. Since I know this mailing list is archived on the Internet,
I'll explain it all here so anyone with similar problems can use my
solution. Also, this is for Lua 5.1, I understand it's different in 5.2.
The good news is, I did not need to change any of my Lua scripts to make
this work.

So you may have heard that lua_setfenv on a thread object sets the
environment for that thread. You may also have heard that
LUA_GLOBALSINDEX is actually the environment table for the
currently-running thread. This is all true-- however, even when working
on the lua_State you got from lua_newthread, the thread is NOT yet
"currently running." So if you call lua_newthread to get a lua_State,
then call lua_setfenv to give it a new environment table, and then call
luaL_openlibs, it won't work. The luaL_openlibs function will populate
the libraries directly into LUA_GLOBALSINDEX, but LUA_GLOBALSINDEX is
not yet aliased to the thread's environment tables. The way I worked
around this is, I manually copied everything over into the environment
table after calling luaL_openlibs. This does NOT need to be a deep copy,
but be sure you have a special case to not copy _G over from the
"doomed" old GLOBALSINDEX.

Okay, so how do you make sure that your thread is the "currently
running" one? It's not so hard-- you can pretty much replace all your
calls to lua_pcall and lua_call with calls to lua_resume. (Well, not
quite: you cannot recursively resume a script, so if your code is
recursive, you should check for recursion and use lua_pcall instead as
appropriate.) But, for example, even after calling luaL_loadfile, you
should call lua_resume instead of lua_pcall. Otherwise it won't load
your script into the correct environment.

Now, as for my other problem: how do you "require" the same library in
multiple threads? It turns out that the "require" function stashes a
cache of already-loaded libraries in a table called _LOADED, which is in
the registry, which is shared between all threads. But this can also be
worked around. You simply set a metatable on _LOADED which checks which
thread is currently running and juggles multiple versions of _LOADED
depending on which it is. Be sure to clear this information when you
unload a script. You will set this metatable once on the "main"
lua_State and it will be applied everywhere. You cannot have your
metamethods check what the value of _G currently is to differentiate
between threads; I've already tried that, it doesn't work.

Other gotchas: lua_newthread pushes a thread object onto the stack of
the "main" lua_State when you call it. Be sure to keep this object in a
variable somewhere or else your thread will be garbage-collected. NEVER
call lua_close on one of your threads, simply delete the thread object
and allow it to be garbage-collected.

Here is my new C code, which still needs to be cleaned up a lot:
  http://pastebin.com/0zbAQhch
Here is prepare_thread.lua, which includes the code for copying
everything into the new thread's environment table (function installg)
and also the code for the _LOADED metatable (function
create_loaded_metatable):
  http://pastebin.com/DkpGuu4J

Hopefully this will help someone!

-Max E.