WARNING: glibc dlopen threading bug

[William Ahern <william@...>]

Most distribution-packaged Lua interpreters are not linked against
libpthread.

On Linux I and some others still considered it relatively safe to use
multithreading from modules, presuming the modules linked in libpthread
themselves, because glibc goes to great lengths to permit linking of
libpthread into the process at runtime. With the possible exception of
strerror, the Lua VM doesn't use any APIs that aren't supposed to be
thread-safe in glibc, whether or not pthread was linked at startup time.

However, I've discovered a bug in dlopen/dlclose (dlerror, specifically) in
glibc that invalidates this tidy assumption. I've reported it here:

	https://sourceware.org/bugzilla/show_bug.cgi?id=18192

The problem with this bug in particular (unlike, e.g., [1]) is that it
totally breaks Lua modules like Lua Lanes, LuaExec, mqlua, or my own
cqueues.thread module. You can't safely spin up a thread which uses
lua_newstate and then proceeds to load modules. I think this bug goes mostly
undetected because the race normally requires either a module to fail to
load, or for dlclose to be called from multiple Lua VM states. Because
modules don't usually fail to load, and because threads tend to be
long-lived in most applications (using thread pools), this bug has flown
under the radar, perhaps with the occassional unexplained coredump.

I also stumbled across this bug report regarding getaddrinfo:

	https://sourceware.org/bugzilla/show_bug.cgi?id=10652

I also discovered a couple of other similar bugs. (See my bug report.)

The short-term solution to this, as on other platforms that require
libpthread to be loaded at startup time (BSDs), is to use LD_PRELOAD. I've
added multi-system LD_PRELOAD support to my runlua script:

	https://github.com/wahern/luapath

Use libpthread.so.0 for Linux/glibc. See the getpth routine in the runlua
script for the library name to load on other systems. Solaris, OS X, and
Linux/musl have unified libc/libphread implementations.

Long-term I think it would be better if Lua packagers linked the system
interpreter with -lpthread. It's what Perl, Ruby, Python, and Node.js do.