Lua, multithreading and Qt 4.8

[Michal Kottman <k0mpjut0r@...>]

Hello list,

I'd like to know your opinion and experience with handling
multithreading in Lua. My main question is - how to handle
multithreading that is forced onto me by the library I am binding?

Right now I am working on updating lqt [1], the (almost) complete
binding of Qt to Lua, for the newest version of Qt. For Qt versions
4.7 and earlier, threading has never been an issue - Qt was running in
a single thread, and the user was not allowed to create new threads
(QThread was blacklisted). However, things changed in Qt 4.8 - even
though the 'qtcore' and 'qtgui' modules still function as expected (at
least what I experimented with), the 'qtwebkit' module seems to create
new thread(s) for downloading/rendering etc.

The problem is that lqt generates "shell" classes for every Qt class
containing virtual methods in order to allow their (re)implementation
in Lua. This means that the background threads eventually end in Lua,
and sooner or later, cause a segfault.

I have tried to handle this by locking the access to every lqt
function which modifies the global Lua state, but I soon found that
this is not enough to solve the issue. Right now I am thinking about
locking *every* access to Lua from Qt, even from virtual methods,
essentially making a GIL for Lua. I would need to release the lock
before a call back to Qt and lock it immediately after the call. I am
not sure this will help though - multiple threads will be operating on
the same Lua state (even though not at the same time) - is this
considered safe?

If this becomes a problem, I will have to think about creating new Lua
states using lua_newthread() and do the processing in them, but I'm
not sure this is necessary...

The worst thing about this is that I have no control over the
threading - suddenly, my functions are called from a different thread.
Does anybody have experience how to handle these kinds of issues with
multithreading and Lua?