Re: is it possible to make longjmp-free Lua?

[David Manura <dm.lua@...>]

On Mon, Jul 27, 2009 at 11:09 AM, Mark Hamburg wrote:
> On Jul 27, 2009, at 4:47 AM, Juris Kalnins wrote:
>> Sorry, this reply was unnecessarily terse. To clarify: problem is not that
>> setjmp doesn't work with -fno-exceptions, but that C++ throw/catch cannot be
>> used with -fno-exceptions, and thus any Lua exception that unwinds C++ stack
>> frame is an error. In other words, it is not possible to call most of Lua
>> API from C++ functions that have non-trivial local variables, if Lua doesn't
>> use C++ throw for exception handling.
>
> You can address that by only using lua_pcall to call from C++ to Lua and
> being prepared for any errors that come back. Or you could if you didn't
> have to also worry about all of the memory and parameter errors that will
> then necessitate going though lua_cpcall. And unfortunately, lua_cpcall is
> moderately expensive since it needs to build a new function object...
>
> This brings up an old thought I had when I last looked at trying to
> bottleneck things through lua_cpcall: Would it be useful to have a light C
> function data type that would have no environment or upvalues and which
> could be stored in a TVALUE without allocating storage?

I was thinking of posting something on this exact performance problem
with lua_cpcall allocating a new function object on each call.  The
allocation can be avoided.  In brief....first construct a Lua C
function that when executed calls the C function pointer stored in the
lightuserdata passed as the first argument.  This is done only once.
Now, whenever you need to call a C function, lua_pushvalue that Lua C
function mentioned previously, lua_pushlightuserdata the C function
pointer you want to call, and finally do the lua_pcall.  All three
operations never longjump and are safe, assuming you have sufficient
stack space.  The approach does assume that C function pointers can
fit inside a lightuserdata, which is usually safe in practice.  I had
done some micro-benchmarking of this and got a good speedup.

Now, back to the original question...wrapping every bunch of unsafe C
API operations in a lua_cpcall, may be the "correct" and proper way to
mix Lua longjump error handling with other types of error handling
(e.g. C++ exceptions), but it can also be a pain/ugly.  Juris's
example from lposix.c is a good one that does not even involve C++.  I
have found "exception safe" solutions to such problems, but they are
not necessarily obvious.  For example, I can replace lua_cpcalls with
lua_pcalls as described above.  I have a C++ RAII object that resets
the Lua stack upon teardown.  I have a version of lua_pushstring that
raises a C++ exception rather than longjumps via some trickery
(lua_pcall a certain Lua function that returns a Lua string based on
the C string passed as lightuserdata).

This could be implemented more straightforwardly if there were a
version of lua_pushstring that returned an error rather than
longjumped, for then I can avoid the pcall.  Likewise, I could likely
find some uses for a lua_newuserdata that did not longjump on error.
For example, I could placement new a C++ object into userdata memory
managed by Lua gc without wrapping the allocation in a pcall.  Now,
perhaps there is already a way to do this, such as, as stated, define
your own memory allocator that is guaranteed not to fail under certain
circumstances, but most will likely want to avoid doing that, if they
are even permitted to do so.

I would hope that Lua 5.2 makes some improvement here.  When there are
not obvious solutions to simple problems, most programmers will not
code solutions correctly.  (An example of that I mentioned previously
was the common problem of safely converting a Lua error object,
returned by lua_pcall, to a string.  In general, this requires
grabbing "tostring" from _G and calling it, which in turn may itself
error.)

Note that some choose to give up on memory allocation exception safety
because of the overcommit feature on certain operating systems [1].

[1] http://lua-users.org/lists/lua-l/2008-10/threads.html#00363