Re: Writing extensions which interact (is not fun)

[Jay Carlson <nop@...>]

[another tale of woe from the fighting-the-dynamic-linker department]

On Aug 4, 2012, at 2:19 PM, Thomas Harning Jr. wrote:

> On Aug 4, 2012 7:10 AM, "Carsten Fuchs" <carsten.fuchs@cafu.de> wrote:
> >
> > Am 2012-08-04 11:12, schrieb Doug:
> >>
> >> Binding everything up and statically linking the extension to the
> >> binary does seem to be the only way to make it work properly.
> >> [...]
> >>
> >> Deeply frustrating to work with.
> >
> > Exactly my experience, too.

I've never really understood why blindly calling for symbols was a good idea. The best argument I know of is that PIC code is a bit bigger and a bit slower--and used to be a lot slower on i386. x86-64 reduced some of these costs. 

Moving the interpreter into the main program improved the Shootout scores, which were great for marketing. Competing systems like Python already had their code in main, so it almost seems like a handicap to have done the right thing for an embedding language: live in a library, use /usr/bin/lua as a demo of that library.

> > Especially "mixing" is difficult: Linking everything statically or linking "everything" dynamically seems to work out relatively naturally, but linking the main application statically and using DLLs/SOs only for extensions/"plugins" comes with an astonishing number of platform-specific differences, and it is very hard to deal with all of them combined.

I agree. Exporting symbols from /usr/bin/lua is just asking for trouble, since dynamically loaded extension modules cannot link against -llua and get the dependency problems solved.

On ELF systems you should be able to have a hybrid system. liblua would declare all of its API as weak; loadable C modules would dynamically link with -llua and hence record their dependency and call in liblua.so.1 at runtime. In the case the module is loaded into /usr/bin/lua, the symbols from main would override the symbols in liblua.so.1. This would lead to two copies of liblua (one in the DLL, one in main) but the shared library code would not use memory as it would not be faulted in.

As it happens, weak symbols only affect the regular linker, not the dynamic linker. But the right thing happens since the main program is first in the search order.

This would not be error-proof. But if you wanted error-proof you would not be swimming upstream against the dynamic linker.

One possibility for error-detection is to put a global symbol like "lua_I_was_linked_static" in the non-shared liblua.a. In our liblua.so.1, the first call to lua_newstate could use dlopen(NULL) to get a handle on the main namespace and look up that symbol. If "lua_I_was_linked_static" exists but we're executing inside liblua.so.1, something has gone horribly wrong.

> One thing that I have done in the past is to alter Lua such that its functions were all lumped into a function table in the "extra space" behind the Lua state pointer. A set of altered headers were used by extensions that I needed to dynamically link in.

On Linux, the dynamic linker wants to be your friend. Set the soname on liblua.so.1 to liblua-iXhAFX0JK9kGfsAi.so.1, link your main program against that; link your Lua extensions using -Bdirect and that library, and you will always get precisely those symbols. There will be only one copy of that library regardless of how many times it is referenced by the main program or other extensions.

Normally when you write "sin()" the linker just looks for any symbol named that. Adding  -Bdirect tells the linker to record "I found sin() in libm.so.6" so if some bozo in another extension comes along and names a function sin, it doesn't affect your symbol resolution, which was fixed at link time.

Implementing your own symbol table seems like a waste.

> This made it so that no matter what method by which the Lua state came into existence, all external code could get a handle on the same codebase running the VM. Though of course all extensions to use it required being built with the special headers. It also solves the potential problems that could arise if you wrote an extension for some other software and didn't want to possibly have symbol/dll-naming collision problems.

-Bdirect kills off the weak symbol hack, but if you're going to recompile everything anyway, it's less effort to make liblua dynamic and switch sonames.

Jay

("openssl rand -base64" also wants to be your friend.)