Re: Using Lua in dynamic modules with static linking

[William Ahern <william@...>]

On Tue, Feb 26, 2019 at 12:42:02PM +0000, Chris Smith wrote:
> Hi all,
> 
> I have read all that I can find on building and linking Lua libraries, but
> I’m still a bit unsure if what I want to do will work. I’d appreciate it
> if someone can clarify things for me. Here is my situation:
> 
> 1. I have an application that uses Lua.
> 2. The application needs to dynamically load extension modules (note these
> are _not_ Lua extension modules, but for the application itself).
> 3. The extension module also uses Lua for its own purposes.
> 4. The use of Lua in the application and module is completely isolated —
> no sharing of any sort.
> 
> I think the best way of doing this is to statically link both the
> application and the module against the Lua libraries. I understand that
> means the Lua code will be duplicated in both application and module, but
> that is actually a good thing for me since it means the application and
> module can have mismatched Lua libraries. Is this correct, or will I
> encounter problems? I need this to work on OS X and Windows platforms.

I've done this on OS X (Mach-O) and Linux (ELF). Only the extension module
needs to be statically linked to Lua. The main application can be
dynamically linked or even just be a stock lua binary. But there are two
caveats as static linking and static compiling are ambiguous phrases on
modern systems--the details matter.

1) What really matters is that the extension module's Lua symbols cannot be
visible. It's not sufficient for symbol definition to exist in the same
shared object. If they are visible then at runtime the dynamic linker will
link the main application's Lua symbols (technically, the first global
symbol loaded from any shared object, such as liblua.so) into the extension
module's call sites rather than the ones defined in the module. This
behavior is what makes LD_PRELOAD work, but in most situations it's the
opposite behavior of what you'd expect. There are at least two (maybe three)
ways to hide the symbols:

1.a) #include the Lua source into the same C (or C++) source file as the
extension module and set LUA_API to "static". This is the obvious,
standard's compliant approach. However, this can be unwieldy (especially for
C++) and not great for incremental builds.

1.b) Use GCC's or clang's -fvisibility=hidden flag when building Lua. (Or
try setting LUA_API to '__attribute__((visibility ("hudden")))'). This has
the same effect as 1.a (truly "static" linking) but you can still build and
cross-link separate .o object files.

1.c) I think Mach-O has the ability to tweak symbol precedence to favor
symbols defined in same shared object. I haven't explored this, though.

Whichever way you choose confirm using the nm utility. On OS X you can do
`nm -g extension-module.so` to see the list of global symbols (defined and
undefined). You shouldn't see any Lua symbols.

2) The extension module's Lua runtime won't be able support dynamic Lua
modules--its Lua symbols won't be visible, but even if they were they would
have lower precedence than other Lua symbols previously loaded. You'll need
compile any Lua modules directly into the extension module and explicitly
load them--e.g. with luaL_requiref.

I'm not very familiar with Visual Studio so don't know how to do things
there. If Visual Studio doesn't support something like -fvisibility=hidden
you could try clang.