Re: require one C module from another

[Aaron Faanes <dafrito@...>]

I'm actually surprised and sort-of sad that luaL_requiref doesn't check package.loaded. I had assumed it did, but I wrote out your test and it confirmed what you said.

The safest solution IMO would be to write your luaopen_* to be resilient (or a no-op) in the face of redundant requires. If all they do are assignments to the global environment, this level of safety is easy to attain. I'd be curious to hear if this wasn't the case for you. :)

I guess I treat require's "load only once" ultimately as a very strong hint rather than as a fundamental guarantee - it's just too easy for the programmer to break. In my experience, the most common "require didn't honor its contract" problems are when a loaded file overwrites one of its own stateful globals (causing some weird fissure in what I assumed were immutable globals), but this is really my own fault for not treating global state with more suspicion.

-- Aaron

On Thu, Jun 19, 2014 at 3:25 PM, Josh Haberman <jhaberman@gmail.com> wrote:

Suppose you have two Lua C modules "foo" and "bar", where "bar"
depends on "foo":

// foo.h
void foo_newfrob(lua_State *L);

// foo.c
void foo_newfrob(lua_State *L) {
  lua_newuserdata(L, 1);
  luaL_getmetatable(L, "frob");
  lua_setmetatable(L, -2);
}

int luaopen_foo(lua_State *L) {
  luaL_newmetatable(L, "frob");
  lua_pop(L, 1);

  // ...
}

// bar.c

#include "foo.h"
int luaopen_bar(lua_State *L) {
  foo_pushnewfrob(L);
  // ...
}

This will break if you require("bar") before you require("foo"),
because foo_pushnewfrob() won't find the "frob" metatable, because
luaopen_foo() never got called.

So what's the right solution to this? No solution seems perfect. Let
me first mention some things that clearly won't work.

You can't make luaopen_bar() directly call luaopen_foo(), because this
won't properly consult/modify package.loaded, so this could cause
luaopen_foo() to be called multiple times. While this might work in
some cases (particularly because luaL_newmetatable() will return any
existing metatable for a type), it will violate the expectations of
luaopen_foo() and doesn't seem like a good idea in general.

You can't call luaL_requiref(L, "foo", luaopen_foo, false) from
luaopen_bar(). While this is a little better than calling
luaopen_foo() directly because it will *set* package.loaded, it won't
consult it first (it runs the given function unconditionally). So this
could still result in multiple calls to luaopen_foo().

You could manually consult REGISTRY._LOADED and/or package.loaded, and
then call luaL_requiref() only if it was not previously loaded. But
this seems a bit perilous because REGISTRY._LOADED is not a publicly
documented interface, and could change at any time. package.loaded is
publicly documented, but is vulnerable to the user mucking around with
it.

Finally, you could call the Lua "require" function manually from C:

  // Add defensive checks against nil as desired.
  lua_getglobal(L, "require");
  lua_pushstring(L, mod);
  lua_call(L, 1, 1);

However this depends on the standard "require" being available as a
global function under its standard name, which could be violated for
several reasons (locked-down environment that has no "require", custom
"require").

Of all these, manually consulting REGISTRY._LOADED seems the safest.
Even though "_LOADED" is not publicly documented, it doesn't seem to
change much in practice, and dealing with the registry means you're
immune to changes in the global environment.

If luaL_requiref() did check "_LOADED" first, it would be ideal for this case.

Have I missed anything? Any option that is superior to all the ones I
investigated?

Thanks,
Josh

--
Aaron Faanes <dafrito@gmail.com>