Re: changes in 'require'

[Mike Pall <mikelu-0507@...>]

Hi,

here are my 2 cents worth (Euro cents, mind you):

* require "a.b" forces require "a":

I'm +1 on this change. It makes my life easier, too.
And I'm +1 on fail-if-parent-is-not-there, too.

* Lua loader before C loader:

I'm mostly agnostic about the loader order. But I'm only +0 for the
'easier to change' argument (problem: you really can't move
C modules around because of the luaopen_<name>() conventions --
but see below).

Since module loading usually happens at startup, this is not a
performance issue. If anyone is loading tons of modules at runtime,
he/she can easily add a specific loader in front of the others.

* Adding .../lib/... to the Lua module search path:

Placing non-architecture specific modules into architecture-specific
paths is against established standards. I guess distribution
maintainers will rip this out right away. This makes it kind of
a moot exercise.

The keep-packages-together vs. split-packages-across-the-filesystem
debate is old. There is a pretty solid precedent in current POSIX
systems for the latter.

Mixed-architecture systems were out of fashion for a long time,
but have gained momentum recently with mixed x86/x64 systems
(this is in fact the common setup for Linux x64 systems):

$ file /usr/lib*/libpng.so.3.1.2.5
/usr/lib/libpng.so.3.1.2.5:   ELF 32-bit LSB shared object, Intel 80386
/usr/lib64/libpng.so.3.1.2.5: ELF 64-bit LSB shared object, AMD x86-64

Yes, this means a Lua x64 binary must be built with the C module
search path changed to '...;/usr/local/lib64/lua/5.1/?.so'
(in the distribution-specific package script for Lua).

Oh, and about the Perl/Python precedent: setting up either package
for a mixed-architecture system is a _real_ pain. Please don't copy
their approach or encourage module authors to do so.

* Length of the module search paths:

I'm all in favour of keeping the search paths as short as possible.
Try "strace python -c 'pass'" and you know what I mean.

This is another argument against adding .../lib/... to the Lua module
search path but also an argument against the 51w6 change to add
another two instances of 'l?.so' (note the 'l') to the C module
search path.

This is really useless and a holdover from the time when C modules
were not stored in extra directories. It adds ambiguities and this
change breaks LuaSocket under 51w6, too ("lsocket" and "socket"
refer to the same module).

I know why this was added, because lhf told me on IRC. But I think
there are less intrusive measures to solve that problem (like
changing the Makefiles of his modules ;-) ).

Please revert the paths to the 51w5 setting. Simple is better.

* Wrapping a C module with a Lua module (require() name conflict):

There is only one namespace for require() and C and Lua modules
must share them. Lua wrappers need to load C modules under a
different name. Workarounds like calling package.loadlib() directly
or calling the C loader directly are ugly and should be avoided.

For one this means there should be a naming convention for this
case, to avoid everyone reinventing the wheel. I have been pushing
for an underscore prefix for C modules that should not be loaded
directly (mirroring the Python convention).

I.e. require("socket") loads socket.lua and this in turn uses
require("_socket") to load socket.so. I think the underscore makes
it clear that this module is 'internal' and applications should not
load it directly (I know Diego doesn't like this convention, but
LuaSocket was the easiest example :-) ).

But as I said above, it's not possible to add a Lua wrapper on top
of a C module when the latter has not been designed for that. The
init function must be renamed (i.e. it must be 'luaopen__socket()'
for _socket.so).

I think a convenient solution would be to strip an underscore prefix
from the module name before generating the init function name.
This way it's easy to move (say) foo.so to _foo.so and load it with
require("_foo") from foo.lua. Here is the trivial change to loader_C():

  funcname = luaL_gsub(L, name, ".", LUA_OFSEP);
+ if (funcname[0] == '_') funcname++;
  funcname = lua_pushfstring(L, "%s%s", POF, funcname);

* Windows module search path issues:

We have seen a number of complaints about the fixed base directory
('C:\Program Files\Lua51') for the module search paths under Windows
in Lua 5.1.

I think no amount of changing this to a 'better' (but still fixed)
path will solve this. It's simply against the conventions to encode
a fixed path in a Windows binary. And of course Windows users are the
least likely to patch C sources and recompile. Even developers apt
enough to build their own C modules commonly ask for prebuilt Lua
binaries.

[Umm, and no -- setting environment variables doesn't cut it either
 on Windows.]

I propose to fix this once and for all and follow the suggestion of
David Burgess to get the path dynamically. This should work roughly
like this:

- Change LUA_ROOT to "!" inside the _WIN32 part at the top of luaconf.h.
- Add a function that uses GetModuleFileName() and luaL_gsub()
  to replace all "!"'s in a string with the dynamic path (with the
  executable name stripped). Put this inside the LUA_DL_DLL #ifdef.
- Apply this function to *both* the C and the Lua module paths
  (default paths or from environment).

A Lua binary stored in (say) 'D:\FooBar\lua51.exe' gets
  package.path ->  '?.lua;D:\FooBar\lua\?.lua;D:\FooBar\lua\?\init.lua'
  package.cpath -> '?.dll;D:\FooBar\dll\?.dll'

This should be a 10 line change and it only adds code for the Windows
case. I can create a patch, if needed.

Bye,
     Mike