installing Lua 5.1 on AIX -- "last" word

[mfmorss@...]

I mentioned on Friday that after my installation of Lua 5.1 on AIX,
luaL_typerror and luaL_ref (and possibly other, untested functions) were
undefined at run time, and that calling these functions from my own .so
libs would cause lua to crash.  This was due, it seems, to these functions
not being called anywhere in liblua.a or in the main program.
Notwithstanding that the -bexpall flag was set, supposedly exporting all
necessary variables, this circumstance caused the AIX compiler, xlc, to
fail to export the given functions.

The solution, however, was NOT -- as suggested by someone who generously
offered assistance here -- to create a shared liblua.so against which
dynamically to link both the lua executables and my own .so libs.  In the
first place, it appears that lua does not want to load .so libs that are
not part of fully defined modules.  But working around that, compiling lua
against liblua.a and my own .so libs against liblua.so, gave rise many
"duplicate symbol" warnings when this procedure was applied to distributed
lua extensions such as Luis Carvalho's very promising numlua-0.2.

The correct solution turns out to be rather simple; but it involves adding
extra steps to the lua 5.1 build process when the build is for AIX.  Adding
these steps cannot be done simply by changing the existing ../src/makefile
parms for AIX.  Rather than taking it upon myself to recode
../src/makefile, however, I simply supply below a shell script that builds
AIX _and_ results in the "problem functions" being exported by the main
program.  (I will endeavor to produce a ../src/makefile that is general for
all current platforms if requested to; alternatively, I will be happy to
test one that someone else codes up.)

There are seven steps, as shown below, in the lua 5.1 build on any
platform; on AIX, it is necessary to split both Step 5 (build lua
executable) and Step 7 (build luac executable) into two substeps.  In each
case, the first substep is to use the xlc compiler to create an "export
file" that lists all the variables that should be exported by the compiled
executable.  This is a peculiar feature of xlc.  The second substep is to
compile the executable using the export file.  This two-substep procedure
for building executables replaces the one-step procedure employed on other
platforms, and which I attempted before to implement on AIX using the
"-bexpall" flag.  For some reason, "-bexpall" fails to convince xlc that
the "problem functions" need to be exported, while the use of an export
file in which these functions (among others) are listed explicitly does the
job.

Here is a reference from IBM documentation (which does not, however,
explain why xlc with -bexpall can fail to export "problem functions"):
http://publib.boulder.ibm.com/infocenter/comphelp/v8v101/index.jsp?topic=/com.ibm.xlcpp8a.doc/proguide/ref/compile_library.htm


The main conclusion for those of us (ha, ha) who work on AIX is that xlc's
-bexpall parm appears to be untrustworthy.

Shell script follows (in addition to -O2, many AIX users will probably want
to use -qmaxmem=-1 for object file compilation):

     # Step 1. make object files for liblua.a

     xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lapi.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lcode.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ldebug.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ldo.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ldump.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lfunc.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lgc.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c llex.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lmem.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lobject.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lopcodes.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lparser.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lstate.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lstring.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ltable.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ltm.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lundump.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lvm.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lzio.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lauxlib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lbaselib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ldblib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c liolib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lmathlib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c loslib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c ltablib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lstrlib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c loadlib.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c linit.c

     # Step 2. make liblua.a from given object files

      ar rcu liblua.a lapi.o lcode.o ldebug.o ldo.o ldump.o lfunc.o lgc.o
llex.o lmem.o lobject.o lopcodes.o lparser.o lstate.o lstring.o ltable.o
ltm.o lundump.o lvm.o lzio.o lauxlib.o lbaselib.o ldblib.o liolib.o
lmathlib.o loslib.o ltablib.o lstrlib.o loadlib.o linit.o

     # Step 3. make liblua.a a random library

      ranlib liblua.a

     # Step 4. make lua object file

      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c lua.c

     # Step 5a. make export file for lua

     xlc -qmkshrobj -qexpfile=expfile_1 lua.o liblua.a -lm -ldl

     # Step 5b. make lua executable with the given export file

      xlc -o lua -brtl -bE:expfile_1 lua.o liblua.a -lm -ldl

     # Step 6.  make luac object files

      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c luac.c
      xlc -O2 -DLUA_USE_POSIX -DLUA_USE_DLOPEN -c print.c

     # Step 7a. make export file for luac

      xlc -qmkshrobj -qexpfile=expfile_2 luac.o print.o liblua.a -lm -ldl

     # Step 7b. make luac executable with the given export file

      xlc -o luac -brtl -bE:expfile_2 luac.o print.o liblua.a -lm -ldl


Mark F. Morss
Principal Analyst, Market Risk
American Electric Power