[ANN] Luabuild - a Highly Extensible Lua 5.2 Build System

[steve donovan <steve.j.donovan@...>]

Hi all,

Luabuild [1] was inspired by the general problem of getting a custom
Lua executable that would have its C extensions statically-linked in,
so that we could use srlua to glue it to our scripts.  The problem of
making a standalone script is mostly solved by Mathew's Squish and
Jay's soar, so the next step was a self-contained executable.

The basic idea is that you provide a configuration file, like so:

  exclude = 'debug math'
  include = 'lfs lpeg'

That is, leave out the standard Lua modules debug and math, and
statically-link in luafilesystem and LPeg.

And the basic idea was surprisingly straightforward, by using Lake [2]
as the cross-platform build engine. Lake already knows how to invoke
the Microsoft and Gnu compilers, and the irritating OS differences.
I've included a fresh version of lake, so all you need is Lua [3] and
lfs.

$ lua lake

The default build constructs the modules as DLLs/shared libraries and
generates a pretty bog-standard Lua 5.2 executable, except for a
luaconf.h patch that sets the Lua module path to be the output lib/
and lua/ directories. (This gives a sandboxed version that's easier to
test on POSIX systems)

Then to run the module tests.

 $ lua lake test

To get a statically linked fat Lua, do

$ lua lake STATIC=1
$ lua lake STATIC=1 test

It also includes a slightly patched version of lhf's srlua, so with a
config file like this:

-- lfs.config
include = 'lfs'
srlua = 'lua-fs'

which you can invoke with

$ lua lake CONFIG=lfs.config

(lake is very make-ish in this way)

Can then wrap up the LuaFileSystem test as an executable with

$ bin/glue bin/lua-fs modules/lfs/test.lua test-lfs

and ./test-lfs should do the right thing!

The following packages are included:

 - luafilesystem
 - luasocket
 - lpeg
 - luaposix
 - winapi
 - ltcltk

This represented some _serious_ work, because most of these chaps are
still Lua 5.1. So, ladies and gentlemen, at least I can present you
with a reasonably-tested set of Lua 5.2 core modules.

luabuild has been probably one of the hardest things I've done, since
there are so many testing possibilities.  Here are the
platforms/compilers I've got going:

- Win 7 64bit, MSVC 2010, gcc 4.6
- Win XP 32bit, MSVC 2010 gcc 4.6
- Ubuntu 32bit, gcc 4.4 and 4.5
- OS X 10.4 ppc
- Gumstix Linux, ARM
- SuSE Enterprise 10, amd64
- Crunchbang Debian, amd64

(You'll note that OS X is lagging quite a bit; you may have to set
MACOS_DEPLOYMENT_TARGET appropriately to get what you want)

Some of the test files have never been fired in anger on Windows, but
do work, apart from some corner cases (on MSVC, lfs.setmode() will
_crash_[4] if given anything except io.stdout or io.stdin).

I did not _intend_ to do this, but the need to test the results meant
that luabuild is basically a small source-only distribution. I note
that LuaDist (a large source-based distribution) is starting to think
about static linking as well, see this interesting comment by David M
[5]

The next stage is to pull in Jay's soar as a dependency analysis tool,
so (Lua application) -> (standalone application) can be made as simple
as possible.

Any comments welcome; this is still pretty raw!  Adding new packages
is very straightforward, if they aren't big monsters, and generally
the criterion would be for modules which are small and useful in an
embedded executable.  The readme does discuss the issues and provide
some background.

steve d.

[1] https://github.com/stevedonovan/luabuild
[2] https://github.com/stevedonovan/Lake
[3] Yes, it's a bootstrap - Lua 5.1 works fine here.
[4] I kid you not - the 'secure CRT enhancements' make the default
error behaviour an obscure crash through the infamous Dr Watson.
[5] https://github.com/LuaDist/Repository/issues/80