luac with multiple files, broken due to _ENV in 5.2.0-alpha

[David Manura <dm.lua@...>]

In 5.2.0-alpha, luac with multiple files doesn't work:

  $ echo x=1 > 1.lua
  $ echo y=1 > 2.lua
  $ luac -o 12.luac 1.lua 2.lua
  $ lua 12.luac
  ./src/lua: 1.lua:1: attempt to index upvalue '_ENV' (a function value)
  stack traceback:
          1.lua:1: in main chunk
          (luac): in main chunk
          [C]: in ?

I can't say that the way this behaved in 5.1 was always that desirable
though.  In 5.1 this provides a way to combine chunks (even previously
compiled ones) into a larger compiled chunk that preserves debugging
info like source/line numbers of the original smaller chunks and
merely executes the smaller chunks in sequence, passing no arguments
and ignoring their return values:

  $ echo "print('test', ...); return 3" > a.lua
  $ luac -o a.luac a.lua
  $ lua a.luac 1 2
  test    1       2
  $ luac -o a.luac a.lua a.luac
  $ lua -e "print(select('#', loadfile'a.luac'(1,2)))"
  test
  test
  0

To combine chunks in other ways, such as chunks that are modules,
there are solutions listed on the bottom of [1], including hacks like
luac.lua [2].

We might, however, generalize luac's bytecode linking to allow chunks
to acquire *references to other chunks*.  Consider, for example, two
files:

  -- a.lua
  print('a', ...); return 1

  -- b.lua
  print('b', ...); return 2

that you want to effectively link together like this:

  -- combined.lua
  local a = function(...) print('a', ...) return 1 end
  local b = function(...) print('b', ...) return 2 end
  return a(...) and b(22)

but preserve debugging info (e.g. "a.lua:2" and "b.lua:2").  The
linking could be specified in a separate file like this:

  -- link.lua
  local a, b = function() end, function() end -- placeholders
  return a(...) and b(22)

where the linker will patch a/b with references to the real chunks.
You could then link these with

  luac -o combined.luac a.lua b.lua link.lua

or even

  luac -o a.luac a.lua
  luac -o b.luac b.lua
  luac -o link.luac link.lua
  luac -o combined.luac a.luac b.luac link.luac

The combined.luac will behave just like a compiled combined.lua, and
it will be bytecode equivalent except that debugging info will be
properly preserved from the original files.  By suitably defining
link.lua, you can reproduce the link behavior of Lua 5.1, luac.lua
[2], and others.

Yes, you can already achieve something similar with

  -- link.lua
  local a = loadstring([[......escaped bytecodes......]], "a.lua")
  local b = loadstring([[......escaped bytecodes......]], "b.lua")
  return a(...) and b(22)

but the bytecode encoding will be more complex than you might prefer.

It almost seems like we should be able to link it in Lua code:

  local debug = require 'debug'
  local a = assert(loadfile '1.lua')
  local b = assert(loadfile '2.lua')
  local c = loadstring[[
    local a,b
    return function(...) return a(...) and b(22) end
  ]]()
  debug.setupvalue(c, 1, a)
  debug.setupvalue(c, 2, b)
  local fh = assert(io.open('combined.luac', 'wb'))
  fh:write(string.dump(c))
  fh:close()

but string.dump doesn't dump upvalues, and we prefer to encode the
chunks not as upvalues but as constants instantiated with CLOSURE
opcodes.

[1] http://lua-users.org/wiki/BinToCee
[2] http://lua-users.org/lists/lua-l/2008-08/msg00092.html - lhf's
luac.lua supporting require