RE: future of bytecode verifier

["John Hind" <john.hind@...>]

I agree strongly that the safety case for bytecode (that term grates with
me, but I'll stick with it this time) should be that the writer is
responsible for verification. But if this is done, I think the route for
bytecode reading should be fully separated from the "normal" route for Lua
source. There should be a different API function, a different Lua library
function and crucially a different lua_Reader (which would be a stub
implementation by default, as would be lua_Writer).

This approach makes Lua "safe by default" and anyone implementing bytecode
support is made responsible for the integrity of the bytecodes between write
and read operations (for example by restricting to a protected store, or
even by implementing cryptographic signing).

Can I also request that consideration be given to sorting out the
"endedness" problem for Lua bytecode? Either pick an endedness for the saved
format and convert if necessary on both write and read, or if this is
thought inefficient, have a flag for the endedness in the format and convert
on read if necessary. One of the benefits of bytecode should be full support
for heterogeneous computing platforms, and Lua does this now except for this
one little wrinkle (which, of course, makes it even more misleading to call
it "bytecode").

-----Original Message-----
From: Luiz Henrique de Figueiredo Sent: 04 March 2009 18:58

Following several bytecode exploits found by the relentless Peter Cawley
and others, we are considering dropping the bytecode verifier completely in
Lua 5.2. It seems useless to make a promise that we can't seem to deliver
without a much more complicated verifier than the current one, and possibly
with the need for costly runtime checks as well.

Our impression is that applications that are open to running arbitrary Lua
code provided by the user should avoid accepting precompiled scripts. So we
think that adding a flag to load (the Lua function from the base library)
to check for and reject precompiled scripts is enough for Lua-based apps
to be able to reject precompiled scripts if they want to. We don't think
anything else is needed in the C side, since you can always write you own
lua_Reader function to reject precompiled scripts.

At the same time, shedding the bytecode verifier would allow applications
that run their own precompiled scripts that are deemed safe to avoid the
cost of the bytecode verifier. The checks would be limited to the sanity
tests done in lundump.c, which should be enough for flagging accidental
file corruption.

All feedback is welcome.  Thanks.
--lhf

Attachment: smime.p7s
Description: S/MIME cryptographic signature