http://lua-users.org/wiki/LuaBugs

[Roland Illig <roland.illig@...>]

Hi,

I patched a segmentation fault in luaV_concat and published it on the 
LuaBugs page.

Then GCW wrote:
> I am puzzled. My computer does not have segmented architecture.
Haha, funny.
> Surely the problem lies with your compiler, not the source code.
That's wrong.

The problem ist not my compiler. Please have a look at how the original 
author tries to avoid a size overflow and fails.

> typedef unsigned long lu_mem;
> #define MAX_SIZET ((size_t)(~(size_t)0)-2)
> [...]
> lu_mem tl;
> if (tl > MAX_SIZET) luaG_runerror(L, "string size overflow");

tl is of type lu_mem (unsigned long) and compared to a size_t constant.
  => tl should be of type size_t, too.
the test (tl > MAX_SIZET) will only detect (tl == ~(size_t)0 || tl == 
~(size_t)1). In most cases tl will overflow in the calculation before.
  => every addition should be checked on overflow.

This is what my patch does. I have sent this patch via private mail to 
Roberto first, as it might be security relevant. He already knew about 
the bug, but wasn't sure when to release the fix.

Roberto wrote:
> Many thanks for your feedback. We are already aware of this bug.
> (Like you, we are not sure whether to publish it without a new
> version of Lua fixing it.)

I strongly suggest that the patch be applied and released as soon as 
possible. An "out of memory" error (reported by the LuaVM) is much 
better than undefined behaviour (see the C standard for details).

Roland