Re: string.unpack segfaulting due to integer overflow and mixed sign comparisons

[Sam Roberts <vieuxtech@...>]

On Wed, Jun 27, 2012 at 1:14 AM, Chris Emerson
<chris-lua@mail.nosreme.org> wrote:
>> -    if (i+N>len) goto done;
>> +    if (i+N < i || i+N>len) goto done;
>
> I don't think this is right - in C overflow of signed integers is undefined
> behaviour.  The C compiler can (and some now do) assume that "i+N < i"
> (with N positive) can't happen, and that test can be optimised out.

Interesting, is this true for unsigned, too?

I hope that modular addition is well-defined over uint32_t, I use it a
bunch, and haven't had any problems, yet, but maybe haven't hit the
"right" kind of optomizing compiler.

Anyhow, signed overflow is possible in other place in struct, such as
it's atoi() variant, that does no bounds checking.

I'd another patch that converted all the int values that are never
allowed to be negative to size_t, but it was a lot more intrusive, and
it wasn't clear, given how struct handles indexes that are out of
bounds, how to maintain a backwards compatible API.

Probably lua should have a luaL_checkunsigned().

Cheers,
Sam