Re: string.unpack segfaulting due to integer overflow and mixed sign comparisons

[Joshua Phillips <jp.sittingduck@...>]

On Wed, Jun 27, 2012 at 09:14:23AM +0100, Chris Emerson wrote:
> On Tue, Jun 26, 2012 at 03:51:43PM -0700, Sam Roberts wrote:
> > Index: pack/lpack.c
> > ===================================================================
> > --- pack/lpack.c        (revision 27854)
> > +++ pack/lpack.c        (working copy)
> > @@ -129,7 +129,7 @@
> >     case OP_STRING:
> >     {
> >      ++N;
> > -    if (i+N>len) goto done;
> > +    if (i+N < i || i+N>len) goto done;
> 
> I don't think this is right - in C overflow of signed integers is undefined
> behaviour.  The C compiler can (and some now do) assume that "i+N < i"
> (with N positive) can't happen, and that test can be optimised out.
> 
> I haven't looked at it in enough detail to provide the right fix, sorry.
> :-)

You're right, you can't do the check after the overflow, because the
overflow itself is undefined behaviour.
This document goes into more detail (and shows how hard it is to use C
safely!): http://www.fefe.de/intof.html