Re: possible bug in lua.c main report lua_tostring malloc

[Sam Roberts <vieuxtech@...>]

On Fri, Jul 3, 2009 at 10:12 PM, David Manura<dm.lua@math2.org> wrote:
> On Fri, Jul 3, 2009 at 11:36 PM, David Manura wrote:
>> I was able to cause "lua -e 'error(5)'" to crash by triggering an
>> allocation failure in lmem.c:
>
> That modification to lmem.c was likely not valid (returns alloc
> failure after successfully allocating); I should have made the change
> instead to lauxlib.c:l_alloc.  The code may be fragile here, but I do
> not currently have a valid crash in practice.

If you are really interested in testing lua's handling of allocation
failure, I'd suggest creating a lua state with your own alloc function
pointer. You should be able to avoid all the fragile hacking of the
lua core code, and systematically fail every single allocation.

We used to do this at a company I worked with where we cared a lot. We
did this for cryptographic toolkits expected to run on embedded
devices, and we wrapped all our unit test suites in this kind of loop.
It basically systematically fails every single malloc that might
occur, not just at every call site, but in every call sequence (that
is exercised by the test code). I've never seen a C library that
survived a test loop like this, including ours, which we would write
knowing that they would have to pass.

The basic approach was this (lua uses realloc, not malloc, so it would
have to be adjusted):


int failon;

int allocations=0;

void *our_malloc(size) {
  if(failon == 0) {

    return 0;
}
  else {
    alloctions++
    return malloc(size);
}
our_free(void*v) {
   if(v) {
      allocations--;
      free(v)
}
}

int success;

for(failon = 0;;failon++) {
     L = lua_newstate( ... our_malloc, );
     int successfail = pcall(... run unit tests)
    lua_destroystate(L);

     // evaluate:
     // all memory should have been freed
     assert(allocations == 0);
     if(failon == 0) {
        // malloc failed, so expectation is unit test must have failed
(errored, i guess, in lua's case)
        // with an "out of memory" exception
       assert(successorfail == ENOMEM);
     }
     if(failon > 0) {
        //malloc didn't fail, so we ran unit test to completion,
result of unit test should be success
      assert(sucessorfail == SUCCESS)
     break;
    }
}