Re: Advice on dlclose crash debugging

["Alexander Gladysh" <agladysh@...>]

>> OS X 10.5.3, i686-apple-darwin9-gcc-4.0.1, gdb 6.3.50-20050815
>> Lua 5.1.3 with several third-party modules, all built as C++.
>>
>> In my application after Lua error I sometimes (in roughly one of five
>> runs) get a segfault on lua_close() in module unload.
>>
>> Lua-related part of the stack trace:
>>
>> #0  0x8fe1d10a in __dyld_munmap$UNIX2003 ()
>> #1  0x8fe13a03 in __dyld__ZN16ImageLoaderMachOD0Ev ()
>> #2  0x8fe03206 in __dyld__ZN4dyld20garbageCollectImagesEv ()
>> #3  0x8fe0bd46 in __dyld_dlclose ()
>> #4  0x97197bae in dlclose ()
>> #5  0x00fcbcdf in gctm (L=0x0) at src/loadlib.c:64
>> #6  0x00fc2bdf in luaD_precall (L=0x1228d80, func=0x18cf20c,
>> nresults=0) at src/ldo.c:319
>> #7  0x00fc3173 in luaD_call (L=0x1209f00, func=0x18cf20c, nResults=0)
>> at src/ldo.c:376
>> #8  0x00fc5766 in GCTM (L=0x1209f00) at src/lgc.c:467
>> #9  0x00fc57e8 in luaC_callGCTM (L=0x1209f00) at src/lgc.c:479
>> #10 0x00fc1a8a in luaD_rawrunprotected (L=0x1209f00, f=0xfd2900
>> <callallgcTM>, ud=0x0) at src/ldo.c:116
>> #11 0x00fd298e in lua_close (L=0x1209f00) at src/lstate.c:205
>>
>> If I build Lua and my application with -DLUA_USE_APICHECK and
>> -Dlua_assert=assert I still get the same segfault. I have checked my
>> application against multiple links to lua shared library, and found
>> nothing.
>>
>> I'd like to know at least which module crashes. Any advice on the best
>> way to get this info?
>
> By "which module" you mean what C module it is dl-closing when it
> crashes?

Yes, I do.

> Lua closes these modules in the reverse order that they were
> created. So, if you print the module name at each call to ll_load, then
> you will know what each call to ll_unloadlib is closing.

Great! It turns out to be crashing on unload of LFS then... (I think
the flaw should be somewhere on my side though, not in LFS itself.)
 Also, I've just googled up useful dyld environment variable to dump
shared module loads and unloads on Mac:

  $ DYLD_PRINT_LIBRARIES_POST_LAUNCH=1 ./my_executable

The output confirms that crash occurs before LFS is unloaded (and
after module before LFS, libmime, is unloaded).

With that environment variable on, on crash (that is in 1/5 of all
runs) instead of stack dump, runtime complains on pure virtual method
call. Which is weird because LFS is written in plain C, and only
compiled as C++ (so, again, it looks more like corruption is somewhere
on my side).

dyld: unloaded: path/libmime.so
pure virtual method called

I was not able to reproduce this crash at all under GDB with
DYLD_PRINT_LIBRARIES_POST_LAUNCH=1.

(On to the other hand, I've seen *once*, of, maybe, hundred, that it
crashed just before it said libmime.so is unloaded.)

> (BTW, there are some strange things in your trace that may be caused by some
> optimizations. In entry #5, loadlib.c:64 is function ll_unloadlib,
> not gctm, and there is no argument 'L' there...)

You're right, thanks! It turns out that I'm building Lua and C module
shared libraries with -O3 (and my application with -O0). I was not
able to reproduce this segfault with Lua and modules built with -O0...

Alexander.