Subject: dlopen()/dlclose()
To: Johnny C. Lam <lamj@stat.cmu.edu>
From: Todd Vierling <tv@wasabisystems.com>
List: tech-userlevel
Date: 02/27/2001 15:22:48
[moved to tech-userlevel]

On Tue, 27 Feb 2001, Johnny C. Lam wrote:

: > Log Message:
: > Put back the patch to avoid dlclose(). Without the patch, it's not
: > possible to use LoadModule at all (httpd crashes soon after it forks).
: > This is most probably an Apache problem. Until the real problem
: > is tracked down, put this in as workaround.
: >
: > This addresses lib/12164 and bin/12283.
:
: I remember removing that change in the update of apache to 1.3.17 as
: the bug referenced in the patch, a mis-interaction between mod_cgi and
: mod_perl, seemed to have gone away.

This was my Apache PR, as well as my pkgsrc hack.  The bug did seem to go
away--but there's some other issues in play here.

I have a feeling that there's a nesting issue with dlopen(), dlclose(), and
shared libraries that have dependencies.  Take PHP or mod_perl, for
instance.  mod_perl used to be linked, before [the pkgsrc team] fully
understood the capabilities of the a.out and ELF ld.so's, such that the
httpd.conf had to manually load libperl.so, and libm.so as a dependency of
libperl.so.

Now mod_perl.so (and mod_php4.so) is linked against its libraries as
dependencies, so all that goop goes away.  libperl.so gets opened
automatically as a dependency of mod_perl.so in dlopen().  There's something
fishy going on relative to either the order in which Apache is dlclose()'ing
items and when ld.elf_so and/or ld.aout_so frees mapped libraries.

On a known buggy system, a way to find out the guts of the problem would be
to add debugging printfs to os/unix.c in Apache (print to stderr at every
dlopen() and dlclose()--print library name and handle pointer at open, and
handle pointer at close); also, run ld.so with as much debugging turned on
in the environment (and possibly with debugging printfs) as possible.

I already know that Apache reorders the shutdown hooks for modules, so it's
also advantageous to do an "apachectl restart" several times with this
debugging stuff in place.  That will cause a complete shutdown and restart
of the server, several cycles of dlclose() and dlopen().

-- 
-- Todd Vierling <tv@wasabisystems.com>  *  Wasabi NetBSD:  Run with it.
-- NetBSD 1.5 now available on CD-ROM  --  http://www.wasabisystems.com/