Re: threaded programs report bogus process size ?

[Nicolas Joly <njoly%pasteur.fr@localhost>]

On Wed, Sep 30, 2009 at 01:59:24PM +0200, Nicolas Joly wrote:
> On Wed, Sep 30, 2009 at 12:16:30AM +0000, Christos Zoulas wrote:
> > In article <h9u2f5$qap$1%serpens.de@localhost>,
> > Michael van Elst <mlelstv%serpens.de@localhost> wrote:
> > >njoly%pasteur.fr@localhost (Nicolas Joly) writes:
> > >
> > >>I just noticed that threaded processes may report increasing process
> > >>size of their life ...
> > >
> > >libpthread never frees the pthread data structure and associated
> > >thread stack when a thread ends but puts it onto a private queue
> > >("deadqueue") and reclaims it again when a new thread is created.
> > >
> > >The reclaim can fail when a new thread is created shortly after
> > >an old thread ended because of a race with the kernel. As a result
> > >a new thread is allocated.
> > >
> > >So _some_ leakage is allowed by design.
> > >
> > >The problem is that the reclaim operation is only tried for the
> > >thread that ended last (the top of the "deadqueue"). So in a
> > >tight pthread_create/join loop the leaked threads just pile up.
> > >
> > >To correct this the reclaim operation should loop over the queue
> > >and try to reclaim any of the dead threads.
> > 
> > This seems to fix it.

I spoke a little too soon :(

Further tests (+ print loop iteration number) show that at some random
time, no more threads are created and the program seems stuck in an
endless loop ...

-- 
Nicolas Joly

Biological Software and Databanks.
Institut Pasteur, Paris.