Subject: Re: Performance Problem: malloc() is calling madvise()
To: Bill Dorsey <dorsey@lila.com>
From: Simon Burge <simonb@netbsd.org>
List: current-users
Date: 05/19/2000 21:38:46
[[ Moving to current-users - netbsd-ports is for discussions
   about porting NetBSD to new architectures. ]]

"Bill Dorsey" wrote:

> I upgraded to the 20000505 snapshot yesterday (1.4Y) and noticed
> a serious performance loss when running one of my programs.
> 
> The program I was running is a number-crunching program that
> uses malloc() pretty heavily (via the C++ new function).
> Between the 1.4.2_Alpha release I was running and the 1.4Y
> release I upgraded to, the program slowed down by a factor
> of 4 on my Personal Workstation!!!
> 
> Looking at top, I had my first clue as to what the problem
> was.  It was showing about 33% of the CPU time being spent
> running my program and 66% of the time in the system.  I
> proceeded to run a ktrace on the program and was rewarded
> with data at the rate of more than 1 megabyte per second
> being written to ktrace.out!
> 
> After stopping the ktrace, I ran kdump to see what was going
> on.  The madvise() system call was being called millions of
> times per second from what I could see.

While not explaining why malloc() does this, using something like

	env MALLOC_OPTIONS=h <your-program>

should turn off the calls to madvise().  You can also add

	extern char *malloc_options;
	...
	malloc_options = "h";

to your program (set malloc_options before the first call to malloc()
- I don't know any C++ so this may or may not be possibile if you use
contrustors or what-not).

The man page for malloc() talks more about this.  Hmm, it also says the
calls to madvise() are off by default!  FreeBSD changed the default
from on to off at some stage - we grabbed their code before the change
and their manpage after it.  They changed it because "Recent VM system
changes have made this too expensive." - our madvise() manpage says that
until NetBSD 1.5 (ie in -current between 1.4 and 1.5), madvise() did
nothing.  Anyone object to me changing the default to match the manpage
and what FreeBSD currently does?

Simon.