Re: high sys time, very very slow builds on new 24-core system

[Eduardo Horvath <eeh%NetBSD.org@localhost>]

On Wed, 23 Mar 2011, Thor Lancelot Simon wrote:

> I have a new machine with 24 2Ghz Opteron cores.  It has 32GB of RAM.
> 
> Building with sources on a fast SSD ("preloaded" into the page
> cache before the build using tar > /dev/null) and obj, dest, and rel
> dirs on tmpfs, system builds are extraordinarily slow.  The system
> takes about 20 minutes to build a netbsd-5 based source tree
> with -j24 -- about the same amount of time as an older 8-core Intel
> based system running netbsd-5 requires with -j8.
> 
> All cores spend well over 50% time in 'sys', even when all or almost
> all are running cc1 processes.  The kernel is amd64 -current GENERIC
> from about 1 week ago -- no DIAGNOSTIC, DEBUG, KMEMSTATS, LOCKDEBUG,
> etc.
> 
> Does anyone have any idea what might be wrong here?

Locking?  Try using lockstat.

Eduardo