Subject: Re: pkgsrc on SMP machines
To: None <segv@netctl.net>
From: Hubert Feyrer <feyrer@cs.stevens.edu>
List: tech-pkg
Date: 12/15/2005 04:39:51
On Thu, 15 Dec 2005 segv@netctl.net wrote:
> Hi, can anyone suggest a good strategy for utilising the most of SMP hardware
> when building software from pkgsrc??
...
> Fixing all Makefiles to support -j > 1 seems quite unreasonable, I guess the
> best solution would be to have a multithreaded compiler, which distributes
> workload to a specified number of threads, i.e.:

How about compiling the pkgs themselves (i.e. whatever gets extracted from 
sources) with 'make -j', e.g. as the patch below does. At least that's how 
Unix is supposed to work...

Of course that's a crude hack, and a real solution would involving a 
user-settable variable plus a per-package maximum, and the real value 
would be the somewhere in between there.

(There is that other idea to build pkgsrc packages in parallel which is 
stuck to some people's head, but it grossly violates the KISS principle 
and involves all sort of nasty things like locking and whatnot; I still 
suggest not going there)


  - Hubert


Index: bsd.pkg.mk
===================================================================
RCS file: /cvsroot/pkgsrc/mk/bsd.pkg.mk,v
retrieving revision 1.1774
diff -u -r1.1774 bsd.pkg.mk
--- bsd.pkg.mk	5 Dec 2005 22:07:07 -0000	1.1774
+++ bsd.pkg.mk	15 Dec 2005 03:34:35 -0000
@@ -1895,7 +1895,7 @@
  .  for _dir_ in ${BUILD_DIRS}
  	${_PKG_SILENT}${_PKG_DEBUG}${_ULIMIT_CMD}			\
  	cd ${WRKSRC}; cd ${_dir_};					\
-	${SETENV} ${MAKE_ENV} ${MAKE_PROGRAM} ${BUILD_MAKE_FLAGS}	\
+	${SETENV} ${MAKE_ENV} ${MAKE_PROGRAM} -j17 ${BUILD_MAKE_FLAGS}	\
  		-f ${MAKEFILE} ${BUILD_TARGET}
  .  endfor
  .endif