tech-pkg archive

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index][Old Index]

go124 not respecting MAKE_JOBS



This is a bug report that when building lang/go124, excessive amounts of
parallelization occur.

I have an 8-cpu computer and usually have MAKE_JOBS set to 4.  I don't
need to justify that, but my reason is it reduces fan noise and
interferes less with uses other than building packages.

Recently lang/go/go-module.mk got a bug fix so that builds of go progams
respect the pkgsrc-configured cpu limit, adding

  ALL_ENV+=      GOMAXPROCS=${MAKE_JOBS:U1}

However, that isn't used when building go itself.  There's a
long-standing not-explained MAKE_JOBS_SAFE=no, presumably an attempt to
work around go's default (mis)behavior of one thread per detected CPU.

Assuming that it is wise to ask go to use MAKE_JOBS cpus vs asking it to
use 1 and letting MAKE_JOBS function (because presumably that's how go's
build works and it doesn't use make that would get -j), it seems obvious
that the pattern of MAKE_JOBS=no and GOMAXPROCS should be for all builds
that use a go compiler with thread-greedy behavior.

I have the following which seems to sort of work and sort of not
entirely.  With MAKE_JOBS=1 my load average is staying in the 1.2-1.5
range.  With MAKE_JOBS=4 it peaked at 13.2.  Still, behavior was better
than without the change, where go's overuse of threads impacted
usability more strongly.

I'm going with the theory that this is a correct change but that there
may be more odd behavior lurking in go that needs adjusting.
Speculating wildly, maybe go_bootstrap is running multiple compiles at
once and perhaps in theory allocating to them from GOMAXPROCS but doing
the bookkeeping wrong.

Can anybody explain if this should be different?  Is there some other
env var for go itself when bootstrapping?

I lean to having lang/go/jobs.mk with (plus comments!):

        MAKE_JOBS_SAFE=		no
        ALL_ENV+=		GOMAXPROCS=${MAKE_JOBS:U1}

which is then included by all makefiles that run go compilers.  Plus
patching the env set into Makefiles with replaced targets that don't
include the environment.

If $MAINTAINER wants to fix this, that's of course great.  I see it as
necessary to end up in a situation where the user-specified concurrency
limit is respected.

Absent comments explaining why a different fix is appropriate, that
go/jobs.mk is bad, or that someone else wants to fix it, I'll add
lang/go/jobs.mk and hook it in to lang/go/go-foo and
lang/go124/Makefile, and then if all is ok we can add it to other
lang/goNN.


Index: Makefile
===================================================================
RCS file: /cvsroot/pkgsrc/lang/go124/Makefile,v
retrieving revision 1.1
diff -u -p -r1.1 Makefile
--- Makefile	25 Feb 2025 20:09:16 -0000	1.1
+++ Makefile	5 Apr 2025 13:30:34 -0000
@@ -93,12 +93,19 @@ PLIST_SUBST+=	GOVERSSUFFIX=${GOVERSSUFFI
 PRINT_PLIST_AWK+=	/^bin\/go${GOVERSSUFFIX}/ { print "bin/go$${GOVERSSUFFIX}"; next; }
 PRINT_PLIST_AWK+=	/^bin\/gofmt${GOVERSSUFFIX}/ { print "bin/gofmt$${GOVERSSUFFIX}"; next; }
 
+# Respect pkgsrc concurrency limits.
+# \todo Somehow hoist this to lang/go/jobs.mk ?
+MAKE_JOBS_SAFE=		no
+ALL_ENV+=		GOMAXPROCS=${MAKE_JOBS:U1}
+
 post-extract:
 	${RM} -r -f ${WRKSRC}/test/fixedbugs/issue27836*
 
+# \todo Respect BUILD_ENV in replaced target.
 do-build:
 	cd ${WRKSRC}/src && \
 		env \
+		GOMAXPROCS=${MAKE_JOBS:U1} \
 		GOROOT_BOOTSTRAP=${GOROOT_BOOTSTRAP:Q} \
 		${GOOPT} \
 		GOCACHE=${WRKDIR}/.cache/go-build \





Home | Main Index | Thread Index | Old Index