Subject: tools/Makefile parallel make issue
To: None <tech-toolchain@NetBSD.org>
From: Lubomir Sedlacik <salo@Xtrmntr.org>
List: tech-toolchain
Date: 07/06/2006 17:25:02
--lRF4gxo9Z9M++D0O
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
hi,
yesterday, my "tools" build failed while building a i386 distribution
=66rom the same sources with which i just hours ago successfully built
sparc64 sets.
i always build from read-only source tree, with a clean TOOLDIR and
OBJDIR (nonexistent, actually), e.g., typically like this:
./build.sh -O /opt/obj -T /opt/tools -U -j 2 tools
the failure was caused by a "cleandir" running in parallel within the
same directory "dependall" was just spawned (running configure at the
moment), in tools/compat.
the problem seems to lie in src/tools/Makefile:
realall realdepend: .MAKE
.if "${PREVIOUSTOOLDIR}" !=3D "${TOOLDIR}"
@echo "*** WARNING: TOOLDIR has moved?"
@echo "*** PREVIOUSTOOLDIR '${PREVIOUSTOOLDIR}'"
@echo "*** !=3D TOOLDIR '${TOOLDIR}'"
@echo "*** Cleaning mis-matched tools"
rm -f PREVIOUSTOOLDIR
(cd ${.CURDIR} && ${MAKE} cleandir)
.endif
there was no recorded PREVIOUSTOOLDIR, since the tree is clean and
OBJDIR was empty, the subshell make "cleandir" is spawned.
incidentally, at the same moment the other make run was already doing
"dependall" in tools/compat and was being swept clean in the middle of a
configure run causing the build to fail (some unrelated targets removed
=66rom the log, full log at http://xtrmntr.org/salo/tmp/build.log.i386):
cleandir =3D=3D=3D> compat
--- dependall-compat ---
checking for u_int... no
checking for u_quad_t... yes
checking for socklen_t... yes
checking for uint8_t... yes
--- realdepend ---
--- cleandepend ---
--- dependall-compat ---
checking for u_int8_t... yes
--- realdepend ---
rm -f .depend atoll.d basename.d bt_close.d bt_conv.d bt_debug.d bt_delet=
e.d bt_get.d bt_open.d bt_overflow.d bt_page.d bt_put.d bt_search.d bt_seq.=
d bt_split.d bt_utils.d db.d dirname.d err.d errx.d fgetln.d flock.d fparse=
ln.d fts.d getmode.d getopt_long.d gettemp.d glob.d hash.d hash_bigkey.d ha=
sh_buf.d hash_func.d hash_log2.d hash_page.d heapsort.d issetugid.d lchflag=
s.d lchmod.d lchown.d libyywrap.d md2.d md2hl.d md4c.d md4hl.d md5c.d md5hl=
=2Ed mkdtemp.d mkstemp.d mpool.d pread.d putc_unlocked.d pw_scan.d pwcache.=
d pwrite.d rec_close.d rec_delete.d rec_get.d rec_open.d rec_put.d rec_sear=
ch.d rec_seq.d rec_utils.d rmd160.d rmd160hl.d setenv.d setgroupent.d setpa=
ssent.d setprogname.d sha1.d sha1hl.d sha2.d sha256hl.d sha384hl.d sha512hl=
=2Ed snprintf.d strlcat.d strlcpy.d strmode.d strsep.d strsuftoll.d strtoll=
=2Ed unvis.d verr.d verrx.d vis.d vwarn.d vwarnx.d warn.d warnx.d /usr/src/=
tools/compat/tags=20
--- dependall-compat ---
checking for uint16_t... yes
checking for u_int16_t... yes
checking for uint32_t... yes
--- realdepend ---
--- cleanlib ---
rm -f a.out [Ee]rrs mklog core *.core libnbcompat.a atoll.lo basename.lo=
dirname.lo fgetln.lo flock.lo fparseln.lo getmode.lo getopt_long.lo gettem=
p.lo heapsort.lo issetugid.lo lchflags.lo lchmod.lo lchown.lo libyywrap.lo =
md2.lo md2hl.lo md4c.lo md4hl.lo md5c.lo md5hl.lo mkdtemp.lo mkstemp.lo pre=
ad.lo putc_unlocked.lo pwcache.lo pwrite.lo pw_scan.lo rmd160.lo rmd160hl.l=
o setenv.lo setgroupent.lo setpassent.lo setprogname.lo sha1.lo sha1hl.lo s=
ha2.lo sha256hl.lo sha384hl.lo sha512hl.lo snprintf.lo strlcat.lo strlcpy.l=
o strmode.lo strsep.lo strsuftoll.lo strtoll.lo unvis.lo vis.lo err.lo errx=
=2Elo verr.lo verrx.lo vwarn.lo vwarnx.lo warn.lo warnx.lo fts.lo glob.lo b=
t_close.lo bt_conv.lo bt_debug.lo bt_delete.lo bt_get.lo bt_open.lo bt_over=
flow.lo bt_page.lo bt_put.lo bt_search.lo bt_seq.lo bt_split.lo bt_utils.lo=
db.lo hash.lo hash_bigkey.lo hash_buf.lo hash_func.lo hash_log2.lo hash_pa=
ge.lo mpool.lo rec_close.lo rec_delete.lo rec_get.lo rec_open.lo rec_put.lo=
rec_search.lo rec_seq.lo rec_utils.lo config.log config.status configure.l=
ineno *.stamp =20
--- cleandir ---
rm -f nbtool_config.h confdefs.h defs.mk
rm -r -f include
rm -f config.cache
--- dependall-compat ---
checking for u_int32_t... cat: confdefs.h: No such file or directory
--- dependall-compat ---
no
checking for uint32_t... (cached) yes
--- dependall-compat ---
checking for uint64_t... no
checking for u_int64_t... no
configure: error: cannot find a suitable type for uint64_t
--- dependall-compat ---
*** [config.cache] Error code 1
1 error
=20
nbmake: stopped in /usr/src/tools/compat
*** [dependall-compat] Error code 2
--- realdepend ---
A failure has been detected in another branch of the parallel make
=20
nbmake: stopped in /usr/src/tools/makewhatis
*** [cleandir-makewhatis] Error code 2
1 error
=20
nbmake: stopped in /usr/src/tools
*** [realdepend] Error code 2
2 errors
=20
nbmake: stopped in /usr/src/tools
=20
ERROR: Failed to make dependall tools
*** BUILD ABORTED ***
at that point i removed the TOOLDIR and OBJDIR again, and tried to build
sparc64 tools. it didn't fail! but after i examined the build log i
noticed the same thing happened again just in a different test and thus
the configure survived just by pure luck! (some unrelated targets
removed, full log at http://xtrmntr.org/salo/tmp/build.log.sparc64):
cd /usr/src/tools && /opt/tools/bin/nbmake cleandir)
--- dependall-compat ---
dependall =3D=3D=3D> compat
--- config.cache ---
rm -f config.cache
CC=3Dcc CFLAGS=3D-O LDFLAGS=3D sh /usr/src/tools/compat/configure --cach=
e-file=3Dconfig.cache
configure: creating cache config.cache
checking for gcc... cc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...=20
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether cc accepts -g... yes
checking for cc option to accept ANSI C... none needed
checking how to run the C preprocessor... cc -E
checking for egrep... grep -E
checking for NetBSD... yes
checking for sh... /bin/sh
checking whether byte ordering is bigendian... no
checking for ANSI C header files... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking zlib.h usability... yes
checking zlib.h presence... yes
checking for zlib.h... yes
checking for gzdopen in -lz... yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/mman.h usability... yes
checking sys/mman.h presence... yes
checking for sys/mman.h... yes
checking sys/mtio.h usability... yes
checking sys/mtio.h presence... yes
checking for sys/mtio.h... yes
checking sys/param.h usability... yes
checking sys/param.h presence... yes
checking for sys/param.h... yes
checking sys/socket.h usability... yes
checking sys/socket.h presence... yes
checking for sys/socket.h... yes
checking for sys/stat.h... (cached) yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking for sys/types.h... (cached) yes
checking sys/utsname.h usability... yes
checking sys/utsname.h presence... yes
checking for sys/utsname.h... yes
checking sys/wait.h usability... yes
checking sys/wait.h presence... yes
checking for sys/wait.h... yes
checking assert.h usability... yes
checking assert.h presence... yes
checking for assert.h... yes
checking ctype.h usability... yes
checking ctype.h presence... yes
checking for ctype.h... yes
checking errno.h usability... yes
checking errno.h presence... yes
checking for errno.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking grp.h usability... yes
checking grp.h presence... yes
checking for grp.h... yes
checking limits.h usability... yes
checking limits.h presence... yes
checking for limits.h... yes
checking locale.h usability... yes
checking locale.h presence... yes
checking for locale.h... yes
checking netdb.h usability... yes
checking netdb.h presence... yes
checking for netdb.h... yes
checking pwd.h usability... yes
checking pwd.h presence... yes
checking for pwd.h... yes
checking signal.h usability... yes
checking signal.h presence... yes
checking for signal.h... yes
checking stdarg.h usability... yes
checking stdarg.h presence... yes
checking for stdarg.h... yes
checking stdio.h usability... yes
checking stdio.h presence... yes
checking for stdio.h... yes
checking for stdlib.h... (cached) yes
checking for string.h... (cached) yes
checking termios.h usability... yes
checking termios.h presence... yes
checking for termios.h... yes
checking for unistd.h... (cached) yes
checking for dirent.h that defines DIR... yes
checking for library containing opendir... none required
checking sys/sysmacros.h usability... no
checking sys/sysmacros.h presence... no
checking for sys/sysmacros.h... no
checking sys/syslimits.h usability... yes
checking sys/syslimits.h presence... yes
checking for sys/syslimits.h... yes
checking getopt.h usability... yes
checking getopt.h presence... yes
checking for getopt.h... yes
checking features.h usability... no
checking features.h presence... no
checking for features.h... no
checking malloc.h usability... yes
checking malloc.h presence... yes
checking for malloc.h... yes
checking sys/poll.h usability... yes
checking sys/poll.h presence... yes
checking for sys/poll.h... yes
checking stddef.h usability... yes
checking stddef.h presence... yes
checking for stddef.h... yes
checking sys/bswap.h usability... yes
checking sys/bswap.h presence... yes
checking for sys/bswap.h... yes
checking machine/bswap.h usability... yes
checking machine/bswap.h presence... yes
checking for machine/bswap.h... yes
checking sys/cdefs.h usability... yes
checking sys/cdefs.h presence... yes
checking for sys/cdefs.h... yes
checking machine/endian.h usability... yes
checking machine/endian.h presence... yes
checking for machine/endian.h... yes
checking sys/featuretest.h usability... yes
checking sys/featuretest.h presence... yes
checking for sys/featuretest.h... yes
checking err.h usability... yes
checking err.h presence... yes
checking for err.h... yes
checking for inttypes.h... (cached) yes
checking libgen.h usability... yes
checking libgen.h presence... yes
checking for libgen.h... yes
checking paths.h usability... yes
checking paths.h presence... yes
checking for paths.h... yes
checking for stdint.h... (cached) yes
checking util.h usability... yes
checking util.h presence... yes
checking for util.h... yes
checking rpc/types.h usability... yes
checking rpc/types.h presence... yes
checking for rpc/types.h... yes
checking netconfig.h usability... yes
checking netconfig.h presence... yes
checking for netconfig.h... yes
checking for size_t... yes
checking for id_t... yes
checking for long long... yes
checking for u_long... no
checking for u_char... no
checking for u_short... no
checking for u_int... no
checking for u_quad_t... yes
checking for socklen_t... yes
checking for uint8_t... yes
checking for u_int8_t... yes
checking for uint16_t... yes
checking for u_int16_t... yes
checking for uint32_t... yes
checking for u_int32_t... yes
checking for uint64_t... yes
checking for u_int64_t... yes
checking for DIR.dd_fd... no
checking for struct dirent.d_namlen... yes
checking for struct stat.st_flags... yes
checking for struct stat.st_gen... yes
checking for struct stat.st_birthtime... yes
--- cleandir-compat ---
cleandir =3D=3D=3D> compat
--- cleandepend ---
rm -f .depend atoll.d basename.d bt_close.d bt_conv.d bt_debug.d bt_delet=
e.d bt_get.d bt_open.d bt_overflow.d bt_page.d bt_put.d bt_search.d bt_seq.=
d bt_split.d bt_utils.d db.d dirname.d err.d errx.d fgetln.d flock.d fparse=
ln.d fts.d getmode.d getopt_long.d gettemp.d glob.d hash.d hash_bigkey.d ha=
sh_buf.d hash_func.d hash_log2.d hash_page.d heapsort.d issetugid.d lchflag=
s.d lchmod.d lchown.d libyywrap.d md2.d md2hl.d md4c.d md4hl.d md5c.d md5hl=
=2Ed mkdtemp.d mkstemp.d mpool.d pread.d putc_unlocked.d pw_scan.d pwcache.=
d pwrite.d rec_close.d rec_delete.d rec_get.d rec_open.d rec_put.d rec_sear=
ch.d rec_seq.d rec_utils.d rmd160.d rmd160hl.d setenv.d setgroupent.d setpa=
ssent.d setprogname.d sha1.d sha1hl.d sha2.d sha256hl.d sha384hl.d sha512hl=
=2Ed snprintf.d strlcat.d strlcpy.d strmode.d strsep.d strsuftoll.d strtoll=
=2Ed unvis.d verr.d verrx.d vis.d vwarn.d vwarnx.d warn.d warnx.d /usr/src/=
tools/compat/tags=20
--- cleanlib ---
rm -f a.out [Ee]rrs mklog core *.core libnbcompat.a atoll.lo basename.lo=
dirname.lo fgetln.lo flock.lo fparseln.lo getmode.lo getopt_long.lo gettem=
p.lo heapsort.lo issetugid.lo lchflags.lo lchmod.lo lchown.lo libyywrap.lo =
md2.lo md2hl.lo md4c.lo md4hl.lo md5c.lo md5hl.lo mkdtemp.lo mkstemp.lo pre=
ad.lo putc_unlocked.lo pwcache.lo pwrite.lo pw_scan.lo rmd160.lo rmd160hl.l=
o setenv.lo setgroupent.lo setpassent.lo setprogname.lo sha1.lo sha1hl.lo s=
ha2.lo sha256hl.lo sha384hl.lo sha512hl.lo snprintf.lo strlcat.lo strlcpy.l=
o strmode.lo strsep.lo strsuftoll.lo strtoll.lo unvis.lo vis.lo err.lo errx=
=2Elo verr.lo verrx.lo vwarn.lo vwarnx.lo warn.lo warnx.lo fts.lo glob.lo b=
t_close.lo bt_conv.lo bt_debug.lo bt_delete.lo bt_get.lo bt_open.lo bt_over=
flow.lo bt_page.lo bt_put.lo bt_search.lo bt_seq.lo bt_split.lo bt_utils.lo=
db.lo hash.lo hash_bigkey.lo hash_buf.lo hash_func.lo hash_log2.lo hash_pa=
ge.lo mpool.lo rec_close.lo rec_delete.lo rec_get.lo rec_open.lo rec_put.lo=
rec_search.lo rec_seq.lo rec_utils.lo config.log config.status configure.l=
ineno *.stamp =20
--- cleandir ---
rm -f nbtool_config.h confdefs.h defs.mk
rm -r -f include
rm -f config.cache
--- dependall-compat ---
checking for struct stat.st_atim... no
checking for struct stat.st_mtimensec... cat: confdefs.h: No such file or=
directory
yes
checking for struct statvfs.f_iosize... yes
checking whether optind is declared... yes
checking whether optreset is declared... yes
checking whether sys_signame is declared... yes
checking for working alloca.h... no
checking for alloca... yes
--- dependall-compat ---
checking for atoll... yes
checking for asprintf... yes
checking for asnprintf... no
checking for basename... yes
--- dependall-compat ---
checking for devname... yes
checking for dirfd... no
--- dependall-compat ---
checking for dirname... yes
checking for fgetln... yes
--- dependall-compat ---
checking for flock... yes
--- dependall-compat ---
checking for fparseln... yes
-- dependall-compat ---
checking for futimes... yes
checking for getopt... yes
checking for getopt_long... yes
-- dependall-compat ---
checking for group_from_gid... yes
-- dependall-compat ---
checking for heapsort... yes
-- dependall-compat ---
checking for isblank... yes
-- dependall-compat ---
checking for issetugid... yes
-- dependall-compat ---
checking for lchflags... yes
-- dependall-compat ---
checking for lchmod... yes
-- dependall-compat ---
checking for lchown... yes
checking for lutimes... yes
checking for mkstemp... yes
checking for mkdtemp... yes
-- dependall-compat ---
checking for poll... yes
-- dependall-compat ---
checking for pread... yes
-- dependall-compat ---
checking for putc_unlocked... yes
-- dependall-compat ---
checking for pwcache_userdb... yes
checking for pwrite... yes
checking for random... yes
-- dependall-compat ---
checking for setenv... yes
-- dependall-compat ---
checking for setgroupent... yes
-- dependall-compat ---
checking for setprogname... yes
checking for setpassent... yes
checking for snprintf... yes
checking for strlcat... yes
-- dependall-compat ---
checking for strlcpy... yes
-- dependall-compat ---
checking for strsep... yes
checking for strsuftoll... yes
checking for strtoll... yes
checking for user_from_uid... yes
-- dependall-compat ---
checking for vasprintf... yes
-- dependall-compat ---
checking for vasnprintf... no
-- dependall-compat ---
checking for vsnprintf... yes
-- dependall-compat ---
checking whether strsuftoll is declared... yes
-- dependall-compat ---
checking whether htobe16 is declared... yes
checking whether htobe32 is declared... yes
checking whether htobe64 is declared... yes
checking whether htole16 is declared... yes
checking whether htole32 is declared... yes
checking whether htole64 is declared... yes
-- dependall-compat ---
checking whether be16toh is declared... yes
checking whether be32toh is declared... yes
checking whether be64toh is declared... yes
checking whether le16toh is declared... yes
checking whether le32toh is declared... yes
checking whether le64toh is declared... yes
checking whether bswap16 is declared... yes
checking whether bswap32 is declared... yes
checking whether bswap64 is declared... yes
checking whether fstatvfs is declared... yes
checking whether setgroupent is declared... yes
checking whether setpassent is declared... yes
checking for regfree in -lregex... no
checking for library containing regfree... none required
not updating unwritable cache config.cache
configure: creating ./config.status
config.status: creating defs.mk
config.status: creating nbtool_config.h
i don't have fix for this, sorry, my parallel make-fu is too weak.
also, i wonder how come nobody else has noticed until now? the code is
there since 2003, it seems.
suggestions how to properly fix the issue welcome, thanks.
regards,
--=20
-- Lubomir Sedlacik <salo@{NetBSD,Xtrmntr,silcnet}.org> --
--lRF4gxo9Z9M++D0O
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (NetBSD)
iD8DBQFErStOiwjDDlS8cmMRAugVAJ9s56f+r6GWH9cKx8X0HrT0Gh4OGgCeLIZn
CeFQSgMqsGmDHxl0KF5iIrQ=
=IQ7T
-----END PGP SIGNATURE-----
--lRF4gxo9Z9M++D0O--