Re: bmake inefficiencies

[Roland Illig <roland.illig%gmx.de@localhost>]

On 26.01.2021 15:41, Mateusz Guzik wrote:
> I had a look with truss at what bmake is doing and I think a lot of
> the syscalls can be easily avoided. Examples come from FreeBSD with
> bmake-20210110 and I don't know to what extent they are representative
> for other systems.
>
> 1. spurious waitpid
>
> when doing buildkernel I counted the following:
> 7010 cases of error code 0 (== nothing to do)
> 371 cases of ECHILD
> 1526 cases of reaping

I'm not familiar enough with waitpid, so I will not touch this topic.
Simon?


> 2. FD_CLOEXEC
>
> for example mkstemp is used and FD_CLOEXEC performed on the result.
> Instead mkostemp can be used and mkTempFile can accept O_CLOEXEC as an
> argument to save explicit fcntl.

On current NetBSD and FreeBSD, this may be fine, for other platforms
there would need to be autoconf support.  Simon?


> 3. avoidable temp files
>
> JobOpenTmpFile opens a file and later checks if anything was written
> to it. Not an actual patch, but a hack to demonstrate the fix can be
> found at the end. Looks like checking can be done upfront in O(1) time
> (not done in the patch below).
>
> With this in place I get the following result for buildworld:
> 87930 JobOpenTmpFile: created!
> 70564 JobOpenTmpFile: skipped!
>
> i.e. about 44% of all temp files can be avoided at least in this workload.

This is something I understood and will tackle.  Thank you for doing the
analysis.  I think the main cause for this are chains of targets such as:

all: build
build: do-build
do-build: do-build-c
do-build-c: ${SRCS:M*.c:.c=.o}

Thanks for the patch, that looks both simple and correct.

Roland