On 13/12/2018 16:47, Jonathan Perkin wrote:
Well the last failing run with my log parser was a fairly simple perl script that just kept track of which package was being built and redirected the detailed output to a separate file (using buffered IO) for each package. I think in this case there was a genuine IO error on stdout as there was a package built with bmake (spamassassin) which just fell over during the make install step with truncated output to stdout.I used to see this error a lot when running bulk builds where the output was being logged to an NFS share. Switching it to a local file system not only improved performance, it made this problem disappear.
While I can appreciate that this pipe or your file IO could get behind that shouldn't cause errors on the pipe sender side. The sender should get blocked until all the io completes at the point the process file handles close. While I can appreciate with a MAKE_JOBS=8 build on an 8 core machine there may not be alot of spare CPU cycles it still indicates a bug somewhere. I guess the key is to see what unexpected error is creeping out of the underlying write syscall to fail (assuming stdout gets routed through __swrite in stdio.c)
Was that a straight io redirect? Perhaps the key is a slightly slow consumer of stdout data then which could point to libc rather than the kernel. At least I know its not some odd quirk of my system.I never figured out why exactly it was happening.
Mike