possible NFS trouble

[Greg Troxel <gdt%lexort.com@localhost>]

My situation is a little complicated; hence "possible".

I had a setup where things were ok:

  2010 computer with 4 cores, 24G RAM, 67% tmpfs (so 16G), SSD with UFS2.
  netbsd-9 amd64

  lower-end 2010 computer 'xen' with 2 cores, 8G RAM, SSD with / and
  /usr UFS2 and most of it zfs.  Sometimes runs xen dom0, sometimes bare
  metal.
  netbsd-10 amd64

On the first computer, I build packages, and generally this is fine.
thunderbird has a crazy WRKDIR size requirement (22G), so I have

  .if "${PKGPATH}" == "mail/thunderbird"
  WRKOBJDIR=              /nfs/ztmp/work
  .endif
  WRKOBJDIR?=             /tmp/work
  CREATE_WRKDIR_SYMLINK=  yes

where ztmp is a zfs fs on the second computer, mounted via nfs, and thus
NFSv3 TCP.  I am doing this so that I abuse the non-critical ssd on the
second box, and not the ssd that I care about on the main box.  This is
slow but works, and it's only thunderbird.

The network between the computers is wired GbE with two switches and has
been reliable.

This setup worked fine, but I tended to use MAKE_JOBS 2 or 3.


The first computer had a hardware failure, and I replaced it with a 2019
8-core 32G box with a new SSD with everything except / sw /var /usr on
ZFS.  It seems to work very well.

However, my tmpfs is now 19G, which is still less than 22G, and I don't
really want to page  out everything  (after tmpfs, zfs ARC, other pools,
etc.), so I'm still using NFS ztmp.


I ran a build, and it was erroring out with IO errors, and restarting
kept having errors.  I suspected NFS concurrency, and reran it with
MAKE_JOBS=1 and it seems to have gone much better.

So it seems that perhaps

  8 jobs all doing NFS doesn't really work, on 10 vs 9, or 8 jobs vs 3
  jobs, or maybe only if server is a xen dom0, or ??

  Thunderbird is buggy and building thunderbird with high MAKE_JOBS
  doesn't really work.

  As always, something else.


I wonder if anyone has had a similar experience or insight.


I am likely to get some low-end SSD/flash to use for WKROBJDIR for
thunderbird, to avoid NFS, anwyay.