Re: port-amd64/57266: Poor disk performance on VirtualBox vioscsi0 and NetBSD 10.0_BETA

[Bartek Krawczyk <bbartlomiej.mail%gmail.com@localhost>]

The following reply was made to PR port-amd64/57266; it has been noted by GNATS.

From: Bartek Krawczyk <bbartlomiej.mail%gmail.com@localhost>
To: Robert Elz <kre%munnari.OZ.AU@localhost>
Cc: gnats-bugs%netbsd.org@localhost, port-amd64-maintainer%netbsd.org@localhost,
 netbsd-bugs%netbsd.org@localhost
Subject: Re: port-amd64/57266: Poor disk performance on VirtualBox vioscsi0
 and NetBSD 10.0_BETA
Date: Sun, 12 Mar 2023 19:06:31 +0100

 On 12/03/2023 14:00, Robert Elz wrote:
 >    | In general this is purely default NetBSD 10.0_BETA installation. The
 >    | only options I changed is adding log, async and noatime to / mount in
 >    | the installer.
 > 
 > That's a combination that makes no sense, and I thought had been made
 > impossible, but you seem to have achieved it.
 > 
 > "async" says "just do all I/O to buffers, in any order, write to the
 > disc when necessary, in any order, filesystem consistency if the system
 > shoukd crash before umount is not guaranteed, and is often a complete
 > disaster".  I use that a lot for trash fikesystems I can just newfs if
 > needed, but want to run as fast as possible - like /usr/obj.  I also use
 > it for src/pkgsrc while unpacking as I can always newfs and start again
 > if something goes wrong.   I'd never recommend it for root though.
 > 
 > "log" delays meta-data writes, but keeps track of what should be done
 > and in what order so the filesystem can be made consistent after a crash.
 > 
 > Using neither causes meta-data to be written synchronously, and in order,
 > so the filesystem should never become inconsistent.  Safe, but slow,
 > particularly for things like unpacking pkgsrc, which is almost all meta-data
 > operations (allocating inodes, making new directory entries, ...)
 > 
 > Async and log are almost opposites in intent, both make the filesystem
 > do meta-data ops faster than the default, but in dramatically different
 > ways.  It is possible the system is simply ignoring both of them, and
 > operating in safe slow mode.   If not, then almost anything might be
 > happening internally.   Just don't do that.
 > 
 > For root just use log.  Not async.   If you use a separate filesystem
 > for pkgsrc, you can mount that async while unpacking.  That is then lots
 > faster.  After the unpack is done, umount and mount again without async
 > (you can use log, or not, as you like - once unpacked, pkgsrc should
 > get few future changes, just perhaps occasional updates, so it doesn't
 > make a lot of difference - put the work directories elsewhere.)
 
 Thanks Robert for the insight.
 I tested with only "log" option added to /, and it takes:
 
 netbsd# time tar xf pkgsrc.tar.xz
       1594.28 real         5.69 user      1528.69 sys
 
 so 26 minutes.
 
 PS. the man page for mount says if both log and async are provided, 
 async is simply disregarded so it shouldn't have impacted my results.
 
 Is anybody else running a NetBSD 10.0_BETA VM and can confirm my 
 numbers? Default install, just "log" added on / mount point.
 
 -- 
 Regards
 Bartek Krawczyk