Re: NetBSD 6.1 NFS server performances

[Swift Griggs <swiftgriggs%gmail.com@localhost>]

On Thu, 3 Dec 2015, Emile `iMil' Heitor wrote:
> [  3]  0.0-10.0 sec  1.06 GBytes   908 Mbits/sec
> 1048576000 bytes (1.0 GB) copied, 9.27363 s, 113 MB/s
> Gigabit link, all clear.

I've seen somewhat higher iperf results at around 998 MBit/s with Intel 
server NICs. However, I consider anything above 900 Mbit/s to be "fast 
enough" since most Realtek NICs simply can't seem to get much higher.

> Now using NFS:
> 1048576000 bytes (1.0 GB) copied, 51.8476 s, 20.2 MB/s
> I know, right? And yes results are identical with differents bs values.

The 'dd' block size only helps you so much. Once you can effectively fill 
the input buffers on the server side... you're done.

> I've tried a bazillion NFS options on the clients (TCP, UDP, {r,w}size 
> from 8192 to 64k...), tried many OSes as a client,

Well alrighty then, it sounds like you've covered most of the NFS 
performance tuning ground but it only takes one little mismatch or 
non-optimal settings to mess things up. Here's some other things to play 
with on your non-NetBSD clients:

1. I've noticed that Linux mounts up NFSv3 exports in 'sync' mode by
   default but has a lot higher performance (and less safety) in 'async'
   mode.

2. What about TCP/IP and NIC tuning? Those can make a big difference. I've
   seen the following change performance before:

   * TCP window size
   * TCP sliding window being on/off
   * TCP buffer sizes
   * Ethernet flow control
   * Ethernet MTU size (1500 / 9000) ie.. Jumbo frames
   * Path MTU discovery (usually just helps with WAN stuff)
   * A handful of other tunables

You may consider taking a packet trace from your slowest client and your 
fastest and then sitting down with wireshark with a WTF-attitude digging 
for deltas. Usually, if it's tuning/stack related, you'll see some 
patterns emerge that will guide you in your tuning efforts.

-Swift