We have run NetBSD 4 on a server for over a year and recently changed
to NetBSD 5.0RC3. But we have been experiencing some network issues.
Apart from the kernel and userland upgrade, nothing has chanced with
hardware, setup, routing etc.
We appear to experiencing network pauses, general sluggishness and
sometimes transfers just appear to stall. Since its main objective is
to be an anonymous FTP mirror, this is peculiar.
I know it is fairly vague, but perhaps it is well known that some part
of the IP stack changed, that I could look into.
Many large transfers look like this, in the last few packets:
sndr: NetBSD 5.0.2
rcvr: NetBSD 5.1RC3
04:31:28.211334 IP rcvr.443 > sndr.52559: S 1684671976:1684671976(0)
win 65535 <mss 1460,nop,wscale 8,sackOK,nop,nop,nop,nop,timestamp 1 0>
04:31:28.518422 IP sndr.52559 > rcvr.443: S 3182409332:3182409332(0)
ack 1684671977 win 65535 <mss 1460,nop,wscale 8,nop,nop,timestamp 1
1,sackOK,nop,nop>
04:31:28.518447 IP rcvr.443 > sndr.52559: . ack 1 win 1260
<nop,nop,timestamp 2 1>
04:31:28.804494 IP sndr.52559 > rcvr.443: . ack 1 win 1785
<nop,nop,timestamp 2 2>
lots, and lots of data, until the end:
05:11:48.972870 IP rcvr.443 > sndr.52559: . ack 522788159 win 3041
<nop,nop,timestamp 4841 4815>
05:11:48.973013 IP sndr.52559 > rcvr.443: . 522788159:522789607(1448)
ack 1115 win 1785 <nop,nop,timestamp 4815 4840>
05:11:48.973156 IP sndr.52559 > rcvr.443: FP 522789607:522790463(856)
ack 1115 win 1785 <nop,nop,timestamp 4815 4840>
05:11:48.973169 IP rcvr.443 > sndr.52559: . ack 522790464 win 3041
<nop,nop,timestamp 4841 4815>
05:11:48.974764 IP rcvr.443 > sndr.52559: P 1115:1152(37) ack
522790464 win 3052 <nop,nop,timestamp 4841 4815>
05:11:48.974802 IP rcvr.443 > sndr.52559: F 1152:1152(0) ack 522790464
win 3052 <nop,nop,timestamp 4841 4815>
05:11:49.252555 IP sndr.52559 > rcvr.443: R 3705199796:3705199796(0)
win 0
05:11:49.252558 IP sndr.52559 > rcvr.443: R 3705199796:3705199796(0)
win 0
and it pretty much sits there forever.
Both old 4.x and new 5.1RC3 server settings are default, with only
these values:
options NMBCLUSTERS=32768
kern.sbmax=8388608
net.inet.tcp.sendspace=321796
net.inet.tcp.recvspace=321796
net.inet.tcp.init_win=8
I have since also tried a few more settings on NetBSD5.1RC3 to see if
they would help, but it would appear to have made little difference.
Current sysctl output is:
net.inet.tcp.rfc1323 = 1
net.inet.tcp.sendspace = 321796
net.inet.tcp.recvspace = 321796
net.inet.tcp.mssdflt = 536
net.inet.tcp.syn_cache_limit = 10255
net.inet.tcp.syn_bucket_limit = 105
net.inet.tcp.init_win = 8
net.inet.tcp.mss_ifmtu = 0
net.inet.tcp.sack.enable = 1
net.inet.tcp.sack.maxholes = 32
net.inet.tcp.sack.globalmaxholes = 1024
net.inet.tcp.sack.globalholes = 1
net.inet.tcp.win_scale = 1
net.inet.tcp.timestamps = 1
net.inet.tcp.compat_42 = 0
net.inet.tcp.cwm = 0
net.inet.tcp.cwm_burstsize = 4
net.inet.tcp.ack_on_push = 1
net.inet.tcp.keepidle = 14400
net.inet.tcp.keepintvl = 150
net.inet.tcp.keepcnt = 8
net.inet.tcp.slowhz = 2
net.inet.tcp.log_refused = 0
net.inet.tcp.rstppslimit = 100
net.inet.tcp.delack_ticks = 20
net.inet.tcp.init_win_local = 4
net.inet.tcp.do_loopback_cksum = 0
net.inet.tcp.msl = 30
net.inet.tcp.minmss = 216
net.inet.tcp.congctl.available = reno newreno
net.inet.tcp.congctl.selected = newreno
net.inet.tcp.keepinit = 150
net.inet.tcp.recvbuf_auto = 1
net.inet.tcp.recvbuf_inc = 65536
net.inet.tcp.recvbuf_max = 16777216
net.inet.tcp.sendbuf_auto = 1
net.inet.tcp.sendbuf_inc = 65536
net.inet.tcp.sendbuf_max = 16777216
net.inet.tcp.ecn.enable = 0
net.inet.tcp.ecn.maxretries = 1
net.inet.tcp.iss_hash = 0
net.inet.tcp.abc.enable = 1
net.inet.tcp.abc.aggressive = 1