NetBSD Network Tuning

[Michael D Spence <spence%panix.com@localhost>]

I use NetBSD 5 as a firewall for my home network, connected to Time 
Warner Cable at a
nominal 100 MB.  With a little help (ok, a lot of help) from this group, 
I rearranged my
NICs and got an observed speed of 75 - 90 MB on my Windows system 
through the firewall.

Here are my NICs (ex0 faces the cable modem):

xp0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        address: 00:e0:18:4a:51:e4
        media: Ethernet 100baseTX full-duplex
        status: active
        inet 192.168.1.6 netmask 0xffffff00 broadcast 192.168.1.255
        inet6 fe80::2e0:18ff:fe4a:51e4%fxp0 prefixlen 64 scopeid 0x1
ex0: 
flags=8a63<UP,BROADCAST,NOTRAILERS,RUNNING,ALLMULTI,SIMPLEX,MULTICAST> 
mtu 1500
        address: 00:60:08:9a:53:6c
        media: Ethernet 100baseTX full-duplex
        status: active
        inet 192.168.0.6 netmask 0xffffff00 broadcast 192.168.0.255
        inet6 fe80::260:8ff:fe9a:536c%ex0 prefixlen 64 scopeid 0x2


and here is dmesg:

opyright (c) 1982, 1986, 1989, 1991, 1993
    The Regents of the University of California.  All rights reserved.

NetBSD 5.0 (GENBIGBUF) #0: Tue Oct 21 20:45:06 EDT 2014
root%nfw.mdsi@localhost.invalid:/usr/obj/sys/arch/i386/compile/GENBIGBUF
total memory = 255 MB
avail memory = 239 MB
timecounter: Timecounters tick every 10.000 msec
timecounter: Timecounter "i8254" frequency 1193182 Hz quality 100
HP Pavilion 04 P5269A-ABA 7955 (70000TG101AMAZO)
mainbus0 (root)
cpu0 at mainbus0 apid 0: Intel 686-class, 1494MHz, id 0xf12
ioapic0 at mainbus0 apid 2: pa 0xfec00000, version 20, 24 pins
acpi0 at mainbus0: Intel ACPICA 20080321
acpi0: X/RSDT: OemId <ASUS  ,P4B_LA  ,42302e31>, AslId <MSFT,31313031>
acpi0: SCI interrupting at int 9
acpi0: fixed-feature power button present
timecounter: Timecounter "ACPI-Fast" frequency 3579545 Hz quality 1000
ACPI-Fast 24-bit timer
acpibut0 at acpi0 (PWRB, PNP0C0C): ACPI Power Button
attimer1 at acpi0 (TMR, PNP0100): AT Timer
attimer1: io 0x40-0x43 irq 0
pcppi1 at acpi0 (SPKR, PNP0800)
pcppi1: io 0x61
midi0 at pcppi1: PC speaker (CPU-intensive output)
sysbeep0 at pcppi1
npx1 at acpi0 (COPR, PNP0C04)
npx1: io 0xf0-0xff irq 13
npx1: reported by CPUID; using exception 16
FDC0 (PNP0700) at acpi0 not configured
ECP (PNP0401) at acpi0 not configured
UAR1 (PNP0501) at acpi0 not configured
UAR2 (PNP0501) at acpi0 not configured
pckbc1 at acpi0 (PS2K, PNP0303): kbd port
pckbc1: io 0x60,0x64 irq 1
mpu0 at acpi0 (MIDI, PNPB006)
mpu0: io 0x330-0x333 irq 10
midi1 at mpu0: Roland MPU-401 MIDI UART (CPU-intensive output)
joy0 at acpi0 (GAME, PNPB02F)
joy0: io 0x200-0x20f
joy0: joystick not connected
apm0 at acpi0: Power Management spec V1.2
attimer1: attached to pcppi1
pci0 at mainbus0 bus 0: configuration mode 1
pci0: i/o space, memory space enabled, rd/line, rd/mult, wr/inv ok
pchb0 at pci0 dev 0 function 0
pchb0: vendor 0x8086 product 0x1a30 (rev. 0x03)
agp0 at pchb0: aperture at 0xf8000000, size 0x4000000
ppb0 at pci0 dev 1 function 0: vendor 0x8086 product 0x1a31 (rev. 0x03)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
vga1 at pci1 dev 0 function 0: vendor 0x10de product 0x002d (rev. 0x15)
wsdisplay0 at vga1 kbdmux 1: console (80x25, vt100 emulation)
wsmux1: connecting to wsdisplay0
drm at vga1 not configured
ppb1 at pci0 dev 30 function 0: vendor 0x8086 product 0x244e (rev. 0x12)
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled
fxp0 at pci2 dev 8 function 0: Intel i82562 Ethernet, rev 3
fxp0: interrupting at ioapic0 pin 20
fxp0: Ethernet address 00:e0:18:4a:51:e4
inphy0 at fxp0 phy 1: i82562ET 10/100 media interface, rev. 0
inphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
ex0 at pci2 dev 9 function 0: 3Com 3c905-TX 10/100 Ethernet (rev. 0x0)
ex0: interrupting at ioapic0 pin 21
ex0: MAC address 00:60:08:9a:53:6c
nsphy0 at ex0 phy 24: DP83840 10/100 media interface, rev. 1
nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vr0 at pci2 dev 10 function 0: VIA VT6105 (Rhine III) 10/100 Ethernet
vr0: interrupting at ioapic0 pin 22
vr0: Ethernet address: 00:24:01:08:93:22
ukphy0 at vr0 phy 1: Generic IEEE 802.3u media interface
ukphy0: OUI 0x0002c6, model 0x0034, rev. 4
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
vendor 0x11c1 product 0x044e (miscellaneous communications, revision 
0x02) at pci2 dev 11 function 0 not configured
ichlpcib0 at pci0 dev 31 function 0
ichlpcib0: vendor 0x8086 product 0x2440 (rev. 0x12)
timecounter: Timecounter "ichlpcib0" frequency 3579545 Hz quality 1000
ichlpcib0: 24-bit timer
ichlpcib0: TCO (watchdog) timer configured.
piixide0 at pci0 dev 31 function 1
piixide0: Intel 82801BA IDE Controller (ICH2) (rev. 0x12)
piixide0: bus-master DMA support present
piixide0: primary channel wired to compatibility mode
piixide0: primary channel interrupting at ioapic0 pin 14
atabus0 at piixide0 channel 0
piixide0: secondary channel wired to compatibility mode
piixide0: secondary channel interrupting at ioapic0 pin 15
atabus1 at piixide0 channel 1
uhci0 at pci0 dev 31 function 2: vendor 0x8086 product 0x2442 (rev. 0x12)
uhci0: interrupting at ioapic0 pin 19
usb0 at uhci0: USB revision 1.0
uhci1 at pci0 dev 31 function 4: vendor 0x8086 product 0x2444 (rev. 0x12)
uhci1: interrupting at ioapic0 pin 23
usb1 at uhci1: USB revision 1.0
auich0 at pci0 dev 31 function 5: i82801BA (ICH2) AC-97 Audio
auich0: interrupting at ioapic0 pin 17
auich0: ac97: Analog Devices AD1885 codec; headphone, Analog Devices 
Phat Stereo
auich0: ac97: ext id 1<VRA>
isa0 at ichlpcib0
lpt0 at isa0 port 0x378-0x37b irq 7
com0 at isa0 port 0x3f8-0x3ff irq 4: ns16550a, working fifo
com1 at isa0 port 0x2f8-0x2ff irq 3: ns16550a, working fifo
fdc0 at isa0 port 0x3f0-0x3f7 irq 6 drq 2
timecounter: Timecounter "clockinterrupt" frequency 100 Hz quality 0
auich0: measured ac97 link rate at 47249 Hz, will use 48000 Hz
audio0 at auich0: full duplex, mmap, independent
fd0 at fdc0 drive 0: 1.44MB, 80 cyl, 2 head, 18 sec
uhub0 at usb0: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub0: 2 ports with 2 removable, self powered
uhub1 at usb1: vendor 0x8086 UHCI root hub, class 9/0, rev 1.00/1.00, addr 1
uhub1: 2 ports with 2 removable, self powered
wd0 at atabus0 drive 0: <WDC WD400EB-00CPF0>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 38166 MB, 77545 cyl, 16 head, 63 sec, 512 bytes/sect x 78165360 sectors
wd0: 32-bit data port
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 5 (Ultra/100)
wd0(piixide0:0:0): using PIO mode 4, Ultra-DMA mode 5 (Ultra/100) (using 
DMA)
atapibus0 at atabus1: 2 targets
cd0 at atapibus0 drive 1: <TOSHIBA DVD-ROM SD-M1502, Y100300348, 1816> 
cdrom removable
cd0: 32-bit data port
cd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
cd0(piixide0:1:1): using PIO mode 4, Ultra-DMA mode 2 (Ultra/33) (using DMA)
Kernelized RAIDframe activated
pad0: outputs: 44100Hz, 16-bit, stereo
audio1 at pad0: half duplex
boot device: wd0
root on wd0a dumps on wd0b
root file system type: ffs
wsdisplay0: screen 1 added (80x25, vt100 emulation)
wsdisplay0: screen 2 added (80x25, vt100 emulation)
wsdisplay0: screen 3 added (80x25, vt100 emulation)
wsdisplay0: screen 4 added (80x25, vt100 emulation)

Time Warner has unilaterally increased my speed from 100 MB to 300 MB 
(was there an ice storm in the nether
regions?).  This actually caused my observed speed to *drop* to around 
45 MB.  When I plug the Windows
system into the modem directly, I get 301 MB using the same tests.

On the theory that there was some tuning to do, I made the following 
setting changes:

kern.mbuf.nmbclusters = 16384
net.inet.tcp.sendspace = 26252500
net.inet.tcp.recvspace = 26252500
kern.somaxkva = 33554432
kern.sbmax = 524288
net.inet.tcp.recvbuf_auto = 1
net.inet.tcp.sendbuf_auto = 1
net.inet.tcp.sendbuf_max = 16777216
net.inet.tcp.recvbuf_max = 16777216

This gets me up to 75MB.

Here are my questions:

Are there more settings to profitably tweak?
Are there better values for the settings I have changed than those shown 
here?
Will the NICs I have support these speeds in the first place?
Will I have to get a real computer to do this (since this one is, IIRC, 
from the last century)?
Are there more appropriate NICs to use for this?

Thanks,

Michael D. Spence
Mockingbird Data Systems, Inc.