Hi!

I've borrowed an Ultra-5 from a friend and set it up as a router/firewall. As 
such it works great. The only problem is the network performance.

I'm on a 100mbit (up/down full duplex) connection but downloading from my 
friends webserver (he lives in the neighbourhood with the same 
connection/subnet as me) directly to the router peaks at ~4.4MB/s. 
Downloading from the university here in Lund peaks at the same rate. If I 
download to any of the machines behind the router the peaks are a bit lower.

My friend peaks at ~9.5MB/s when downloading from the university with an 
Ultra-10 as router.

I've noticed that whenever I dowload at those speeds the load on the router 
isn't at 100%. When downloading to the router top reports ~45-50% system time 
and the rest interrupt time.

When I download to a machine behind the router ~40% is spent in interrupt and 
the rest is reported as idle.

What could be wrong here? It seems to me that it is because all that time is 
spent waiting for something, but for what? and how do I fix it?

I haven't used any *BSD before, just Linux, so I don't really know what to 
attach.. But here comes dmesg, ipf.conf and ipnat.conf

=== DMESG ===
NetBSD 3.0 (GENERIC) #0: Mon Dec 19 04:05:17 UTC 2005
        
builds@b4.netbsd.org:/home/builds/ab/netbsd-3-0-RELEASE/sparc64/200512182024Z-obj/home/builds/ab/netbsd-3-0-RELEASE/src/sys/arch/sparc64/compile/GENERIC
total memory = 384 MB
avail memory = 364 MB
bootpath: /pci@1f,0/pci@1,1/ide@3,0/disk@0,0
mainbus0 (root): SUNW,Ultra-5_10: hostid 80a22042
cpu0 at mainbus0: SUNW,UltraSPARC-IIi @ 270 MHz, version 0 FPU
cpu0: 32K instruction (32 b/l), 16K data (32 b/l), 256K external (64 b/l)
psycho0 at mainbus0 addr 0xfffc4000
SUNW,sabre: impl 0, version 0: ign 7c0 bus range 0 to 3; PCI bus 0
DVMA map: c0000000 to e0000000
IOTSB: 646000 to 6c6000
pci0 at psycho0
pci0: i/o space, memory space enabled
ppb0 at pci0 dev 1 function 1: Sun Microsystems Simba PCI bridge (rev. 0x13)
pci1 at ppb0 bus 1
pci1: i/o space, memory space enabled
ebus0 at pci1 dev 1 function 0
ebus0: Sun Microsystems PCIO Ebus2, revision 0x01
auxio0 at ebus0 addr 726000-726003, 728000-728003, 72a000-72a003, 
72c000-72c003, 72f000-72f003
power at ebus0 addr 724000-724003 ipl 37 not configured
SUNW,pll at ebus0 addr 504000-504002 not configured
sab0 at ebus0 addr 400000-40007f ipl 43: rev 3.2
sabtty0 at sab0 port 0: console i/o
sabtty1 at sab0 port 1
com0 at ebus0 addr 3083f8-3083ff ipl 41: ns16550a, working fifo
kbd0 at com0
com1 at ebus0 addr 3062f8-3062ff ipl 42: ns16550a, working fifo
ms0 at com1
wsmouse0 at ms0 mux 0
lpt0 at ebus0 addr 3043bc-3043cb, 30015c-30015d, 700000-70000f ipl 34
fdthree at ebus0 addr 3023f0-3023f7, 706000-70600f, 720000-720003 ipl 39 not 
configured
clock0 at ebus0 addr 0-1fff: mk48t59
flashprom at ebus0 addr 0-fffff not configured
audiocs0 at ebus0 addr 200000-2000ff, 702000-70200f, 704000-70400f, 
722000-722003 ipl 35 ipl 36: CS4231A
audio0 at audiocs0: full duplex
hme0 at pci1 dev 1 function 1: Sun Happy Meal Ethernet, rev. 1
hme0: interrupting at ivec 3021
hme0: Ethernet address 08:00:20:a2:20:42
nsphy0 at hme0 phy 1: DP83840 10/100 media interface, rev. 1
nsphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
machfb0 at pci1 dev 2 function 0: ATI Technologies 3D Rage Pro (rev. 0x5c)
machfb0: 16 MB aperture at 0xe1000000, 4 KB registers at 0xe2000000
memctl: 00651a77
machfb0: 4096 KB SGRAM 98.924 MHz, maximum RAMDAC clock 230 MHz
machfb0: initial resolution 1152x864 at 8 bpp
initializing the DSP
wsdisplay1 at machfb0 kbdmux 1
wsmux1: connecting to wsdisplay1
wsdisplay1: screen 0-3 added (default, sun emulation)
cmdide0 at pci1 dev 3 function 0
cmdide0: CMD Technology PCI0646 (rev. 0x03)
cmdide0: bus-master DMA support present
cmdide0: primary channel configured to native-PCI mode
cmdide0: using ivec 1820 for native-PCI interrupt
atabus0 at cmdide0 channel 0
cmdide0: secondary channel configured to native-PCI mode
atabus1 at cmdide0 channel 1
ppb1 at pci0 dev 1 function 0: Sun Microsystems Simba PCI bridge (rev. 0x13)
pci2 at ppb1 bus 2
pci2: i/o space, memory space enabled
ppb2 at pci2 dev 3 function 0: Intel S21152BA,S21154AE/BE PCI-PCI Bridge (rev. 
0x00)
pci3 at ppb2 bus 3
pci3: i/o space, memory space enabled
Sun Microsystems PCIO Ebus2 (miscellaneous bridge, revision 0x01) at pci3 dev 
0 function 0 not configured
hme1 at pci3 dev 0 function 1: Sun Happy Meal Ethernet, rev. 1
hme1: interrupting at ivec 3019
hme1: Ethernet address 08:00:20:ea:ce:f0
ukphy0 at hme1 phy 1: Generic IEEE 802.3u media interface
ukphy0: OUI 0x0006b8, model 0x000c, rev. 1
ukphy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Sun Microsystems PCIO Ebus2 (miscellaneous bridge, revision 0x01) at pci3 dev 
1 function 0 not configured
hme2 at pci3 dev 1 function 1: Sun Happy Meal Ethernet, rev. 1
hme2: interrupting at ivec 301a
hme2: Ethernet address 08:00:20:ea:ce:f1
ukphy1 at hme2 phy 1: Generic IEEE 802.3u media interface
ukphy1: OUI 0x0006b8, model 0x000c, rev. 1
ukphy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Sun Microsystems PCIO Ebus2 (miscellaneous bridge, revision 0x01) at pci3 dev 
2 function 0 not configured
hme3 at pci3 dev 2 function 1: Sun Happy Meal Ethernet, rev. 1
hme3: interrupting at ivec 301b
hme3: Ethernet address 08:00:20:ea:ce:f2
ukphy2 at hme3 phy 1: Generic IEEE 802.3u media interface
ukphy2: OUI 0x0006b8, model 0x000c, rev. 1
ukphy2: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
Sun Microsystems PCIO Ebus2 (miscellaneous bridge, revision 0x01) at pci3 dev 
3 function 0 not configured
hme4 at pci3 dev 3 function 1: Sun Happy Meal Ethernet, rev. 1
hme4: interrupting at ivec 3018
hme4: Ethernet address 08:00:20:ea:ce:f3
ukphy3 at hme4 phy 1: Generic IEEE 802.3u media interface
ukphy3: OUI 0x0006b8, model 0x000c, rev. 1
ukphy3: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, auto
pcons at mainbus0 not configured
No counter-timer -- using %tick at 270MHz as system clock.
wskbd0 at kbd0 mux 1
kbd0: reset failed
wskbd0: connecting to wsdisplay1
Kernelized RAIDframe activated
wd0 at atabus0 drive 0: <ST315320A>
wd0: drive supports 16-sector PIO transfers, LBA addressing
wd0: 14594 MB, 29651 cyl, 16 head, 63 sec, 512 bytes/sect x 29888820 sectors
wd0: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 4 (Ultra/66)
wd0(cmdide0:0:0): using PIO mode 4, DMA mode 2 (using DMA)
atapibus0 at atabus1: 2 targets
cd0 at atapibus0 drive 0: <CREATIVE CD5233E, MT1199 A Firmware, C2.02> cdrom 
removable
cd0: drive supports PIO mode 4, DMA mode 2
wd1 at atabus1 drive 1: <ST36531A>
wd1: drive supports 32-sector PIO transfers, LBA addressing
wd1: 6204 MB, 13446 cyl, 15 head, 63 sec, 512 bytes/sect x 12706470 sectors
wd1: drive supports PIO mode 4, DMA mode 2, Ultra-DMA mode 2 (Ultra/33)
cd0(cmdide0:1:0): using PIO mode 4, DMA mode 2 (using DMA)
wd1(cmdide0:1:1): using PIO mode 4, DMA mode 2 (using DMA)
root on wd0a dumps on wd0b
root file system type: ffs
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:01
arp info overwritten for 83.233.156.1 by 00:07:b4:00:fd:02
kdb breakpoint at 1385420
arp info etc for anoter eternity



=== /etc/ipf.conf ===
# --- In Rules ---
# FTP
pass in quick on hme1 proto tcp from any to 192.168.0.100 port 19 >< 22 flags 
S keep state
# SSH
pass in quick on hme1 proto tcp from any to 192.168.0.200 port = 22 flags S 
keep state
pass in quick on hme1 proto tcp from any to 192.168.0.100 port = 22 flags S 
keep state
# Web server
pass in quick on hme1 proto udp from any to 192.168.0.100 port = 80 keep state
pass in quick on hme1 proto tcp from any to 192.168.0.100 port = 80 flags S 
keep state
pass in quick on hme1 proto tcp from any to 192.168.0.100 port = 443 flags S 
keep state
pass in quick on hme1 proto udp from any to 192.168.0.100 port = 443 keep 
state
# Jabber
pass in quick on hme1 proto tcp from any to 192.168.0.100 port 5221 >< 5224 
flags S keep state
pass in quick on hme1 proto tcp from any to 192.168.0.100 port = 5269 flags S 
keep state
# BitTorrent
pass in quick on hme1 proto tcp from any to 192.168.0.100 port 6880 >< 7000 
flags S keep state
# Icecast
pass in quick on hme1 proto tcp from any to 192.168.0.200 port = 8000 flags S 
keep state
# Passive FTP ports
pass in quick on hme1 proto tcp from any to 192.168.0.100 port 11999 >< 12101 
flags S keep state
# Block everything else
block in log quick on hme1 all
# --- Out Rules ---
pass out quick on hme1 proto tcp from hme1 to any flags S keep state
pass out quick on hme1 proto udp from hme1 to any keep state
pass out quick on hme1 proto icmp from hme1 to any keep state
pass out quick on hme1 proto tcp from 192.168.0.0/24 to any flags S keep state
pass out quick on hme1 proto udp from 192.168.0.0/24 to any keep state
pass out quick on hme1 proto icmp from 192.168.0.0/24 to any keep state


=== /etc/ipnat.conf ===
# NAT
map hme1 192.168.0.0/24 -> 0/32
map hme1 192.168.0.0/24 -> 0/32 proxy port ftp ftp/tcp
map hme1 192.168.0.0/24 -> 0/32 portmap tcp/udp 40000:60000
# Port redirects
rdr hme1 0/0 port 20 - 21       -> 192.168.0.100 port 20    tcp/udp
rdr hme1 0/0 port 80            -> 192.168.0.100 port 80    tcp/udp
rdr hme1 0/0 port 443           -> 192.168.0.100 port 443   tcp/udp
rdr hme1 0/0 port 5222 - 5223   -> 192.168.0.100 port 5222  tcp
rdr hme1 0/0 port 5269          -> 192.168.0.100 port 5269  tcp
rdr hme1 0/0 port 6881 - 6999   -> 192.168.0.100 port 6881  tcp/udp
rdr hme1 0/0 port 8000          -> 192.168.0.200 port 8000  tcp
rdr hme1 0/0 port ###         -> 192.168.0.200 port 22    tcp/udp
rdr hme1 0/0 port ###         -> 192.168.0.100 port 22    tcp/udp
rdr hme1 0/0 port 12000 - 12100 -> 192.168.0.100 port 12000 tcp/udp

Regards Raniz