On Thu, Nov 30, 2006 at 07:41:45PM -0500, Mike Tancsa wrote:
> At 06:49 PM 11/30/2006, Thor Lancelot Simon wrote:
> 
> >   1) The efficiency of the switch itself will differ in these 
> >   configurations
> 
> Why ?  The only thing being changed from test to test is the OS.

Because the switch hardware does not forward packets at the same rate
when it is inserting and removing VLAN tags as it does when it's not.
The effect will be small, but measurable.

> >   2) The difference in frame size will actually measurably impact the PPS.
> 
> Framesize is always the same.  UDP packet with a 10byte payload.

No.  The Ethernet packets with the VLAN tag on them are not, in fact,
the same size as those without it; and for a packet as small as a 10
byte UDP packet, this will make quite a large difference if you actually
have a host that can inject packets at anywhere near wire speed.

> generators are the same devices all the time. I am not using 
> different frame sizes for different setups to try and make something 
> look good and other things bad.

I didn't say that you were, just to be clear.  But that does not mean
that running some tests with tagging turned on, and others not, is
good benchmarking practice: you should run the exact same set of tests
for all host configurations, because doing otherwise yields distorted
results.

> >3) There is a problem with autonegotiation either on your switch, on the
> >   particular wm adapter you're using, or in NetBSD -- there's not quite
> >   enough data to tell which.  But look at the number of input errors on
> >   the wm adapter in your test with NetBSD-current: it's 3 million.  This
> >   alone is probably responsible for most of the performance difference
> 
> .... Or the kernel just was not able forward fast enough.

No; that will simply not cause the device driver to report an input
error, whereas your netstat output shows that it reported three *million*
of them.  Something is wrong at the link layer.  It could be in the NetBSD
driver for the Intel gigabit PHY, but there's not enough data in your
report to be sure.  FWIW, I work for a server load balancer vendor that
ships a FreeBSD-based product, and I consequently do a lot of load testing.
Even with tiny UDP packets, I get better forwarding performance from
basically _every_ OS you tested than you seem to, which is why I think
there's something that's not quite right with your test rig.  I am just
doing my best to point out the first things that come to mind when I look
at the data you've put online.

I note that you snipped the text where I noted that because you're
testing the wm card with mismatched kernel and ifconfig, you're not
using its hardware checksum offload.  That's one thing you should
definitely fix, and if you don't have that turned on for other
kernels you're testing, of course you should probably fix it there too.

-- 
Thor Lancelot Simon	                                     tls@rek.tjls.com
  "The liberties...lose much of their value whenever those who have greater
   private means are permitted to use their advantages to control the course
   of public debate."					-John Rawls