Re: [SOLVED]Re: Only 8 MB/sec write throughput with NetBSD 5.1 AMD64

[David Laight <david%l8s.co.uk@localhost>]

On Fri, Oct 14, 2011 at 03:08:51PM -0700, Tony Bourke wrote:
> > 
> > I really can't agree with you about path MTU discovery, either.  With
> > proper blackhole detection (and if you don't have that, then you
> > should not be using path MTU discovery at all) it's plenty reliable;
> > and in any event, using a large local MTU won't cause a sudden magic
> > change of default to use the link layer MTU as the inital MTU for
> > remote peers anyway; only local ones.
> 
> If you're talking about MSS clamping, I agree. But at the same time,
> is the original poster upping his MTU going to help? Unlikely,
> and it would likely complicate his network needlessly. 

PMTU discovery relies on ICMP errors (needs fragment) which rely
on the router having actually received the packet.
If you send an overlong packet it will be discarded.
So 'Jumbo' frames can only be used if the devices they might be sent
to are expecting them.

I can't quite remember the exact length of the headers for TCP, but
it is about 50 bytes. So using very long frames gains you 50/1500
or about 3% if the network is 100% loaded. In software terms, the
receiving side can (and in some cases has) been optimised for
receipt of in sequence packets for the same connection.
(Optimising the sender is easier...)
So I suspect the actual speed improvement for jumbo frames can be
arranged to be minimal.
The costs of them are significant, especially if you start considering
the bufferring requirements in ethernet switches.

OTOH I've NFI why they chose 1536 for the ethernet frame limit.
Unless the idea was 1k of 'userdata' plus some headers??
When doing token-ring (& FDDI) the packet limit is based on the token
rotation time - and is defined as a time interval, not a byte count.
IIRC this effectively allows 4k userdata at 4MHz and 16k userdata at 16MHz
although all the protocol stack code is designed to limit the physical
mtu rather then the userdata.

9k packets would be rather magical if you could receive the last 8k
of the packet into a separate, page aligned, buffer! That would allow
page loading on the rx side!

        David

-- 
David Laight: david%l8s.co.uk@localhost