On Fri, 7 Dec 2001, Greg A. Woods wrote:

> > Someone mentioned that lowering the MTU on a route should cause the
> > TCP MSS to get lowered correspondingly.  I thought it was working, but I
> > must have been drunk or something, because the per-route MTU doesn't seem
> > to affect the advertised MSS at all (which I should have known since I've
> > was staring at the tcp_mss_to_advertise code).
>
> Hmmm.... isn't Path-MTU Discovery implemented in NetBSD by simply
> adding a host route with a per-route MTU?  Path-MTU-discovery certainly
> seemed to work the last time I needed it....   Hmmm.... or does that
> only need to lower the MSS of outgoing packets, leaving the peer to deal
> with it on its own...

Yeah, the per-route MTU is used for determing the segment size of outgoing
packets (in tcp_segsize), but not for the advertised MSS (which specifies
the maximum size segment that can be RECEIVED).

> > So, other than the patch I recently posted, there doesn't appear to be
> > any way to convince NetBSD to lower the advertised MSS without lowering
> > the interface MTU.  I consider being able to lower the advertised MSS a
> > handy feature, especially considering all the broken networks.  Its
> > conceivable that someone might want to use NetBSD behind a reduced-mtu
> > gateway which they have no control over, and which doesn't implement TCP
> > MSS clamping.  In that scenario, the only way to cope with blackholed
> > sites is to convince NetBSD to lower the MSS it advertises.
>
> If you're behind such a reduced-MTU gateway then it might be simpler to
> just the MTU on your interface and be done with it.  Of course if you've
> got a lot of local traffic to consider then you might try to find a way
> to subnet and multi-home your machine so that you only have to reduce
> the MTU on the interface connected to the gateway.

In most situations that may be best.  But what if you don't
control the gateway machine or all of the clients?  If you lower
your interface MTU, but there are others hosts on the network with a
higher MTU then NetBSD will end up discarding packets from them.  There's
no distinction between interface MTU and MRU, is there?  What we really
want is a lower MTU but a normal MRU.

> > Anyway, I know some people belive we shouldn't provide an option to cope
> > with broken networks, but I believe the situation is already desperate
> > enough that there is nothing to be gained by holding this feature back.
> > Does anyone else think there should be a sysctl for lowering the MSS, or
> > is there some better way?  Perhaps a per-route MRU or something to that
> > effect?
>
> Is there any reason why the setting of a per-route MTU cannot be made to
> work?  If per-route MTUs don't affect the MSS does that mean the current
> Path-MTU discovery code assumes the peer is also doing the same?

The per-route MTU could easily be used in the calculation of the
advertised MSS, but I'm not sure its the right thing to do.  The per-route
MTU specifies the maximum size datagram to TRANSMIT on that route.  The
advertises MSS is the maximum size segment you're able to RECEIVE.  As far
as I know, the per-route MTU doesn't effect incoming packets at all.
Thats why I was suggesting that maybe we should have per-route MRUs.

> You could also just try setting net.inet.tcp.mssdflt to some smaller
> value.  Back when I was using a GIF tunnel for my default route I was
> experimenting with the following settings in /etc/sysctl.conf:
>
> [..snip..]
> 	net.inet.tcp.mssdflt=1024
> 	net.inet.ip.mtudisc=1
> 	net.inet.tcp.mss_ifmtu=1
>
> As you can see from my comments I wasn't 100% successful with
> controlling packet sizes, and at the time I was having trouble resolving
> differences between what I read in Stevens' books and the NetBSD code.

First of all, I think something is funny with mssdflt.  It's used for a
number of things:
- the minimum TCP MSS to advertise
- the output segment size to use in adsense of PTMUD, or per-route MTUs
- the peer MSS to assume if the peer didn't advertise it

The last one is what concerns me.  RFC 1122 (sec 4.2.2.6) mandates that if
no MSS is received, a value of 536 must be assumed.  I don't see why this
should be configurable, and I definantly don't see why we default to 512
(I have an open PR on this).

If we hard-code the 3rd use and make mss_dflt control the first two, then
thats fine, but I don't think it can be used to limit packet sizes except
when PMTUD is disabled and there is no per-route MTU.

> Maybe all we need is a 'net.inet.tcp.mss_rtmtu' flag.... ;-)

Yeah, thats probably a good idea.  It should be really easy to implement
too.  If people agree that providing an option (default OFF of course) to
use the per-route MTU for determining the advertises MSS is a good idea,
then I'll whip up a patch and send-pr it.

Thanks,
	Rick