Hi!

I'll update this with what I've learned so far (which isn't much):

On Mon, 29 Mar 2004, Arto Selonen wrote:

> I have the following type of situation:
>
> 	A(ep0) <-->IPSEC--> (fxp0)GW(fxp1) <-->plain--> (fxp0)B

> After the upgrade, I've observed the following:
>
> 	- ssh from A to B works, but fairly soon stops
> 	- running 'tcpdump -i interface ip-of-B' on GW shows:
> 		20:01:02.946013 B.ssh > GW.1084: . 1:1461(1460) ack 104 win 33580
> 		20:01:09.737922 GW > B: icmp: ip reassembly time exceeded

> I've also seen the following "death":
>
> 	20:38:36.782326 B.ssh > GW.1098: . 7388:8848(1460) ack 2913 win 33580 (DF)
> 	20:38:36.782371 GW > B: icmp: GW unreachable - need to frag

The latter seems to be the dominant mode, which seems to be related to
PMTU(D?). Making repeated file transfers etc, and running tcpdump provides
this rough interpretation:

	1) a largish packet like 1448 bytes is sent from B to A with DF set
	2) the packet doesn't fit into the IPSEC encapsulated payload
	   (it needs to be fragmented to be sent using IPSEC)
	3) ICMP_UNREACH_NEEDFRAG is sent from GW to B
	4) steps 1-3 repeat for a while (GW/B adjusting?)
	5) a largish packets like 1448 bytes sent from B to A without DF
	6) traffic flows

Not beeing familiar with PMTUD, I could assume that this is at least
close to how it should work (both GW and B initially give 1460 as MSS).
Unfortunately, it takes B about ten seconds to figure out
that DF should not be set and start sending again, and this slows
things down.

Also, making several file transfers between A and B, it looks like
A can transfer to B with expected speeds (~120kBps), but B to A was
really slow (~50kBps). Comparing this to file transfer speeds between
A and GW shows: A->GW ~150kBps and GW->A ~200kBps. I'm assuming the
overhead comes from NAT, since that is about the only thing added.

So, the connections won't die altogether, even though extensive delays
have been observed. I'll try to gather more statistics on this, since
the first connection attempts really did take so long that ssh client
started timing out on them, etc.

I have no idea why everything worked before (other than knowing that
all three NetBSD-current systems are in that respect using
default settings), but before the upgrade of GW I never noticed any
delays, and after the upgrade they were obvious from the start.


Artsi
-- 
#######======------  http://www.selonen.org/arto/  --------========########
Everstinkuja 5 B 35                               Don't mind doing it.
FIN-02600 Espoo        arto@selonen.org         Don't mind not doing it.
Finland              tel +358 50 560 4826     Don't know anything about it.