Space Case writes:
> On Dec 17,  7:06pm, "Kevin M. Lahey" wrote:
> >In message <199812180206.SAA26700@eskimo.com>Space Case writes
> >>way (Win98 -> NetBSD) and got a dismal 4.8KB/second.  Any ideas on what
> >>might cause this or how to track it down?
> >Yow!  That's pretty awful.  Could you use tcpdump to see what is actually
> >going over the wire?  Are both of these on the same LAN?  Wierd.
> 
> Here's an excerpt from the tcpdump (the whole dump is available at
> ftp://ftp.eskimo.com/u/w/wormey/dump2.out if anybody's interested in it).
> Note that the Win box (jared) just quits sending for periods ranging from
> fractions of a second to, in one case, 38 seconds!

Can you send me a partial trace, where you run:

  tcpdump -s 100 -x host jared and host gate2

What I'm interested in is what your TCP ACK packets look like -- in 
particular, the IP packet lengths, the TCP packet lengths, the TCP checksums, 
and the TCP options.  I'm seeing similar sort of behaviour, and it's driving 
me up the wall.  A number of packets get transferred, and then it waits, 
then it sends a few more, then waits, and so on.   Inspection of the TCP 
ACK packets indicates that something is wrong -- in particular, and as in 
your case, ACKs are being ignored.  I want to figure out why...

Here's what I'm seeing on a (passive) FTP transfer between two machines
which I'm calling, for simplicity, "home" and "away".  "home" is an IPNAT 
box running NetBSD-1.3I as of Dec. 5, on an i386 box, with 2 ethernet 
cards.  On the "inside" network is a 3rd machine, which is the one 
actually doing the ftp transfer.   (The inside machine is also running 1.3I)
The 2nd ethernet connection goes to a cablemodem.  "away" is a Linux box that 
I have had no problems connecting to from other places with other NetBSD 
boxes (mostly 1.3.x vintage).  "away" lives 15 network hops from "home".

Here are the contents of a "typical" TCP ACK packet on an FTP transfer
(the file is being transfered from "away" to "home").  This packet
as seen by "home" looks like:

21:57:42.778794 home.46132 > away.5335: . ack 17521 win 17520 [tos 0x8]
                         4508 0028 a0e3 0000 3f06 1a9f 1840 615b
                         c6a9 8001 b434 14d7 baf9 83a9 0f3a bbbd
                         5010 4470 d877 0000

At the destination (viewed by running tcpdump on "away") this same packet 
looks like:

21:34:38.992283 home.46132 > away.5335: . 1:5(4) ack 17521 win 17520 [tos 0x8]
                         4508 002c a0e3 0000 3006 299b 1840 615b
                         c6a9 8001 b434 14d7 baf9 83a9 0f3a bbbd
                         5010 4470 d877 0000 0101 080a 0a0a 08fc

Note here that if there really is a TCP timestamp option being added, 
as the 0101 080a 0a0a 08fc would suggest, then 002c should be 0030, 
and 5010 should be 6010!!  So whatever's re-writing the packet doesn't 
look like it's doing it's job properly. :-(   Never mind the fact that
the TCP checksum (d877 in this case) would also be wrong due to the 
addition of the TCP timestamp option stuff.  And if the checksum is 
wrong, chances are that "away" just ignores the packet  (like it's 
supposed to), which explains the pause and the retransmit. :-( 

After the pause, and then the TCP retrsnamit of the data, the following
ACK is observed on "home":
21:57:54.285017 home.46132 > away.5335: . ack 17521 win 17520 [tos 0x8]
                         4508 0028 a0fc 0000 3f06 1a86 1840 615b
                         c6a9 8001 b434 14d7 baf9 83a9 0f3a bbbd
                         5010 4470 d877 0000

and at the other end:
21:34:50.692283 home.46132 > away.5335: . ack 17521 win 17520 [tos 0x8]
                         4508 0028 a0fc 0000 3006 2986 1840 615b
                         c6a9 8001 b434 14d7 baf9 83a9 0f3a bbbd
                         5010 4470 d877 0000 

For whatever reason this packet makes it through unscathed, the other end is 
happy to (finally) receive the ACK and so it ships more data.  The ACK packets
for that data however, also get messed up in the same fashion as described
earlier.  Repeat until stalled or frustrated. :-( 

My next step is to add a box between the IPNAT box and the cablemodem, 
and really see what packets are being shipped to the cablemodem...

Any suggestions are most welcome...  If anyone needs more info I'd be happy to 
provide it...

Later...

Greg Oster

oster@cs.usask.ca
Department of Computer Science
University of Saskatchewan, Saskatoon, Saskatchewan, CANADA