Subject: Re: Network trouble with bridges
To: Manuel Bouyer <bouyer@antioche.eu.org>
From: Andreas Neth <ane+netbsd@n3th.de>
List: port-xen
Date: 05/14/2006 23:08:16
On 14.05.2006, at 22:33, Manuel Bouyer wrote:

> On Sun, May 14, 2006 at 10:28:16PM +0200, Andreas Neth wrote:
>>
>>>> Traffic passing from other physical machines passes just fine to  
>>>> the
>>>> domU's, but anything between 2 domUs has a delay of about 20  
>>>> seconds.
>>>> ICMP works, but does not even show the latency, instead  
>>>> reportint an
>>>> RTT of 0,4ms.
>>>> The delay even happens when pinging the domU by itself
>>>
>>> You could run tcdump on dom0, to get more details on what's going on
>>
>> I already did that. Apart from the delay (which does not happen
>> always from every machine), I can't see anything unusual.
>
> What would be interesting to know if what is delayed, and if it's
> delayed the same way on the different xvif interfaces (you may need to
> run 2 tcpdump for that)

hope the following dumps illustrate it better.

ok, isis ist the nfsd-domain, and seshat is the nfs-client:

23:32:53.590403 IP isis.1020 > seshat868: UDP, length: 68
23:32:53.590435 IP seshat > isis: icmp 36: seshat udp port 868  
unreachable

...is something new I logged with tcpdump from seshat on xennet0

23:33:15.630261 IP isis.1020 > seshat.865: UDP, length: 68
23:33:51.660212 IP seshat.864 > isis.sunrpc: UDP, length: 56
23:33:51.662714 IP isis.sunrpc > seshat.864: UDP, length: 28
23:33:51.662928 IP seshat.863 > isis.sunrpc: UDP, length: 56
23:33:51.663464 IP isis.sunrpc > seshat.863: UDP, length: 28
23:33:51.663706 IP seshat.862 > isis.1020: UDP, length: 116


dom0 logs on seshats' xvif7.0: (ethereal exported)

No.     Time        Source                Destination            
Protocol Info
      15 16.572887   isis               seshat             RPC       
Continuation
      29 28.369415   seshat             isis               Portmap   
V2 GETPORT Call (Reply In 30) NFS(100003) V:3 UDP
      30 28.371812   isis               seshat             Portmap   
V2 GETPORT Reply (Call In 29) Port:2049
      31 28.372118   seshat             isis               Portmap   
V2 GETPORT Call (Reply In 32) MOUNT(100005) V:3 UDP
      32 28.372709   isis               seshat             Portmap   
V2 GETPORT Reply (Call In 31) Port:1020
      33 28.373060   seshat             isis               MOUNT     
V3 MNT Call (Reply In 62)[Short Frame]
      46 38.465266   seshat             isis               Portmap   
V2 GETPORT Call (Reply In 47) NFS(100003) V:3 UDP
      47 38.467642   isis               seshat             Portmap   
V2 GETPORT Reply (Call In 46) Port:2049
      48 38.467948   seshat             isis               Portmap   
V2 GETPORT Call (Reply In 49) MOUNT(100005) V:3 UDP
      49 38.470260   isis               seshat             Portmap   
V2 GETPORT Reply (Call In 48) Port:1020
      50 38.470609   seshat             isis               MOUNT     
V3 MNT Call (Reply In 104)[Short Frame]
      62 50.413954   isis               seshat             MOUNT     
V3 MNT Reply (Call In 33)[Short Frame]
     104 72.452883   isis               seshat             MOUNT     
V3 MNT Reply (Call In 50)[Short Frame]
     148 108.482685  seshat             isis               Portmap   
V2 GETPORT Call (Reply In 149) NFS(100003) V:3 UDP
     149 108.485102  isis               seshat             Portmap   
V2 GETPORT Reply (Call In 148) Port:2049
     150 108.485385  seshat             isis               Portmap   
V2 GETPORT Call (Reply In 151) MOUNT(100005) V:3 UDP
     151 108.485846  isis               seshat             Portmap   
V2 GETPORT Reply (Call In 150) Port:1020
     152 108.486157  seshat             isis               MOUNT     
V3 MNT Call (Reply In 183)[Short Frame]


dom0 logs on isis' xvif5.0: (ethereal exported)

note the delayed mount calls (19, 43

No.     Time        Source                Destination            
Protocol Info
       1 0.000000    isis              seshat             RPC       
Continuation
      15 11.796553   seshat            isis               Portmap  V2  
GETPORT Call (Reply In 16) NFS(100003) V:3 UDP
      16 11.798925   isis              seshat             Portmap  V2  
GETPORT Reply (Call In 15) Port:2049
      17 11.799253   seshat            isis               Portmap  V2  
GETPORT Call (Reply In 18) MOUNT(100005) V:3 UDP
      18 11.799825   isis              seshat             Portmap  V2  
GETPORT Reply (Call In 17) Port:1020
      19 11.800195   seshat            isis               MOUNT    V3  
MNT Call (Reply In 62)[Short Frame]
      39 21.892404   seshat            isis               Portmap  V2  
GETPORT Call (Reply In 40) NFS(100003) V:3 UDP
      40 21.894755   isis              seshat             Portmap  V2  
GETPORT Reply (Call In 39) Port:2049
      41 21.895083   seshat            isis               Portmap  V2  
GETPORT Call (Reply In 42) MOUNT(100005) V:3 UDP
      42 21.897373   isis              seshat             Portmap  V2  
GETPORT Reply (Call In 41) Port:1020
      43 21.897744   seshat            isis               MOUNT    V3  
MNT Call (Reply In 100)[Short Frame]
      62 33.841068   isis              seshat             MOUNT    V3  
MNT Reply (Call In 19)[Short Frame]
     100 55.879996   isis              seshat             MOUNT    V3  
MNT Reply (Call In 43)[Short Frame]
     144 91.909822   seshat            isis               Portmap  V2  
GETPORT Call (Reply In 145) NFS(100003) V:3 UDP
     145 91.912216   isis              seshat             Portmap  V2  
GETPORT Reply (Call In 144) Port:2049
     146 91.912518   seshat            isis               Portmap  V2  
GETPORT Call (Reply In 147) MOUNT(100005) V:3 UDP
     147 91.912960   isis              seshat             Portmap  V2  
GETPORT Reply (Call In 146) Port:1020
     148 91.913290   seshat            isis               MOUNT    V3  
MNT Call (Reply In 183)[Short Frame]
     183 113.949976  isis              seshat             MOUNT    V3  
MNT Reply (Call In 148)[Short Frame]


ping from seshat to isis is currently without delay, vice-versa is  
delayed.

seshat# ping isis
PING isis (isis ): 56 data bytes
64 bytes from isis : icmp_seq=0 ttl=255 time=0.280 ms
64 bytes from isis : icmp_seq=1 ttl=255 time=0.250 ms
64 bytes from isis : icmp_seq=2 ttl=255 time=0.265 ms

isis# ping seshat
PING seshat (seshat): 56 data bytes
.....(imagine 20secs delay here)
64 bytes from seshat: icmp_seq=0 ttl=255 time=45.535 ms
64 bytes from seshat: icmp_seq=1 ttl=255 time=0.332 ms
64 bytes from seshat: icmp_seq=2 ttl=255 time=0.323 ms