Re: vlans and bridges in Xen

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Tue, Sep 06, 2011 at 02:42:06PM +0200, Roger Pau Monné wrote:
> Hello,
> 
> I'm trying to set up a DomU that is connected to a vlan trunk in a
> transparent way. I've read other mails in the list regarding this
> configuration, but I cannot get it working. My current configuration:
> 
> bnx1 is connected to a trunk of several tagged vlans (with ids: 2301, 2302...)
> vlan230x is the vlan interface that corresponds to each vlan id.
> bridge230x is a bridge of vlan230x.
> 
> The vlans are created using this scripts:
> 
> create
> vlan 230x vlanif bnx1
> 
> (where x is the id of the vlan)
> 
> And the bridges:
> 
> create
> !brconfig $int add vlan230x up
> 
> Le'ts take vlan 2301 as an example, here is the relevant configuration
> after the scripts are executed:
> 
> bnx1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
> mtu 1500
>         
> capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
>         enabled=0
>         address: 18:03:73:0a:89:33
>         media: Ethernet autoselect (1000baseSX full-duplex)
>         status: active
>         inet6 fe80::1a03:73ff:fe0a:8933%bnx1 prefixlen 64 scopeid 0x2
> vlan2301: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
>         
> capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
>         enabled=0
>         vlan: 2301 parent: bnx1
>         address: 18:03:73:0a:89:33
>         inet6 fe80::1a03:73ff:fe0a:8933%vlan2301 prefixlen 64 scopeid 0x4
> bridge2301: flags=41<UP,RUNNING>
>         Configuration:
>                 priority 32768 hellotime 2 fwddelay 15 maxage 20
>                 ipfilter disabled flags 0x0
>         Interfaces:
>                 vlan2301 flags=3<LEARNING,DISCOVER>
>                         port 4 priority 128
>         Address cache (max cache: 100, timeout: 1200):
> 
> Configure vlan2301 for Dom0:
> 
> ifconfig vlan2301 inet 172.23.1.11 netmask 255.255.255.0
> 
> Dom0 works fine, can ping the switch and other elements on the same vlan.
> 
> Now DomU configuration, in the config file:
> 
> vif = [
>     (...)
>     'bridge=bridge2301',
>     (...)
> ]
> 
> Start DomU, network config inside DomU:
> 
> re2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
>       options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
>       ether 00:16:3e:49:d9:ad
>       inet 172.23.1.1 netmask 0xffffff00 broadcast 172.23.1.255
>       inet6 fe80::216:3eff:fe49:d9ad%re2 prefixlen 64 scopeid 0x3
>       nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
>       media: Ethernet autoselect (100baseTX <full-duplex>)
>       status: active
> 
> Now from the DomU I can ping the Dom0, but not the outside, and from
> the Dom0 I can ping both the DomU and the outside. brconfig shows the
> following after the DomU start:
> 
> bridge2301: flags=41<UP,RUNNING>
>         Configuration:
>                 priority 32768 hellotime 2 fwddelay 15 maxage 20
>                 ipfilter disabled flags 0x0
>         Interfaces:
>                 tap2 flags=3<LEARNING,DISCOVER>
>                         port 23 priority 128
>                 xvif1i2 flags=3<LEARNING,DISCOVER>
>                         port 16 priority 128
>                 vlan2301 flags=3<LEARNING,DISCOVER>
>                         port 4 priority 128
>         Address cache (max cache: 100, timeout: 1200):
>                 20:fd:f1:77:b2:78 vlan2301 960 flags=0<>
>                 00:16:3e:49:d9:ad tap2 880 flags=0<>
> 
> If I do a tcpdump of the interface vlan2301 I see that the DomU sends
> the ping, but the response never reaches the interface:
> 
> 14:34:34.689985 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
> 14:34:35.691147 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
> 
> And if I perform a tcpdump of the physical (bnx1) interface I see the 
> following:
> 
> 14:35:22.211996 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
> 14:35:22.213187 ARP, Reply 172.23.1.2 is-at 20:fd:f1:77:b2:78 (oui
> Unknown), length 46
> 14:35:23.213141 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
> 14:35:23.214421 ARP, Reply 172.23.1.2 is-at 20:fd:f1:77:b2:78 (oui
> Unknown), length 46
> 
> So it seems like 172.23.1.2 gets the request and issues the reply, but
> the reply never gets to the DomU. I've tried with both HVM and PV
> domains (running FreeBSD and Debian) but the result is the same.

Can you run 'tcpdump -e -v' on bnx1 ?

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--