vlans and bridges in Xen

[Roger Pau Monné <roger.pau%entel.upc.edu@localhost>]

Hello,

I'm trying to set up a DomU that is connected to a vlan trunk in a
transparent way. I've read other mails in the list regarding this
configuration, but I cannot get it working. My current configuration:

bnx1 is connected to a trunk of several tagged vlans (with ids: 2301, 2302...)
vlan230x is the vlan interface that corresponds to each vlan id.
bridge230x is a bridge of vlan230x.

The vlans are created using this scripts:

create
vlan 230x vlanif bnx1

(where x is the id of the vlan)

And the bridges:

create
!brconfig $int add vlan230x up

Le'ts take vlan 2301 as an example, here is the relevant configuration
after the scripts are executed:

bnx1: flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>
mtu 1500
        
capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
        enabled=0
        address: 18:03:73:0a:89:33
        media: Ethernet autoselect (1000baseSX full-duplex)
        status: active
        inet6 fe80::1a03:73ff:fe0a:8933%bnx1 prefixlen 64 scopeid 0x2
vlan2301: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        
capabilities=3f00<IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx,TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx>
        enabled=0
        vlan: 2301 parent: bnx1
        address: 18:03:73:0a:89:33
        inet6 fe80::1a03:73ff:fe0a:8933%vlan2301 prefixlen 64 scopeid 0x4
bridge2301: flags=41<UP,RUNNING>
        Configuration:
                priority 32768 hellotime 2 fwddelay 15 maxage 20
                ipfilter disabled flags 0x0
        Interfaces:
                vlan2301 flags=3<LEARNING,DISCOVER>
                        port 4 priority 128
        Address cache (max cache: 100, timeout: 1200):

Configure vlan2301 for Dom0:

ifconfig vlan2301 inet 172.23.1.11 netmask 255.255.255.0

Dom0 works fine, can ping the switch and other elements on the same vlan.

Now DomU configuration, in the config file:

vif = [
    (...)
    'bridge=bridge2301',
    (...)
]

Start DomU, network config inside DomU:

re2: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500
        options=98<VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM>
        ether 00:16:3e:49:d9:ad
        inet 172.23.1.1 netmask 0xffffff00 broadcast 172.23.1.255
        inet6 fe80::216:3eff:fe49:d9ad%re2 prefixlen 64 scopeid 0x3
        nd6 options=3<PERFORMNUD,ACCEPT_RTADV>
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active

Now from the DomU I can ping the Dom0, but not the outside, and from
the Dom0 I can ping both the DomU and the outside. brconfig shows the
following after the DomU start:

bridge2301: flags=41<UP,RUNNING>
        Configuration:
                priority 32768 hellotime 2 fwddelay 15 maxage 20
                ipfilter disabled flags 0x0
        Interfaces:
                tap2 flags=3<LEARNING,DISCOVER>
                        port 23 priority 128
                xvif1i2 flags=3<LEARNING,DISCOVER>
                        port 16 priority 128
                vlan2301 flags=3<LEARNING,DISCOVER>
                        port 4 priority 128
        Address cache (max cache: 100, timeout: 1200):
                20:fd:f1:77:b2:78 vlan2301 960 flags=0<>
                00:16:3e:49:d9:ad tap2 880 flags=0<>

If I do a tcpdump of the interface vlan2301 I see that the DomU sends
the ping, but the response never reaches the interface:

14:34:34.689985 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
14:34:35.691147 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28

And if I perform a tcpdump of the physical (bnx1) interface I see the following:

14:35:22.211996 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
14:35:22.213187 ARP, Reply 172.23.1.2 is-at 20:fd:f1:77:b2:78 (oui
Unknown), length 46
14:35:23.213141 ARP, Request who-has 172.23.1.2 tell 172.23.1.1, length 28
14:35:23.214421 ARP, Reply 172.23.1.2 is-at 20:fd:f1:77:b2:78 (oui
Unknown), length 46

So it seems like 172.23.1.2 gets the request and issues the reply, but
the reply never gets to the DomU. I've tried with both HVM and PV
domains (running FreeBSD and Debian) but the result is the same.

Thanks for the help, Roger.