Re: Xen and VLANs

[Manuel Bouyer <bouyer%antioche.eu.org@localhost>]

On Thu, Jan 05, 2006 at 09:46:05PM +0100, Johan Ihren wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Hi,
> 
> I'm running a large number (~20) of DomU machines with a rather  
> complex network topology
> that is based on a number of VLANs being configured in each DomU (and  
> the Dom0). Most
> things are working just great: all 16 VLANs work, there are NFS  
> mounted partions, NFS swap,
> ten different DHCP infrastructures on different VLANs, etc, etc.  
> (This is a lab environment
> that usually takes a full classroom of computers that I'm collapsing  
> into a single machine).
> 
> There is just a single problem that I'm having trouble with and that  
> is that my Dom0
> is unable to communicate with my DomUs when using VLANs.
> 
> I have a single bridge, bridge0, configured and it is up and all  
> xennetN devices plus
> the physical interface of the Dom0 (an ste0) are connected to the  
> bridge. The DomUs and
> the Dom0 can communicate just fine over the bridge when not using  
> VLANs but with VLANs
> configured the DomUs can still talk while the Dom0 is isolated.
> 
> Here's an example with one VLAN interface, the Dom0 and two DomUs.  
> All three talk just
> fine over the [ste0, xennet0, xennet0] interfaces, but over [vlan1,  
> vlan1, vlan1] the
> Dom0 is isolated. I've removed the v6 stuff for brevity (it is of  
> course there).
> 
> Dom0:
> ste0:  
> flags=8b43<UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST>  
> mtu 1500
>         address: 00:05:5d:1a:88:d8
>         media: Ethernet autoselect (100baseTX full-duplex)
>         status: active
>         inet 192.168.32.32 netmask 0xffffff00 broadcast 192.168.32.255
> vlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         vlan: 1 parent: ste0
>         address: 00:05:5d:1a:88:d8
>         inet 192.168.1.11 netmask 0xffffff00 broadcast 192.168.1.255
> 
> DomU #1:
> xennet0:  
> flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         address: aa:00:00:0c:00:09
>         inet 192.168.32.33 netmask 0xffffff00 broadcast 192.168.32.255
>         inet alias 10.99.99.1 netmask 0xffffff00 broadcast 10.99.99.255
> vlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1496
>         vlan: 1 parent: xennet0
>         address: aa:00:00:0c:00:09
>         inet 192.168.1.10 netmask 0xffffff00 broadcast 192.168.1.255
> 
> DomU #2:
> xennet0:  
> flags=8863<UP,BROADCAST,NOTRAILERS,RUNNING,SIMPLEX,MULTICAST> mtu 1500
>         address: aa:00:00:00:00:01
>         inet 192.168.32.1 netmask 0xffffff00 broadcast 192.168.32.255
> vlan1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1496
>         vlan: 1 parent: xennet0
>         address: aa:00:00:00:00:01
>         inet 192.168.1.1 netmask 0xffffff00 broadcast 192.168.1.255
> 
> All NetBSD 3.0REL, xentools 2.0.7 and modern stuff everywhere.
> 
> Any ideas anyone?

I tried this too, and this doesn't work. The problem is with the way
bridge and vlan works, once you have a brige packets will never be
passed to the vlan interfaces (or the opposite, I don't remember).

My workaround was to bridge the N vlan interface in dom0 (creating N
xennet interface for each domU) instead of bridging the physical interface
and creating the vlans in each domU

-- 
Manuel Bouyer <bouyer%antioche.eu.org@localhost>
     NetBSD: 26 ans d'experience feront toujours la difference
--