Howto use agr to aggregate VPN tunnels

[BERTRAND Joël <joel.bertrand%systella.fr@localhost>]

	Hello,

	I open a new thread as I have made some tests and I'm now pretty sure 
that issue I see comes from NetBSD.

	I'm able to use agr with two physical ethernet controllers. But I'm not 
able to obtain a running agr interface with two OpenVPN tunnels.
Maybe problem comes from NetBSD kernel, maybe from misconfiguration, I 
have no idea to fix it.

	I have created two OpenVPN tap tunnels between a server an a NetBSD 
workstation (DEC PWS500au running 7.99.43, but I have seen same issue 
with 7.0.2 on amd64). Both tunnels runs as expected.

	I have removed inet/inet6 address from both tunnels :

tap0: flags=0x8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
        ec_enabled=0
        address: f2:0b:a4:b2:cb:28
        media: Ethernet autoselect
tap1: flags=0x8802<BROADCAST,SIMPLEX,MULTICAST> mtu 1500
        ec_capabilities=5<VLAN_MTU,JUMBO_MTU>
        ec_enabled=0
        address: f2:0b:a4:e9:16:fe
        media: Ethernet autoselect

and I have created agr0 (round robin):

agr0: flags=0xb843<UP,BROADCAST,RUNNING,SIMPLEX,LINK0,LINK1,MULTICAST> 
mtu 1500
        agrport: tap0, flags=0x3<COLLECTING,DISTRIBUTING>
        agrport: tap1, flags=0x3<COLLECTING,DISTRIBUTING>
        address: f2:0b:a4:b2:cb:28
        inet 192.168.100.2/24 broadcast 192.168.100.255 flags 0x0
        inet6 fe80::f00b:a4ff:feb2:cb28%agr0/64 flags 0x2<TENTATIVE> 
scopeid 0x6

	I have checked that 192.168.100.0/24 route goes through agr0 :
Internet:
Destination        Gateway            Flags    Refs      Use    Mtu 
Interface
default            weierstrass        UG          -        -      -L epic0
127/8              localhost          UGR         -        -  33112L lo0
localhost          lo0                UHl         -        -  33112L lo0
192.168.0/24       link#3             U           -        -      -L epic0
einstein           link#3             UHl         -        -      -L lo0
192.168.100/24     link#6             U           -        -      -L agr0
192.168.100.2      link#6             UHl         -        -      -L lo0

	If I try to ping 192.168.100.1 (server), kernel sends packets to agr0 :
einstein# tcpdump -i agr0 -p
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on agr0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:34:25.250725 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
10:34:26.253355 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
10:34:27.252354 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
10:34:28.253310 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
10:34:29.252338 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
10:34:30.252331 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
10:34:31.256259 ARP, Request who-has 192.168.100.1 tell 192.168.100.2, 
length 28
^C
7 packets captured
7 packets received by filter
0 packets dropped by kernel

but no packet is sent by tap0 or tap1 :

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel
einstein# tcpdump -i tap1 -p
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap1, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

	In reception, when server tries to ping NetBSD client, tap0 and tap1 
receive ethernet packets, but these packets are never transmitted to agr0 !

einstein# tcpdump -i tap0 -p
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:45:53.866399 ARP, Request who-has 192.168.100.2 tell 192.168.100.1, 
length 28
10:45:55.914946 ARP, Request who-has 192.168.100.2 tell 192.168.100.1, 
length 28
...
einstein# tcpdump -i agr0 -p
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on agr0, link-type EN10MB (Ethernet), capture size 262144 bytes
^C
0 packets captured
0 packets received by filter
0 packets dropped by kernel

	I don't understand why there is no logical connection between tap0/tap1 
and agr0. Of course, I have verified that agr0 uses tap0 and tap1 as 
slave interfaces.

	The same configuration runs fine with two physical ethernet 
controllers. I have create agr1 that aggregates wm1 and wm2 (802.3ad):

agr1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
	capabilities=7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
        capabilities=7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
        enabled=0
        agrport: wm1, flags=0x3<COLLECTING,DISTRIBUTING>
        agrport: wm2, flags=0x3<COLLECTING,DISTRIBUTING>
        address: 68:05:ca:02:b2:59
        inet 192.168.10.128 netmask 0xffffff00 broadcast 192.168.10.255
        inet6 fe80::6a05:caff:fe02:b259%agr0 prefixlen 64 scopeid 0x5
        inet6 2001:7a8:a8ed:10::128 prefixlen 64

and agr1 runs as expected.

	When I compare agr0 and agr1, I note that agr0 doesn't indicate IPv4 
and IPv6 capabilities. Why ? If I understand, agr0 has to indicate these 
capabilities to work as expected.

	Best regards,

	JKB