>Number:         29149
>Category:       kern
>Synopsis:       packet scheduling for IPv6-over-IPv4 tunnels is weird
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jan 28 21:22:00 +0000 2005
>Originator:     Gert Doering
>Release:        NetBSD 2.99.15
>Organization:
	
>Environment:
System: NetBSD scotty.greenie.muc.de 2.99.15 NetBSD 2.99.15 (SCOTTY-IPSEC) #24: Thu Jan 27 11:03:11 CET 2005 gert@scotty.greenie.muc.de:/home/sparc64.current/obj/usr/src-current/sys/arch/sparc64/compile/SCOTTY-IPSEC sparc64
Architecture: sparc64
Machine: sparc64
>Description:
	IPv6 packets arriving in an IPv4 tunnel (GIF tunnel from the
	regular kernel sources, or IPv6-over-GRE with my changes) get
	delayed 200-1200 ms before being scheduled for processing.  If there
	is "other" IPv6 traffic (native), packets are processed much quicker.
>How-To-Repeat:
	This PR is based on a problem report on tech-net, to make sure
	it's not lost.  It's related to IPv6-over-GRE work, but the
	problem is seen on IPv6-over-GIF as well.

	Setup a v6-over-v4 tunnel Cisco-NetBSD (it should work with
	NetBSD-NetBSD tunnel as well), tunnel mode "gif".  

	Send IPv6 ping packets towards NetBSD, and wonder about the RTTs ->
	then watch packets with "tcpdump".

	Here's is the setup on the NetBSD side:

	ifconfig gif0 create
	ifconfig gif0 tunnel 193.149.48.168 195.30.70.42
	ifconfig gif0 10.58.58.2 10.58.58.1 netmask 255.255.255.0 link0 up
	ifconfig gif0 inet6 2001:608:4:5555::2/64
	route add -inet6 2001:608:4:5555::/64 2001:608:4:5555::1
	route change -inet6 2001:608:4:5555::/64 -ifp gif0

	And this is the Cisco side:

	interface Tunnel11
	 ip address 10.58.58.1 255.255.255.0
	 ipv6 address 2001:608:4:5555::1/64
	 tunnel source 195.30.70.42
	 tunnel destination 193.149.48.168
	 tunnel mode ipv6ip
	end

	This happens when I run a ping on the Cisco side:

	cisco2514#ping 2001:608:4:5555::2 repeat 2

	Type escape sequence to abort.
	Sending 2, 100-byte ICMP Echos to 2001:608:4:5555::2, timeout is 2 seconds:
	!!
	Success rate is 100 percent (2/2), round-trip min/avg/max = 264/358/452 ms

	(these machines are connected over *LAN*, so RTTs over 200ms are plain
	impossible under normal circumstances)

	This is the tcpdump output on the NetBSD side, taken on the LAN interface
	("tcpdump -i hme0 -v -s0 -n host 195.30.70.42"):

	20:58:43.153611 195.30.70.42 > 193.149.48.168: 2001:608:4:5555::1 > 2001:608:4:5555::2: icmp6: echo request (len 60, hlim 64) (ttl 255, id 23, len 120)
	20:58:43.412376 193.149.48.168 > 195.30.70.42: 2001:608:4:5555::2 > 2001:608:4:5555::1: icmp6: echo reply (len 60, hlim 64) (ttl 30, id 480, len 120)
	20:58:43.424927 195.30.70.42 > 193.149.48.168: 2001:608:4:5555::1 > 2001:608:4:5555::2: icmp6: echo request (len 60, hlim 64) (ttl 255, id 24, len 120)
	20:58:43.873478 193.149.48.168 > 195.30.70.42: 2001:608:4:5555::2 > 2001:608:4:5555::1: icmp6: echo reply (len 60, hlim 64) (ttl 30, id 481, len 120)

	as you can see from the time stamps, the (encapsulated) ICMPv6 echo request
	is received on the LAN, and it takes full 250ms to send back the 
	(encapsulated) IPv6 echo reply packet.  This is the same packet scheduling
	weirdness that I've seen with my IPv6-over-GRE tests, and I read it as
	something more fundamental in the way the different protocol queues are
	handled.


	For completeness, I've run the same tests while at the same time running
	"ping6 -i 0.02 $netbsd-machine" from another host on the LAN, and this
	is the result:

	cisco2514#ping 2001:608:4:5555::2 repeat 2

	Type escape sequence to abort.
	Sending 2, 100-byte ICMP Echos to 2001:608:4:5555::2, timeout is 2 seconds:
	!!
	Success rate is 100 percent (2/2), round-trip min/avg/max = 12/14/16 ms

	21:07:04.630021 195.30.70.42 > 193.149.48.168: 2001:608:4:5555::1 > 2001:608:4:5555::2: icmp6: echo request (len 60, hlim 64) (ttl 255, id 25, len 120)
	21:07:04.639910 193.149.48.168 > 195.30.70.42: 2001:608:4:5555::2 > 2001:608:4:5555::1: icmp6: echo reply (len 60, hlim 64) (ttl 30, id 495, len 120)
	21:07:04.653535 195.30.70.42 > 193.149.48.168: 2001:608:4:5555::1 > 2001:608:4:5555::2: icmp6: echo request (len 60, hlim 64) (ttl 255, id 26, len 120)
	21:07:04.659880 193.149.48.168 > 195.30.70.42: 2001:608:4:5555::2 > 2001:608:4:5555::1: icmp6: echo reply (len 60, hlim 64) (ttl 30, id 496, len 120)

	-> the delay times are much lower, if there are other IPv6 packets being
	processed anyway (as in my GRE tests, see previous e-mail).

>Fix:
	I have no idea how to fix this, sorry.  It's related to how the
	packet queues are scheduled (IPv4 queue is run, packet gets queued
	to ipv6intrq -> schednetisr(NETISR_IPV6) is called -> and then
	"some random delay" happens before the IPv6 packet queue is actually
	run.  IPv6 packets on physical interfaces trigger packet processing
	immediately).

>Unformatted:
 Thu, Jan 27 09:11:00 GMT 2005