Re: kern/52036: netbsd-7 kernel crash at high network load

[6bone%6bone.informatik.uni-leipzig.de@localhost]

The following reply was made to PR kern/52036; it has been noted by GNATS.

From: 6bone%6bone.informatik.uni-leipzig.de@localhost
To: gnats-bugs%NetBSD.org@localhost
Cc: 
Subject: Re: kern/52036: netbsd-7 kernel crash at high network load
Date: Wed, 15 Mar 2017 10:18:35 +0100 (CET)

 In the last crash, I was able to collect some information.
 
 The switch has sent almost one million packets per second to the router:
 
    Load-Interval #1: 30 seconds
      30 seconds input rate 17345872 bits/sec, 21620 packets/sec
      30 seconds output rate 668750232 bits/sec, 972039 packets/sec
 
 CPU0 from the router was 100% used interrupts.
 
 load averages:  2.37,  1.92,  1.23;               up 0+00:12:32 
 16:19:21
 24 processes: 22 sleeping, 2 on CPU
 CPU00 states:  0.0% user,  0.0% nice,  0.0% system,  100% interrupt,  0.0% idle
 CPU01 states:  0.2% user,  0.0% nice, 12.8% system,  0.0% interrupt, 87.0% idle
 CPU02 states:  0.0% user,  0.0% nice,  5.0% system,  0.0% interrupt, 95.0% idle
 CPU03 states:  1.6% user,  0.0% nice, 12.8% system,  0.0% interrupt, 85.6% idle
 CPU04 states:  0.0% user,  0.0% nice,  3.0% system,  0.0% interrupt, 97.0% idle
 CPU05 states:  0.0% user,  0.0% nice,  6.0% system,  0.0% interrupt, 94.0% idle
 CPU06 states:  0.0% user,  0.0% nice,  4.6% system,  0.0% interrupt, 95.4% idle
 CPU07 states:  0.2% user,  0.0% nice,  9.4% system,  0.0% interrupt, 90.4% idle
 CPU08 states:  0.4% user,  0.0% nice,  6.8% system,  0.0% interrupt, 92.8% idle
 CPU09 states:  0.0% user,  0.0% nice,  6.4% system,  0.0% interrupt, 93.6% idle
 CPU10 states:  0.0% user,  0.0% nice,  7.8% system,  0.0% interrupt, 92.2% idle
 CPU11 states:  0.6% user,  0.0% nice,  8.2% system,  0.0% interrupt, 91.2% idle
 CPU12 states:  0.2% user,  0.0% nice,  4.6% system,  0.0% interrupt, 95.2% idle
 CPU13 states:  0.0% user,  0.0% nice, 10.6% system,  0.0% interrupt, 89.4% idle
 CPU14 states:  1.6% user,  0.0% nice, 30.7% system,  0.0% interrupt, 67.7% idle
 CPU15 states:  0.4% user,  0.0% nice,  9.4% system,  0.0% interrupt, 90.2% idle
 Memory: 1313M Act, 608K Inact, 6464K Wired, 19M Exec, 1181M File, 22G Free
 Swap: 49G Total, 49G Free
 
 In the time with high packet rates, dmesg delivered the following output:
 
 ...
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2015:211:aff:fef2:xxxx, if=vlan964
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2008:e844:bb94:5f6a:xxxx, if=vlan55
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:14e3:ff32:3f05:xxxx, if=vlan57
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:7c1c:7bb9:7972:xxxx, if=vlan57
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:14e3:ff32:3f05:xxxx, if=vlan57
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:14e3:ff32:3f05:xxxx, if=vlan57
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:14e3:ff32:3f05:xxxx, if=vlan57
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:7c1c:7bb9:7972:xxxx, if=vlan57
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2002:f01b:f17:a865:xxxx, if=vlan8
 nd6_storelladdr: sdl_alen == 0, dst=2001:638:902:2003:14e3:ff32:3f05:xxxx, if=vlan57
 ...
 
 (xxxx removed for safety reasons)
 
 There may be two problems. I am using an Intel 10GE card, ixg driver. All 
 packages come via interface ixg0 into the router. Should not the interrupt 
 throttling protect the CPU? Can it be that the interrupt throttling of the 
 ixg driver does not work properly?
 
 The second problem is ipfilter. If ipfilter is switched off, I could not 
 observe a crash. It can be that the cause of the crash is in ipfilter.
 
 Possibly you can attack netbsd router with this configuration over the 
 network.
 
 
 Regards
 Uwe