Re: something is randomly closing ssh-tunnels (was: ipfilter randomly dropping..)

[Petar Bogdanovic <petar%smokva.net@localhost>]

On Tue, Jun 24, 2014 at 11:39:47PM +1000, Darren Reed wrote:
> 
> Oh, I forgot, there are internal code paths in ipfilter/npf that can
> return ENETUNREACH.
> 
> If you are using NetBSD 6 with ipfilter, comparing the output of this:
> 
> ipfstat | grep 'block reason'
> 
> from before and after might be illuminating.
> 
> Or maybe just compare the entire output of "ipfstat" and "ipfstat -s"
> from before and after.

No problem, ipfstat before and after failed tunnel (reproducing it now
is very easy):

# ls -la
total 12
drwxrwxrwt   2 root  wheel  512 Jun 25 10:10 .
drwxr-xr-x  19 root  wheel  512 Jun 20 20:51 ..
-rw-r--r--   1 root  wheel  535 Jun 25 10:09 ipfstat-s.1403683750
-rw-r--r--   1 root  wheel  535 Jun 25 10:10 ipfstat-s.1403683819
-rw-r--r--   1 root  wheel  805 Jun 25 10:09 ipfstat.1403683750
-rw-r--r--   1 root  wheel  806 Jun 25 10:10 ipfstat.1403683819


# diff -u ipfstat-s.1403683750 ipfstat-s.1403683819
--- ipfstat-s.1403683750        2014-06-25 10:09:10.000000000 +0200
+++ ipfstat-s.1403683819        2014-06-25 10:10:19.000000000 +0200
@@ -1,27 +1,27 @@
 IP states added:
-       17761 TCP
-       58310 UDP
+       17772 TCP
+       58329 UDP
        92 ICMP
-       76918765 hits
-       439924 misses
+       77026414 hits
+       450385 misses
        0 bucket full
        0 maximum rule references
        0 maximum
        0 no memory
-       14 bkts in use
-       14 active
-       58402 expired
-       17747 closed
+       18 bkts in use
+       18 active
+       58418 expired
+       17757 closed
 State logging enabled
 
 State table bucket statistics:
-       14 in use       
+       18 in use       
        100% hash efficiency
-       0.24% bucket usage
+       0.31% bucket usage
        0 minimal length
        1 maximal length
        1.000 average length
 
 TCP Entries per state
      0     1     2     3     4     5     6     7     8     9    10    11
-     0     0     0     0     3     0     0     0     0     0     8     3
+     0     0     0     0     3     0     0     0     0     0     8     4


# diff -u ipfstat.1403683750 ipfstat.1403683819
--- ipfstat.1403683750  2014-06-25 10:09:10.000000000 +0200
+++ ipfstat.1403683819  2014-06-25 10:10:19.000000000 +0200
@@ -1,22 +1,22 @@
 bad packets:           in 0    out 0
- IPv6 packets:         in 0 out 5153
- input packets:                blocked 53 passed 44336 nomatch 0 counted 0 
short 0
-output packets:                blocked 5218 passed 60118 nomatch 0 counted 0 
short 0
+ IPv6 packets:         in 0 out 5155
+ input packets:                blocked 53 passed 92750 nomatch 0 counted 0 
short 0
+output packets:                blocked 5239 passed 129793 nomatch 0 counted 0 
short 0
  input packets logged: blocked 0 passed 0
-output packets logged: blocked 65 passed 0
+output packets logged: blocked 84 passed 0
  packets logged:       input 0 output 0
- log failures:         input 0 output 13
+ log failures:         input 0 output 32
 fragment state(in):    kept 0  lost 0  not fragmented 0
 fragment state(out):   kept 0  lost 0  not fragmented 0
-packet state(in):      kept 1601       lost 0
+packet state(in):      kept 1631       lost 0
 packet state(out):     kept 27 lost 0
 ICMP replies:  2       TCP RSTs sent:  50
 Invalid source(in):    0
-Result cache hits(in): 1935    (out):  6778
+Result cache hits(in): 4644    (out):  9491
 IN Pullups succeeded:  0       failed: 0
 OUT Pullups succeeded: 0       failed: 0
 Fastroute successes:   3538    failures:       0
 TCP cksum fails(in):   0       (out):  0
-IPF Ticks:     781113
+IPF Ticks:     781251
 Packet log flags set: (0)
        none