Re: npf bug(?)

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Thu, 13 Apr 2017 08:13:12 +0200 (CEST)
    From:        6bone%6bone.informatik.uni-leipzig.de@localhost
    Message-ID:  <Pine.NEB.4.64.1704130759160.22851%6bone.informatik.uni-leipzig.de@localhost>


  | npf:
  | 
  | Fragmentation:
  |          870 fragments
  |          774 reassembled
  |          102 failed reassembly
  | 
  | netstat:
  |          1644 fragments received
  |          0 fragments dropped (dup or out of space)
  |          0 fragments dropped (out of ipqent)
  |          102 malformed fragments dropped
  |          33 fragments dropped after timeout
  |          774 packets reassembled ok
  | 
  | There is again a difference between the npf counters and the counters of 
  | the IP stack.

Not a lot - perhaps not any - it just looks to me as if they're counting
slightly different things.

Both agree that 774 packets were reassembled.   To be reassembled at least
2 incoming fragments need to be joined, so we can expect that at least 1548
fragments need to have been received.

But npf says only 870 fragments received - considering all of that info
together tells me that npf is either only counting initial fragments (offset 0)
or only considering non-initial fragments (offset != 0) or one of the
two similar cases with the last fragment (M != 0 or M == 0).   That is.
in each completed packet, one (at least) of the incoming fragments is not
counted in the npf "fragments" counter.

On the other hand. with the netstat stats (the IP level data) it looks
obvious that all fragments (initial, last, and any in the middle) are
being counted as fragments received.

Now from netstat we see 1644 frags were received, and 135 were dropped
(102 considered "malformed" and 33 after a timeout - which would mean, I
assume, that the companion fragment never appeared - it can also mean that
one of the frags was duplicated in transit (though that is rare) and while
the packet was reassembled from one of the duplicated pair, and the other
fragment from the original packet, the other of the duplicated pair never
receives its companion (both fragments from one initial packet being duplicated
in the network would be too improbable to believe)).   Take the (minimum)
1548 fragments that were joined into packets and the 135 that were not,
and we should have received (at least) 1548+135 == 1683 frags, but apparently
we only got 1644.   The only real explanation for that is that something is
being counted twice (or less likely, that the incoming fragments count is
missing some fragments.)   Note that even if the 33 timed out frags were
also counted as malformed, 1548+102 == 1650 which is more than 1644.

The npf stats show a similar anomaly - aside from agreeing that 102 of the
received frags failed reassembly (presumably the same 102 that netstat
reported were malformed -- and apparently not revealing anything that
failed reassembly as a companion never showed up), there were 774 packets
reassembled, and 102 that weren't, which should mean at least 876 fragments
received, but npf counts only 870.   Here there are other possible explanations
for the discrepancy though, some of the malformed fragments counted might
have been whichever fragment npf failed to count as a "fragment" in its
count of fragments, or npf might be counting only initial, or final, fragments
in its "fragments" count, and there could have been a few packets fragmented
into more than 2 fragments.

It is perphaps interesting that the 1650 (netstat) frags that we should have
received (to produce 774 packets while dropping 102 frags) is 6 more than
the count of frags actually received, so is the 876 that npf should have
counted (774 + 102) 6 more than its actual count.

That the difference is 6 in both cases indicates (to me, as a probability
anyway) that the same counting "issue" is happening in both netstat and
npf, but again this makes it look more as if (aside from the counters
counting different things, which is OK if it is understood) that both
are seeing (and counting) the same fragments/packets.

While these oddities in the counting probably ought be explained, there's
nothing really different in what the two of them are reporting happened,
both agree that 774 packets were eventually produced from recombining
fragments, and that 102 fragments apparently had some defect which prevented
reasembly.

Perhaps if npf happens to be counting only initial, or final, packets in
its "fragments" counter, it could be reported as "fragmented packets"
rather than "fragments" in the npf ouput.    All its numbers would be
consistent (or at least might be) if that is the case, 870 fragmented
packets received, 774 successfully, 102 incoming frags dropped (of which
it is possible that some were from the same initially fragmented packet,
probably 6, as 870-774 == 96, so 96 packets actually failed to be
reassembled - under this hypothesis - with 102 frags dropped, the extra
6 dropped frags would have been duplicates, or extra frags from packets
that failed reassembly).

kre