Re: Occasional weird TCP behavior

[Greg Troxel <gdt%lexort.com@localhost>]

Brian Marcotte <marcotte%panix.com@localhost> writes:

> On the client machine, I saw this:
>
>   client      server
>   ------      --------
>   SYN
>               RST,ACK  ***
>               SYN,ACK
>   RST
>
> That "RST,ACK" was not actually sent from the server, it was sent from a
> third host.

Wow, I didn't see that coming from your first mail!  It's great you found it
so fast.

> The Linux dom0 uses the "bridge" device which acts similar to a switch.
> It has a MAC address table so it knows what port to send the packet.
> Like a switch, if the MAC is not known, the packet will be sent to all
> ports. This is why the third host saw the first SYN packet.

I think this is a clue as to the first bug.

With a real computer and a real switch, an ethernet packet arriving over
the wire for some other mac makes sense; that's what would happen with a
hub in the pre-switch days anyway.

But, the ethernet hardware would filter out the packet, because the
destination address is unicast and not ours.  And even if in promiscuous
mode, the stack would drop it.

I suspect that the xvif input side is not doing these checks.  IMHO it
should be exactly like a wired interface, except there is no wire and no
hardware.  But the semantics should match.

> On that third host, this was in /etc/npf.conf:
>
>       # Allow ssh only from selected hosts.
>       pass in final proto tcp from 10.0.1.0/24 to any port 22
>       block return-rst in final proto tcp from any to any port 22
>             ^^^^^^^^^^
>
> The third host sent the RST to the client even though the packet was not
> destined for it. It even responded with the server's IP address.
>
> So, should NPF work like this? The fact that it responded with an address
> it didn't own seems especially unusual.

Two more bugs!

1) npf and all firewalls sit in an awkward middle ground where they intend
to somewhat operate on incoming interfaces -- this is necessary for
filtering traffic one is routing.  They also intend to operate on
packets arriving at the local stack.  I'd have to go read the npf docs
and see if rules can be written for "ip_input" vs received on the
interface.  Usually it's some combination of not mattering and trial and
error.

I'd call this an architecture bug in npf, if you can't write rules that
affect the stack but not the links, and an npf coding error if you can.
I suspect that view is controversial :-)

2) Sending a RST with the address it doesn't own feels like a bug in this
case.  The computer is not a router and the rule is not intended to
protect a subnet.

But if you had a subnet behind a machine with npf, such as the usual
home router and LAN, and you wanted to return-rst to an ssh packet
intended for a host on the LAN, it would have to do this.

I'd say that npf probably ought to be able to have rules that handle
this, separating return-rst for my addrs vs me being a router, and even
then, having a way to declare the set of addresses that this host can
speak for.

Overall, with xvif not receiving the packet (as not addressed to us on
ethernet), the rest should be avoided.

(That's assuming all your hosts have distinct mac addresses but I don't
see any hints of dups.)