On Tuesday 18 February 2003 19:24, der Mouse wrote:
> > [Linux]
> > route add -host 4.3.2.1 dev eth0
> 
> I think I see.  This amounts to my (b), I think, done with a new type
> of route, one which routes an arbitrary address (or possibly a whole
> network) to an interface rather than to a gateway.  A little like an
> interface route in a normal IP stack, but unrelated to any interface
> address.  See below about breaking the IP routing model.
> 
> I'm curious, though - why not just "route add default dev eth0"?  Does
> that not work?

If I recall my Linux tinkering from the last time I set up a lab full of them, 
yes, that does work fine. But that means Linux will do an ARP who-has for 
everything you try to communicate with unless a more specific route is in 
effect for the destination. I think. I'd be breaking a coworkers network 
connectivity to test it out, so tests will have to wait. :)

> >> I still can't figure out what it would even _mean_ to have a route
> >> pointing to a gateway what's not on-net for any configured interface
> >> address, or where you would expect such packets to be sent.
> > You can't program the default route with an IP address that isn't in
> > your netmask, even if it's still on the same ethernet segment as you
> > are.
> 
> Of course not, because you're not on the same network (in the IP sense,
> not in the layer-2 sense).

But technically there's no reason not to allow it. It's a common 
(mis)configuration that I've seen repeated all over the darn place.

> > How did that IP get in there?
> 
> It's a more or less arbitrary IP that is not on-net for any configured
> interface address.  I _thought_ we were talking about routes with
> gateways that are not on-net for any configured interface address....

Ah, I see. It was a rhetorical example in my ideal world. I understand.

> > According to what you're describing above, 192.168.14.88 can't *be*
> > the default route because NetBSD chokes on it.
> 
> Exactly.  You seem to want to make it not choke, and I was trying to
> find out how you proposed to define it so the behaviour would be
> well-defined.

Precisely! Not to choke...

> > On Linux, we could do this to get around that minor problem:
> 
> > route add -host 192.168.14.88 dev eth0
> 
> Okay, that answers one question (the one of how it is supposed to tell
> what interface to send the packets out): the admin must configure it.

Yes. If the admin configures it, you can get some pretty strange things going 
under Linux--but by default it just seems to behave as a human would 
intuitively expect. "It's there on-ethernet.. I can ping it.. why can't it be 
my default route?"

> How does it tell what MAC address to send to?  Just arp for
> 192.168.14.88 as if eth0 had an address it's on-net for?  Based on a
> quick test I just did (using srt, which I discussed here briefly a
> little while ago, to convince it to send packets out an Ethernet with
> an off-subnet destination address), I don't think the arp code will
> like that.  I suppose you could try to "fix" it....

Correct. In my example, I told the OS to expect to find it directly 
on-ethernet, so it'll arp for it, then attempt to send packets destined for 
the rest of the internet to that machine. I don't think the same method works 
for gateways that are "one hop out" and not directly on the same ethernet 
segment.

The ARP code has a different problem. You can't publish an ARP entry only to a 
specific interface, which stymied me a while back when trying to build a 
proxy-arp gateway for an office network. I was used to the Linux capability:

arp -i eth1 -s 129.22.8.32 00:0a:0b:0c:0d:0e pub

But under the NetBSD firewall I built, I couldn't do this. Doh! I was so sure 
that NetBSD could do it, at the time. :-)

> It's no problem at all - provided the machine you're on has an address
> on that "other network" on the interface in question.

Seems to me that is a tremendous waste of IP addresses. Each machine "should" 
then have an IP address for every network that lives on that segment. Seems 
like a lot of unnecessary aliases flying around. :)

> Just running
> multiple (sub)nets on a single cable doesn't cause any problems.  It's
> only when you expect a machine to speak directly to a network it
> doesn't have any address on that things break. 

Apparently communicating with the network itself it simple--I added a route to 
that network and was able to ping a number of IPs on a foreign network in my 
last email.

> IP just isn't designed
> for that; the routing architecture involves one or more gateways
> between any two distinct (sub)networks, gateways with addresses on the
> relevant networks.  It appears you're trying to change that, without
> actually redesigning IP routing to match.  I'm actually somewhat
> astonished NetBSD comes as close to letting you get away with it as it
> does.

Unfortunately, without the intuitive behaviour I'm describing, converts coming 
to NetBSD from Linux are going to be sitting there scratching their heads, 
coming here to ask questions, and then ultimately giving up after realizing 
that NetBSD isn't as capable, routing-wise, as we all claim it to be. Broken 
behaviour or not (and believe me, I can see why it's considered broken) Linux 
can just do what you tell it to if the request is reasonable.

> I'm running 216.46.5.0/28 and 10.0.2.0/24 on the same "cable"
> (actually, 10baseT hub) right now.  It works fine - because each
> machine involved has two addresses on the relevant interface, one on
> each network.

But intuitively, you should need only one.

> The message is misleading.  It actually means "network not directly
> attached", and arguably another errno should have been added for that.
> You can probably ping 204.152.184.116 (www.netbsd.org), too, but that
> doesn't mean you should be able to "route add default 204.152.184.116".

Definitely a difference--that IP address is not directly ARP'able.

> > Now, routing directly to foreign networks seems possible:
> 
> > :douglas:17:13:52 /doke# route add -net 192.168.0.0 -interface 10.0.2.5
> > :douglas:17:13:54 /doke# ping 192.168.0.1
> > PING 192.168.0.1 (192.168.0.1): 56 data bytes
> > 64 bytes from 192.168.0.1: icmp_seq=0 ttl=255 time=0.897 ms
> 
> > But not default routes.
> 
> You haven't shown that.  You didn't try "route add default -interface
> 10.0.2.5", did you?  I don't see it above.

Intuitively, I infer from the above command that all IP addresses would be 
directly reachable on the ethernet interface, which isn't the case.

> What you have shown doesn't work is adding an ordinary (= routed
> through a gateway) default route whose gateway is a host that's behind
> one of these funny "destination is an interface" routes.  I suspect
> that you'll find that you can't "route add 172.16.0.0/16 192.168.0.1"
> any more than you can "route add default 192.168.0.1" - that it's got
> nothing to do with its being a default route, but rather with how
> you're trying to set it up.

Actually, in my example I did show you could add a subnet to an interface:

:douglas:17:13:52 /doke# route add -net 192.168.0.0 -interface 10.0.2.5
:douglas:17:13:54 /doke# ping 192.168.0.1

It results in the following route in the routing table:

192.168/16         link#1             UCS         0        0      -  ex0

And actually, I do have a route like what you mention in my home network, 
where I have a wireless segment attached to a three-interface firewall. The 
route looks like this:

route add -net 172.16.0.0 -netmask 0xfff00000 10.0.0.5

... which seems to work fine.

> > My other gripe is that NetBSD sends out the wrong interface, traffic
> > which must be fast-routed with an ipfilter rule.  A machine with:
> 
> > fxp0: 64.1.2.3
> > fxp1: 10.0.0.5
> 
> > ... will answer out 64.1.2.3 when any non-specifically-routed traffic
> > comes in on 10.0.0.5.
> 
> Only if that's where the route to the peer address points.

And what about multi-homed systems? In my case, I have a firewall attached to 
a cable modem, and then the system where a webserver lives attached to an 
ADSL modem.

In order for the system to answer properly to traffic on both networks, I need 
a fastroute in ipf.conf:

pass out quick on fxp1 to fxp0:10.0.0.10 from 10.0.0.40/32 to any

Again--intuitively, if traffic comes in destined for 10.0.0.0/8, and the 
interface for the 10.0.0.0/8 is sitting there, and the 10.0.0.0/8 is routed 
to that interface.. then outgoing traffic should be answered on that 
interface as well. It makes no sense to force it out the wrong interface with 
no way for an administrator to fix the problem without doing tricky things 
with ipfilter.

I think your source-routing interface was written to address this problem, 
wasn't it?

> > With a source address of 10.0.0.5.  That's just not right.
> 
> The packet should go out the interface through which the route to the
> peer address points.  (If you can state opinion as if it were fact, so
> can I.)

But you don't agree with it or you wouldn't have written the source-routing 
interface that you use.

> If you want to route based on the ip_src of the packet, well, that's
> what I wrote the srt pseudo-interface for; the normal routing tables
> are not designed for that.  (I wanted something very similar: I wanted
> traffic to go out the interface appropriate for traffic's _local_
> address.  Realizing that the routing table isn't designed for that, I
> wrote something that was.)

Linux works just fine in this regard. Paul Vixie wrote a multiple-default 
route patch to get NetBSD working. The need exists, and the reality is that 
there are thousands of configurations just like mine that need the 
capability. You wouldn't have written your source-route pseudo-interface if 
you hadn't been beholden to the need for the capability yourself.

It surprises me that this capability hasn't been introduced to NetBSD while 
Linux has had it .. well as far back as I can remember anyway. :-)