Greetings:

I have been successfully using the NetBSD kernel with the SNAP-KAME
extensions for the multipath support in radix routing table for over
a year.  In my NetBSD+KAME kernel, I simply enable the "RADIX_MPATH"
kernel option.

For over a year, I had a gateway server with 3 external interfaces
configured with 3 external IP addresses on separate networks with
3 unrelated Internet providers.  The "RADIX_MPATH" kernel option
enabled multiple default routing which allowed the gateway server to
receive packets on any of the 3 interfaces, and route return packets
back through the same interface the corresponding connection came
from.  This worked great for over a year!

Recently I purchased 3 DSL router firewalls, to use as an additional
firewall layer, and to remove the overhead of PPPoE from the gateway
server.  Each of the 3 DSL routers are connected on the LAN side to a
single external interface on the gateway server via a cross-over
ethernet cable, and on the WAN side to the gateway of the
corresponding ISP.  The DSL routers are also running NAT, and the new
IP addresses of the external interfaces on the gateway server and the
LAN interfaces on the routers have been changed to non-routeable IP.
Unfortunately, the multiple default routing no longer work in this
setup.  :-(

So for the past year, the gateway server was connected to the Internet
as follows:

-----------------------------------------------------------------------
(server ether0) 200.200.200.200	=> (isp0 gateway) 200.200.200.1
(server ether1) 211.211.211.211	=> (isp1 gateway) 211.211.211.1
(server ether2) 222.222.222.222	=> (isp2 gateway) 222.222.222.1
-----------------------------------------------------------------------

And successfully used the following route commands for the multiple
gateways:

-----------------------------------------------------------------------
route add -net default 200.200.200.1
route add -net default 211.211.211.1
route add -net default 222.222.222.1
-----------------------------------------------------------------------

So when a packet is sent via "isp2" to "ether2" on the gateway server,
then the return packet is sent back through "ether2" and "isp2".

Now that I added the 3 DSL routers, the gateway server is now connected
to the internet as follows:

-----------------------------------------------------------------------
(server ether0) 192.168.254.2  => (router0 lan) 192.168.254.1 => (router0 wan) 200.200.200.200	=> (isp0 gateway) 200.200.200.1
(server ether1) 192.168.254.6  => (router1 lan) 192.168.254.5 => (router1 wan) 211.211.211.211	=> (isp1 gateway) 211.211.211.1
(server ether2) 192.168.254.10 => (router2 lan) 192.168.254.9 => (router1 wan) 222.222.222.222	=> (isp2 gateway) 222.222.222.1
-----------------------------------------------------------------------

And I am using the following route commands on the gateway server.

-----------------------------------------------------------------------
route add -net default 192.168.254.1
route add -net default 192.168.254.5
route add -net default 192.168.254.9
-----------------------------------------------------------------------

But this time, when a packet is sent via "isp2" to "router2" then "ether2"
on the gateway server, the return packet is unfortunately sent back
through "ether0" then "router0" and "isp0".

Other than the NAT on the DSL routers, I can't think of anything else that
would confuse the multipath default routing.  Does anyone have a solution
on how to get the above configuration working, other than removing the DSL
routers and reverting to my previous setup.  Any assistance would be
greatly appreciated.

Thank you in advance.
Sincerely, Alicia.