Re: Network Oddities

To: Ron Georgia <netverbs%gmail.com@localhost>
Subject: Re: Network Oddities
From: Andy Ruhl <acruhl%gmail.com@localhost>
Date: Fri, 6 May 2022 18:44:48 -0700
On Fri, May 6, 2022 at 4:12 PM Ron Georgia <netverbs%gmail.com@localhost> wrote:
>
> 1. Reading through the ifconfig.if man page I found this,
> "For each interface (nnX) that is to be configured, there should be
> either an ifconfig_nnX variable in rc.conf(5), or an /etc/ifconfig.nnX file"
> Also in "Setting up TCP/IP on NetBSD in practice" document,  I assumed
> the following meant that each interface needed an ifconfig.if file.
> /etc/ifconfig.xxx
>      This file is used for the automatic configuration of the network
> interfaces at boot, see ifconfig.if(5)
>
> https://netbsd.org/docs/guide/en/chap-net-practice.html#chap-net-practice-network-config
>

"That is to be configured" is the important phrase in the context of
this thread.

> 2. I was setting up a QEMU and NVMM, following  Using virtualization:
> QEMU and NVMM (https://www.netbsd.org/docs/guide/en/chap-virt.html). In
> section 30.3, "Configuring bridged networking on a NetBSD host" they
> walk through creating a tap and a bridge. The first run through my qemu
> vm worked, but the host network stopped working. I rebooted and neither
> the host or the guest had working networks. I am not sure why but it may
> be that I had ifconfig.wm1, ifconfig.tap0, ifconfig.bridge0. I thought
> maybe I would use wm1 for the host and wm0 for the guest. Bad idea.
>

I don't know, maybe that would work. Seems like a reasonable idea but
I'm not an expert on this topic.

It's also possible you bridged the VM + both real interfaces causing a
loop or something.

> I did remove the cable from wm0, removed ifconfig.wm0 and
> ifconfig.bridge0 from /etc. I reconfigured ifconfig.tap0 and now
> everything works. In my /etc I have ifconfig.wm1 and ifconfig.tap0
>

Yes. 1 interface per subnet for the most part is how you want to
configure things. 1 default gateway per machine (or VRF if you want to
get fancy).

> 3. "Why are you assuming that this should work?" Honestly, I don't know.
> Pure ignorance I suppose. However, I did learn a lot from this foray.

Good, this is the point. Once you drill into how IP networks work from
a host perspective, this all starts making sense. When you put 2
interfaces in the same subnet inside of the same layer 2 network, the
host starts to have problems figuring out which interface to send
packets out of. I deal with this type of thing all the time at work. I
think part of the problem is there is an assumption that "more
interfaces are better". Someone else said the right thing, anytime you
are putting 2 interfaces on the same network you need some "bundling"
or "bonding" method to make it work right. I'm most familiar with LACP
but there are other methods. Maybe that should be your next foray.

Andy

> On 5/4/22 12:32 AM, Andy Ruhl wrote:
> > On Tue, May 3, 2022 at 10:27 AM Ron Georgia <netverbs%gmail.com@localhost> wrote:
> >>
> >> I am having some odd behavior from my NICs or maybe from the network or
> >> dhcpcd, not sure.
> >>
> >> With ethernet cables plugged into both wm0 and wm1. Everything works;
> >> however the only ifconfig file is ifconfig.wm1. Everything works.
> >> Reading through the docs, the recommendation is to have an ifconfig.if
> >> for each interface. The dhcpcd service is up and running without issue.
> >> I copy ifconfig.wm1 to ifconfig.wm0 and restart the network. But I get
> >> an error:
> >>
> >> sudo service network restart
> >> Stopping network.
> >> Deleting aliases.
> >> Downing network interfaces: wm0 wm1.
> >> Starting network.
> >> Hostname: netverbs57.ronverbs.dev
> >> IPv6 mode: host
> >> Configuring network interfaces: wm0 wm1.
> >> Adding interface aliases:.
> >> Waiting for DAD to complete for statically configured addresses...
> >> ifconfig: SIOCGIFAFLAG_IN6: Can't assign requested address
> >>
> >> After a reboot none of the nics have IP addresses. If I restart dhcpcd I
> >> get this error
> >>
> >> sudo service dhcpcd restart
> >> dhcpcd not running? (check /var/run/dhcpcd.pid).
> >> Starting dhcpcd.
> >> main: control_open: Connection refused
> >> [1]   Segmentation fault (core dumped) RC_PID= _rc_pid=
> >> _rc_original_stdout_fd= _rc_o...
> >>
> >> If I remove /etc/ifconfig.wm0 and reboot, everything returns to normal.
> >> However, if I restart the network, then restart dhcpcd, the same
> >> Sementation fault appears.
> >>
> >> I found this when I tried to create a bridge with a tap.
> >>
> >> Summary:
> >> With only /etc/ifconfig.wm1, both wm0 and wm1 work. The network can be
> >> restarted and dhcpcd can be restarted.
> >> Add /etc/ifconfig.wm0, reboot, no ip addresses assigned. Cannot restart
> >> dhcpcd, seg fault.
> >> Remove /etc/ifconfig.wm0 and reboot, back to normal. Both nics have
> >> addresses. Restart the network works, but restarting dhcpcd results in a
> >> seg fault.
> >>
> >> Anything I can look for?
> >>
> >> ===============================================================
> >> HELPFUL INFO
> >>
> >> ~> uname -a
> >> NetBSD netverbs57.ronverbs.dev 9.2 NetBSD 9.2 (GENERIC) #0: Wed May 12
> >> 13:15:55 UTC 2021
> >> mkrepro%mkrepro.NetBSD.org@localhost:/usr/src/sys/arch/amd64/compile/GENERIC amd64
> >>
> >>
> >> from /etc/rc.conf
> >> # Add local overrides below.
> >> #
> >> hostname=netverbs57.ronverbs.dev
> >> dhcpcd=YES
> >> #dhcpcd_flags="-qM"
> >>
> >> Listing /etc
> >> ll /etc | grep ifconfig
> >> -rw-r--r--   1 root  wheel      20B Apr 10 07:09 ifconfig.wm1
> >>
> >> cat /etc/ifconfig.wm1
> >> up
> >> media autoselect
> >>
> >> ifconfig | grep -w inet
> >>           inet 192.168.1.182/24 broadcast 192.168.1.255 flags 0x0
> >>           inet 192.168.1.186/24 broadcast 192.168.1.255 flags 0x0
> >>           inet 127.0.0.1/8 flags 0x0
> >>
> >>
> >> sudo service dhcpcd status
> >> dhcpcd is running as pid 241.
> >>
> >>
> >> ifconfig
> >> wm0: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> >>           capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
> >>           capabilities=7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
> >>           capabilities=7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
> >>           enabled=0
> >>           ec_capabilities=7<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU>
> >>           ec_enabled=2<VLAN_HWTAGGING>
> >>           address: 68:05:ca:1b:15:f8
> >>           media: Ethernet autoselect (1000baseT
> >> full-duplex,flowcontrol,master,rxpause,txpause)
> >>           status: active
> >>           inet 192.168.1.182/24 broadcast 192.168.1.255 flags 0x0
> >>           inet6 fe80::638a:f74a:53da:36bb%wm0/64 flags 0x0 scopeid 0x1
> >>           inet6 2600:6c5e:1d00:56de:7c0e:e015:3a9a:339c/64 flags 0x0
> >> wm1: flags=0x8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
> >>           capabilities=7ff80<TSO4,IP4CSUM_Rx,IP4CSUM_Tx,TCP4CSUM_Rx>
> >>           capabilities=7ff80<TCP4CSUM_Tx,UDP4CSUM_Rx,UDP4CSUM_Tx,TCP6CSUM_Rx>
> >>           capabilities=7ff80<TCP6CSUM_Tx,UDP6CSUM_Rx,UDP6CSUM_Tx,TSO6>
> >>           enabled=0
> >>           ec_capabilities=17<VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,EEE>
> >>           ec_enabled=2<VLAN_HWTAGGING>
> >>           address: 60:45:cb:71:10:be
> >>           media: Ethernet autoselect (1000baseT
> >> full-duplex,flowcontrol,master,rxpause,txpause)
> >>           status: active
> >>           inet 192.168.1.186/24 broadcast 192.168.1.255 flags 0x0
> >>           inet6 fe80::6245:cbff:fe71:10be%wm1/64 flags 0x0 scopeid 0x2
> >>           inet6 2600:6c5e:1d00:56de:b25c:5f71:1a4b:ea17/64 flags 0x0
> >> lo0: flags=0x8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 33624
> >>           inet 127.0.0.1/8 flags 0x0
> >>           inet6 ::1/128 flags 0x20<NODAD>
> >>           inet6 fe80::1%lo0/64 flags 0x0 scopeid 0x3
> >>
> >> ~> dmesg | grep wm0
> >> [     1.050563] wm0 at pci2 dev 0 function 0: Intel i82574L (rev. 0x00)
> >> [     1.050563] wm0: for TX and RX interrupting at msix2 vec 0 affinity to 1
> >> [     1.050563] wm0: for TX and RX interrupting at msix2 vec 1 affinity to 2
> >> [     1.050563] wm0: for LINK interrupting at msix2 vec 2
> >> [     1.050563] wm0: PCI-Express bus
> >> [     1.050563] wm0: 2048 words FLASH, version 1.8.0, Image Unique ID
> >> 0000ffff
> >> [     1.050563] wm0: ASPM L0s and L1 are disabled to workaround the errata.
> >> [     1.050563] wm0: Ethernet address 68:05:ca:1b:15:f8
> >> [     1.050563] wm0: 0x224480<FLASH,IOH_VALID,PCIE,ASF_FIRM,WOL>
> >> [     1.050563] makphy0 at wm0 phy 1: Marvell 88E1149 Gigabit PHY, rev. 1
> >>
> >> ~> dmesg | grep wm1
> >> [     1.050563] wm1 at pci0 dev 31 function 6: I219 V Ethernet
> >> Connection (rev. 0x00)
> >> [     1.050563] wm1: interrupting at msi4 vec 0
> >> [     1.050563] wm1: PCI-Express bus
> >> [     1.050563] wm1: 4096 words FLASH, version 0.2.4
> >> [     1.050563] wm1: Ethernet address 60:45:cb:71:10:be
> >> [     1.050563] wm1: 0x6a4080<FLASH,PCIE,ASF_FIRM,AMT,WOL,EEE>
> >> [     1.050563] ihphy0 at wm1 phy 2: i217 10/100/1000 media interface,
> >> rev. 6
> >>
> >>
> >>
> >> --
> >> Ron Georgia
> >> "There seems to be a scratch in the prism of my understanding."
> >
> > 1. Where does it say in the docs that each interface should have an
> > ifconfig file?
> >   -> If so, why are you assuming both interfaces should be configured up?
> >
> > 2. What are you trying to accomplish by plugging in both interfaces?
> >
> > 3. Why are you assuming that this should work?
> >
> > dhcpcd probably should not crash but you're probably trying to ask it
> > to do something nonsensical.
> >
> > Andy
> >
>
> --
> Ron Georgia
> "There seems to be a scratch in the prism of my understanding."
References:
- Network Oddities
  - From: Ron Georgia
- Re: Network Oddities
  - From: Andy Ruhl
- Re: Network Oddities
  - From: Ron Georgia
Prev by Date: Re: Network Oddities
Next by Date: Re: Network Oddities
Previous by Thread: Re: Network Oddities
Next by Thread: Re: Network Oddities
Indexes:
Home | Main Index | Thread Index | Old Index