kern/58576: ifnet driver model is undocumented

[campbell+netbsd%mumble.net@localhost]

>Number:         58576
>Category:       kern
>Synopsis:       ifnet driver model is undocumented
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          doc-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Aug 10 22:35:00 +0000 2024
>Originator:     Taylor R Campbell
>Release:        current, 10, 9, ...
>Organization:
The ifnetbsd(9) Foundation
>Environment:
>Description:
The driver model for network interfaces is undocumented and non-obvious.
>How-To-Repeat:
$ man ifnet
man: no entry for ifnet in the manual.

>Fix:
Yes, please!

Here's a sketch that should be put into an ifnet(9) man page, similar to the usbnet(9) model we have (https://man.NetBSD.org/usbnet.9):

1. (under IFNET_LOCK) if_init:
   (a) resets hardware, while nothing else is touching registers
   (b) programs multicast filter
   (c) starts Tx/Rx and tick for mii/watchdog
   (d) sets IFF_RUNNING

2. concurrently:
   - (mii lock) periodic mii ticks or (with IFNET_LOCK too) ifmedia ioctls
   - (tx lock) Tx submissions
   - (rx lock) Rx notifications
   - (multicast filter lock) SIOCADDMULTI/SIOCDELMULTI
   - (ic lock) 802.11 state machine notifications
   - (IFNET_LOCK) maybe other ioctls (some of which return ENETRESET
     to cause an if_stop/init cycle in thread context)

3. (under IFNET_LOCK) if_stop:
   (a) clears IFF_RUNNING
   (b) halts Tx/Rx/tick and waits for completion
   (c) disables concurrent multicast updates
   (d) calls mii_down to wait for concurrent mii activity to quiesce
   (e) resets hardware, now that nothing is touching registers any more

The man page should also document all of the public struct ifnet::if_* members, like what the difference between if_start and if_init is (totally unrelated!), and what the difference between if_output and if_transmit is, and how if_percpuq works and where you're supposed to call bpf_mtap or if_deferred_start_init, and so on.