Re: kernel frameworks for jumbo frame

[Jason Thorpe <thorpej%me.com@localhost>]

> On Jan 29, 2019, at 6:57 AM, Rin Okuyama <rokuyama%rk.phys.keio.ac.jp@localhost> wrote:
> 
> Hi,
> 
> I'm working on jumbo frame support for axen(4) and mue(4). However,
> unfortunately, I find that we do not have proper kernel frameworks
> for jumbo frame; there are two problems:
> 
> (1)
> We do not have external mbuf cluster capable for receiving jumbo
> frames. We need to use m_devget(9) [ale(4), sk(4), ...], or have
> per driver pool [bge(4), dge(4), ...]. The former has a performance
> problem, whereas the latter makes drivers complicated.
> 
> FreeBSD has 4 cluster sizes, MCLBYTES, pagesize, 9KiB, and 16KiB.
> They are usable via m_getjcl(how, type, flags, size):
> 
> https://www.freebsd.org/cgi/man.cgi?query=m_getjcl
> 
> Here, the "size" argument should be chosen from 4 values above
> (description in man page is misleading).
> 
> OpenBSD has 8 cluster sizes, MCLBYTES, MCLBYTES + 2 (ether aligned
> 2KiB buffer), 4, 8, 9, 12, 16, and 64KiB. They are usable via
> MCLGETI(m, how, ifp, len):
> 
> http://man.openbsd.org/MCLGETI
> 
> Unlike FreeBSD, the "len" argument can be arbitrary; proper cluster
> is chosen automatically. The "ifp" argument is not used.

OpenBSD's approach seems a little better in this case, but both FreeBSD and OpenBSD have terrible names for the API.  I would suggest something like:

	int	mbuf_get_cluster(struct mbuf *m, size_t desired_size);

Returns an errno, takes the desired size, figures out the best fit.  If there is nothing available at the desired size, ENOMEM is returned and the original mbuf is unmodified.  Make sure there is a function that gets the actual length.  I don't think it's worth having a 2KiB+2 bucket.

Probably worth having a call that returns the maximum cluster size available:

	size_t mbuf_max_cluster_size(void);

> (2)
> We do not support maximum MTU other than 1500 or 9000 bytes. However,
> for example, ixg(4) supports max MTU of 16114, and it is restricted to
> 4088 for axen(4).
> 
> FreeBSD handles SIOCSIFMTU ioctl in each driver, whereas OpenBSD added
> if_hardmtu member to struct ifnet for this purpose.
> 
> 
> For (2), it seems better for me to switch from ETHERCAP_JUMBO_MTU flag
> to the OpenBSD way of if_hardmtu. What should we do for (1)?

Centralizing it is always better, with a driver-specific hook as needed.  We can already do that either e.g. ether_ioctl(), which allows the driver to trap SIOCSIFMTU itself if it wants to, before forwarding on to the generic code.  (Maybe it needs to poke a register to enable jumbo frames...)

But if_hardmtu is a bad name, and it seems like it's short-sighted in general to start adding stuff like this willy-nilly.  It think a better approach is to add support for generic interface properties, hidden behind an if_{get,set}_prop_<whatever> API.  Ethernet interfaces would get the standard 1500 ETHERMTU unless, for example, the "ethernet-max-mtu" property were set by the individual driver.

As an aside, I think we should probably change if_media work as properties, as well.

-- thorpej