On Mon, Mar 19, 2007 at 08:47:08PM -0400, Brian A. Seklecki wrote:
> [some format/typos corrections from pasting into MUA corrected]
> 
> Note:  This is strictly an informal discussion; not a full proposal at this 
> point.
> 
> Some of you may know that I'm starting a small project called 
> 'bsd-appliance' to hopefully help centralize and coordinated embedded 
> technology related development for the *BSD families (a la 
> linuxdevices.com).  I've released a small set of tools (framework/toolkit) 
> to help build and assemble dynamic file systems and bootable file-system 
> images following a strategy we developed at Collaborative Fusion, Inc.
> 
> As an extension of that project, I'd like to propose this SoC project as 
> (hopefully) a catalyst for additional projects.
> 
> Background:
> 
> One of the topics that the framework does not address is _what_ file system 
> objects to include in such images (we just say 'pull from this $DESTDIR').  
> As you know, many resource-constricted embedded platforms rely on 
> translating archived read-only userland data on CF media into memory-backed 
> filesystems using mfs(4) and md(4)/rd(4).
> 
> Profiling the system is really beyond the mandate of that framework.  It is 
> also very implementation-specific, so the project only offers advice on how 
> to do so.  However, every time I give someone the advice: "Just extract 
> base.tgz and etc.tgz and create a list of binaries and libraries relative 
> to / and /usr in rd_prune.txt and cf_prune.txt respectively to prune 
> automatically", I cringe because I know that this is the least graceful 
> method of accomplishing this goal.
> 
> Proposal:
> 
> I propose, a interim solution to syspkg, making a number of additional 
> subsystems conditionally built using the mk.conf(5) variable method. This 
> approach, although obviously much less clean and efficient than a the 
> syspkg approach, is beneficial in many ways none the less.  For starters, 
> it begins to identify subsystems, their respective $DESTDIR and source tree 
> objects, and their sizes.

Brian,

I have found in my work on a network appliance that NetBSD is already
modular through the use of syspkgs, however, several modules I desire
to put on networked appliances are way too big.

In CUWiNware, we use the combination of syspkgs and a "trim list"
to filter the NetBSD METALOG before doing an unprivileged "install"
and creating either an NFS root + kernel, a kernel w/ MD root, or an
FFS or ISO9660 image for CF or ISO9660, respectively.  syspkg helps
us filter the system with about the same granularity as you desire.
In more recent work, my colleague Bryan has introduced a new "crunched"
build using crunchgen(1).  Our goal is to fit a NetBSD router onto the
Meraki Mini's cramped flash (8MB - 1.5MB overhead = 6.5MB!).  [This is all
open source at <http://svn.cuwireless.net/svn/cuw/trunk/src/boot-image/>.
I'm sorry our server is so slow.]

We have found at CUWiN that even at the level of modularity you propose,
it is tremendously difficult to create a miniature NetBSD router.
Suppose we leave out these modules:

> tcpdump(8), GNU diff(1)/patch(1), ISC NTP, RAIDFrame 
> (userland), LFS (userland), systrace (userland), ISDN4BSD, CGD (userland), 
> CCD (userland), AltQ (userland), Racoon, LP Utilies, SUP, Kerberos (userland), NFS (userland), 
> Quota Utilities (userland), AMD, OpenSSH, iSCSI/libiscsi (userland)
> libradius (pam), 

Already I am a little uneasy, having left out OpenSSH, ALTQ, and TCPDump.
Note that by my omissions, I have ruled out such uses as embedded storage
appliance or embeddd print server.  For a router, we simply cannot leave
the following modules out---or, more abstractly, we cannot leave out
the capabilities they provide:

> libpcap, ISC DHCP, WPA Supplicant (userland), HostAP, 
> PPP / SLIP / PPPOE,
> libevent, libprop

At least three of these modules present a problem.  ISC DHCP is huge.
Last time I checked, wpa_supplicant and hostapd were inexplicably big,
too.  libpcap, which hostapd uses, is big, as well.

So, I would like to suggest that the problem of breaking NetBSD into
modules for embedded systems is handily solved by syspkg + CUWiNware
build scripts, while the problem of the modules each being too big to
combine usefully and "embed-ably" with others badly needs a solution.

Dave

-- 
David Young             OJC Technologies
dyoung@ojctech.com      Urbana, IL * (217) 278-3933