Subject: Google SoC 2007 Project: Subsystem Conditional Building (resend)
To: None <tech-userlevel@netbsd.org>
From: Brian A. Seklecki <lavalamp@spiritual-machines.org>
List: tech-userlevel
Date: 03/19/2007 20:47:08
[some format/typos corrections from pasting into MUA corrected]
Note: This is strictly an informal discussion; not a full proposal at this
point.
Some of you may know that I'm starting a small project called 'bsd-appliance'
to hopefully help centralize and coordinated embedded technology related
development for the *BSD families (a la linuxdevices.com). I've released a
small set of tools (framework/toolkit) to help build and assemble dynamic file
systems and bootable file-system images following a strategy we developed at
Collaborative Fusion, Inc.
As an extension of that project, I'd like to propose this SoC project as
(hopefully) a catalyst for additional projects.
Background:
One of the topics that the framework does not address is _what_ file system
objects to include in such images (we just say 'pull from this $DESTDIR'). As
you know, many resource-constricted embedded platforms rely on translating
archived read-only userland data on CF media into memory-backed filesystems
using mfs(4) and md(4)/rd(4).
Profiling the system is really beyond the mandate of that framework. It is
also very implementation-specific, so the project only offers advice on how to
do so. However, every time I give someone the advice: "Just extract base.tgz
and etc.tgz and create a list of binaries and libraries relative to / and /usr
in rd_prune.txt and cf_prune.txt respectively to prune automatically", I cringe
because I know that this is the least graceful method of accomplishing this
goal.
Proposal:
I propose, a interim solution to syspkg, making a number of additional
subsystems conditionally built using the mk.conf(5) variable method. This
approach, although obviously much less clean and efficient than a the syspkg
approach, is beneficial in many ways none the less. For starters, it begins to
identify subsystems, their respective $DESTDIR and source tree objects, and
their sizes.
Conversely, the lack of scalability of this approach in the long-term becomes
apparent based on the need, for each system conditionalized, to touch so many
other systems related to the build.
For each subsystem, I will follow the examples in the already conditionalized
subsystems (list below). The changes frequently reach into:
- Makefile $SUBDIR conditions
- etc/${subsys_name} Entries
- etc/mtree/set.* Entries
- etc/rc.d/ Entries
- postinstall(8) checks
- usr/share/examples/* contents
- usr/share/doc/* contents
- usr/share/man/whatis*db Entries
- Further categorizing in distrib/sets/lists/*/mi
- Further categorizing in distrib/syspkg/sets/
Already conditional: GNU CVS, GCC, GDB, ipf(4) / ipfilter, kerberos, lint, pam,
OpenBSD pf(4), Postfix, S/Key, YP, Hesoid.
These are a great start. For this project, I've earmarked the following
additional subsystems:
tcpdump(8) / libpcap, GNU diff(1)/patch(1), ISC NTP, ISC DHCP, RAIDFrame
(userland), LFS (userland), systrace (userland), ISDN4BSD, CGD (userland), CCD
(userland), AltQ (userland), WPA Supplicant (userland), HostAP, Racoon, PPP /
SLIP / PPPOE, LP Utilies, SUP, Kerberos (userland), NFS (userland), Quota
Utilities (userland), AMD, OpenSSH, iSCSI/libiscsi (userland), libradius (pam),
libevent, libprop
Below is a cursory summary of the most of these. Of course, the reality
if conditionalizing these is likely to be a series of interdependencies
that only the userland maintainer knows of.
The sum of the space savings of the below systems (by no means comprehensive)
approaches 8mb (~7382748 K). At first glance, this project proposal may seem
like an awful lot of work to save 8mb in $DESTDIR, however when working on
embedded hardware platforms with often cumbersome memory limitations (32, 64,
128), 8mb can make a world of difference.
Additionally hopefully we can create a precedent for modularizing additional
features that require new userland utilities and libraries.
~BAS
---------------------------------------------------------
Subsystems Overview:
---------------------------------------------------------
Subsystem Name: tcpdump(8)/pcap(3)/libpcap
Userland: Yes
Kernel: Yes (not affected)
Source tree objects:
usr.sbin/tcpdump/
dist/tcpdump
dist/lib/libpcap
File system objects:
/usr/lib/libpcap.a
/usr/lib/libpcap.so
/usr/lib/libpcap.so.2
/usr/lib/libpcap.so.2.0
/usr/lib/libpcap_p.a
/usr/lib/libpcap_pic.a
/usr/sbin/tcpdump
Estimated size K (i386): 1195235
Subsystem Name: raid(4)
Userland: Yes
Kernel: Yes (Not related)
Source tree objects:
sbin/raidctl/
File system objects:
/sbin/raidctl
Estimated size K (i386): 24892
Subsystem Name: isdn(4)
Userland: Yes
Kernel: Yes (no related)
Source tree objects:
usr.sbin/isdn
share/examples/isdn/
share/man/man4/isdn*
File system objects:
/usr/sbin/isdnd
/usr/sbin/isdnmonitor
/usr/sbin/isdntel
/usr/sbin/isdntelctl
/usr/sbin/isdntrace
Estimated Size in K (i386): 295032
Subsystem Name: LFS
Userland: Yes
Kernel: Yes (not affected)
Source tree objects:
usr.sbin/dumplfs
sbin/mount_lfs
sbin/newfs_lfs
sbin/resize_lfs
libexec/lfs_cleanerd
File system objects:
/libexec/lfs_cleanerd
/sbin/mount_lfs
/sbin/newfs_lfs
/sbin/resize_lfs
/usr/sbin/dumplfs
Estimated Size in K (i386): 148934
Subsystem Name: systrace(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
bin/systrace
File system objects:
/bin/systrace
Estimated Size in K (i386): 151671
Subsystem Name: cgd(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
sbin/cgdconfig
share/man/man4/cgd.4
File system objects:
/sbin/cgdconfig
Estimated Size in K (i386): 41490
Subsystem Name: ccd(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
sbin/ccdconfig
share/man/man4/ccd.4
File system objects:
/sbin/ccdconfig
Estimated Size in K (i386): 15513
Subsystem Name: altq(9)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
usr.sbin/atlq
File system objects:
/usr/sbin/altqd
/usr/sbin/altqstat
Estimated Size in K (i386): 114200
Subsystem Name: ISC DHCP
Userland: Yes
Kernel: No
Source tree objects:
dist/dhcp
usr.sbin/dhcp
File system objects:
/sbin/dhclient
/sbin/dhclient-script
/usr/sbin/dhcpd
/usr/sbin/dhcrelay
/usr/share/man/cat5/dhcp-eval.0
/usr/share/man/cat5/dhcp-options.0
/usr/share/man/cat5/dhcpd.conf.0
/usr/share/man/cat5/dhcpd.leases.0
/usr/share/man/cat8/dhcpd.0
/usr/share/man/man5/dhcp-eval.5
/usr/share/man/man5/dhcp-options.5
/usr/share/man/man5/dhcpd.conf.5
/usr/share/man/man5/dhcpd.leases.5
/usr/share/man/man8/dhcpd.8
Estimated Size in K (i386): 1431563
Subsystem Name: ISC NTP
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
dist/ntp
File system objects:
/usr/sbin/ntp-keygen
/usr/sbin/ntpd
/usr/sbin/ntpdate
/usr/sbin/ntpdc
/usr/sbin/ntpq
/usr/sbin/ntptime
/usr/sbin/ntptrace
/usr/sbin/sntp
Estimated Size in K (i386): 777795
Subsystem Name: wpa_supplicant(8)
Userland: Yes
Kernel: No
Source tree objects:
dist/wpa_supplicant
File system objects:
/usr/sbin/wpa_cli
/usr/sbin/wpa_passphrase
/usr/sbin/wpa_supplicant
Estimated Size in K (i386): 715922
Subsystem Name: hostapd(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/wpa/hostapd
usr.sbin/wpa/hostapd_cli
share/examples/hostapd
File system objects:
/usr/sbin/hostapd
/usr/sbin/hostapd_cli
Estimated Size in K (i386): 189744
Subsystem Name: ipsec-tools / racoon(8)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
crypto/dist/ipsec-tools
share/examples/racoon
usr.sbin/racoon
usr.sbin/racoonctl
File system objects:
/usr/sbin/racoon
/usr/sbin/racoonctl
Estimated Size in K (i386): 563791
Subsystem Name: sl(4)/ppp(4)/pppoe(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
sbin/slattach
sbin/pppoectl
usr.sbin/sliplogin
usr.sbin/pppd
dist/pppd
share/man/man4/*ppp*
File system objects:
/sbin/pppoectl
/sbin/slattach
/usr/sbin/pppd
/usr/sbin/pppdump
/usr/sbin/pppstats
/usr/sbin/sliplogin
/usr/sbin/slstats
Estimated Size in K (i386): 346654
Subsystem Name: lp(4) and friends
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/lpr
usr.sbin/lpctl
File system objects:
/usr/bin/lp
/usr/bin/lpq
/usr/bin/lpr
/usr/bin/lprm
/usr/bin/pr
/usr/sbin/lpc
/usr/sbin/lpd
/usr/sbin/lptest
Estimated Size in K (i386): 216203
Subsystem Name: sup(8) / supservers(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/sup/
share/examples/supfiles
File system objects:
/usr/sbin/sup
/usr/sbin/supfilesrv
/usr/sbin/supscan
/usr/share/examples/supfiles/sup.jp.netbsd.org
/usr/share/examples/supfiles/sup.netbsd.org
/usr/share/examples/supfiles/sup2.fr.netbsd.org
Estimated Size in K (i386): 151497
Subsystem Name: quota(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/quotaon
usr.sbin/repquota
usr.sbin/quotacheck
usr.sbin/edquota
usr.bin/quota
libexec/rpc.rquotad/
File system objects:
/usr/libexec/rpc.rquotad
/usr/sbin/edquota
/usr/sbin/quotacheck
/usr/sbin/quotaoff
/usr/sbin/quotaon
/usr/sbin/repquota
Estimated Size in K (i386): 90902
Subsystem Name: amd(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/amd
share/examples/amd
File system objects:
/usr/sbin/amd
Estimated Size in K (i386): 146519
Subsystem Name: amd(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/amd
share/examples/amd
File system objects:
/usr/sbin/amd
Estimated Size in K (i386): 146519
Subsystem Name: iscsi-target(8) / libiscsi
Userland: Yes
Kernel: Yes (Not affected)
Source tree objects:
dist/iscsi/
lib/libiscsi
usr.sbin/iscsi
File system objects:
/usr/lib/libiscsi.a
/usr/lib/libiscsi.so
/usr/lib/libiscsi.so.0
/usr/lib/libiscsi.so.0.0
/usr/lib/libiscsi_p.a
/usr/lib/libiscsi_pic.a
/usr/sbin/iscsi-target
Estimated Size in K (i386): 618672
7~
l8*
-lava (Brian A. Seklecki - Pittsburgh, PA, USA)
http://www.spiritual-machines.org/