Subject: Google SoC 2007 Project: Subsystem Conditional Building
To: None <tech-userlevel@netbsd.org>
From: Brian A. Seklecki <lavalamp@spiritual-machines.org>
List: tech-userlevel
Date: 03/19/2007 20:36:16
Note: This is strictly an informal discussion; not a full proposal at
this point.
Some of you may know that I'm starting a small project called
'bsd-appliance' to hopefully help centralize and coordindate embedded
technology related development for the *BSD families (a la
linuxdevices.com). I've released a small set of tools (framework/toolkit)
to help build and assemble dynamic file systems and bootable file-system
images following a strategy we developed at Collaborative Fusion, Inc.
As an extension of that project, I'd like to propose this SoC project as
(hopefully) a catalyst for additional projects.
Background:
One of the topics that the framework does not address is _what_ file
system objects to include in such images (we just say 'pull from this
$DESTDIR'). As you know, many resource-constricted embedded platforms
rely on translating archived read-only userland data on CF media into
memory-backed filesystems using mfs(4) and md(4)/rd(4).
Profiling the system is really beyond the mandate of that framework. It
is also very implementation-specific, so the project only offers advice on
how to do so. However, every time I give someone the advice: "Just
extract base.tgz and etc.tgz and create a list of binaries and libraries
relative to / and /usr in rd_prune.txt and cf_prune.txt respectively to
prune automatically", I cringe because I know that this is the least
graceful method of accomplishing this goal.
Proposal:
I propose, a interim solution to syspkg, making a number of additional
subsystems conditionally built using the mk.conf(5) variable method.
This approach, although obviously much less clean and efficient than a the
syspkg approach, is beneficial in many ways none the less. For starters,
it begins to identify subsystems, their respective $DESTDIR and source
tree objects, and their sizes.
Conversley, the lack of scalability of this approach in the long-term
becomes apparent based on the need, for each system conditionalized, to
touch so many other systems related to the build.
For each subsystem, I will follow the examples in the already
conditionalized subsystems (list below). The changes frequently reach
into:
- Makefile $SUBDIR conditions
- etc/${subsys_name} Entries
- etc/mtree/set.* Entries
- etc/rc.d/ Entries
- postinstall(8) checks
- usr/share/examples/* contents
- usr/share/doc/* contents
- usr/share/man/whatis*db Entries
- Further categorizing in distrib/sets/lists/*/mi
- Further categorizing in distrib/syspkg/sets/
Already conditional: GNU CVS, GCC, GDB, ipf(4) / ipfilter, kerberos, lint,
pam, OpenBSD pf(4), Postfix, S/Key, YP, Hesoid.
These are a great start. For this project, I've earmarked the following
additional subsystems:
tcpdump(8) / libpcap, GNU diff(1)/patch(1), ISC NTP, ISC DHCP, RAIDFrame
(userland), LFS (userland), systrace (userland), ISDN4BSD, CGD (userland),
CCD (userland), AltQ (userland), WPA Supplicant (userland), HostAP,
Racoon, PPP / SLIP / PPPOE, LP Utilies, SUP, Kerberos (userland), NFS
(userland), Quota Utilities (userland), AMD, OpenSSH, iSCSI/libiscsi
(userland), libradius (pam), libevent, libprop
Below is a cursory summary of the most of these. Of course, the reality
if conditionalizing these is likely to be a series of interdepedencies
that only the userland maintainer knows of.
The sum of the space savings of the below systems (by no means
comprehensive) approaches 8mb (~7382748 K). At first glance, this project
proposal may seem like an awful lot of work to save 8mb in $DESTDIR,
however when working on embedded hardware platforms with often cumbersome
memory limitations (32, 64, 128), 8mb can make a world of difference.
Additionally hopefully we can create a precident for modularizing
additional features that require new userland utilities and libraries.
~BAS
---------------------------------------------------------
Subsystems Overview:
---------------------------------------------------------
Subsystem Name: tcpdump(8)/pcap(3)/libpcap
Userland: Yes
Kernel: Yes (not affected)
Source tree objects:
usr.sbin/tcpdump/
dist/tcpdump
dist/lib/libpcap
File system objects:
/usr/lib/libpcap.a
/usr/lib/libpcap.so
/usr/lib/libpcap.so.2
/usr/lib/libpcap.so.2.0
/usr/lib/libpcap_p.a
/usr/lib/libpcap_pic.a
/usr/sbin/tcpdump
Estimated size K (i386): 1195235
Subsystem Name: raid(4)
Userland: Yes
Kernel: Yes (Not related)
Source tree objects:
sbin/raidctl/
File system objects:
/sbin/raidctl
Estimated size K (i386): 24892
Subsystem Name: isdn(4)
Userland: Yes
Kernel: Yes (no related)
Source tree objects:
usr.sbin/isdn
share/examples/isdn/
share/man/man4/isdn*
File system objects:
/usr/sbin/isdnd
/usr/sbin/isdnmonitor
/usr/sbin/isdntel
/usr/sbin/isdntelctl
/usr/sbin/isdntrace
Estimated Size in K (i386): 295032
Subsystem Name: LFS
Userland: Yes
Kernel: Yes (not affected)
Source tree objects:
usr.sbin/dumplfs
sbin/mount_lfs
sbin/newfs_lfs
sbin/resize_lfs
libexec/lfs_cleanerd
File system objects:
/libexec/lfs_cleanerd
/sbin/mount_lfs
/sbin/newfs_lfs
/sbin/resize_lfs
/usr/sbin/dumplfs
Estimated Size in K (i386): 148934
Subsystem Name: systrace(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
bin/systrace
File system objects:
/bin/systrace
Estimated Size in K (i386): 151671
Subsystem Name: cgd(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
sbin/cgdconfig
share/man/man4/cgd.4
File system objects:
/sbin/cgdconfig
Estimated Size in K (i386): 41490
Subsystem Name: ccd(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
sbin/ccdconfig
share/man/man4/ccd.4
File system objects:
/sbin/ccdconfig
Estimated Size in K (i386): 15513
Subsystem Name: altq(9)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
usr.sbin/atlq
File system objects:
/usr/sbin/altqd
/usr/sbin/altqstat
Estimated Size in K (i386): 114200
Subsystem Name: ISC DHCP
Userland: Yes
Kernel: No
Source tree objects:
dist/dhcp
usr.sbin/dhcp
File system objects:
/sbin/dhclient
/sbin/dhclient-script
/usr/sbin/dhcpd
/usr/sbin/dhcrelay
/usr/share/man/cat5/dhcp-eval.0
/usr/share/man/cat5/dhcp-options.0
/usr/share/man/cat5/dhcpd.conf.0
/usr/share/man/cat5/dhcpd.leases.0
/usr/share/man/cat8/dhcpd.0
/usr/share/man/man5/dhcp-eval.5
/usr/share/man/man5/dhcp-options.5
/usr/share/man/man5/dhcpd.conf.5
/usr/share/man/man5/dhcpd.leases.5
/usr/share/man/man8/dhcpd.8
Estimated Size in K (i386): 1431563
Subsystem Name: ISC NTP
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
dist/ntp
File system objects:
/usr/sbin/ntp-keygen
/usr/sbin/ntpd
/usr/sbin/ntpdate
/usr/sbin/ntpdc
/usr/sbin/ntpq
/usr/sbin/ntptime
/usr/sbin/ntptrace
/usr/sbin/sntp
Estimated Size in K (i386): 777795
Subsystem Name: wpa_supplicant(8)
Userland: Yes
Kernel: No
Source tree objects:
dist/wpa_supplicant
File system objects:
/usr/sbin/wpa_cli
/usr/sbin/wpa_passphrase
/usr/sbin/wpa_supplicant
Estimated Size in K (i386): 715922
Subsystem Name: hostapd(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/wpa/hostapd
usr.sbin/wpa/hostapd_cli
share/examples/hostapd
File system objects:
/usr/sbin/hostapd
/usr/sbin/hostapd_cli
Estimated Size in K (i386): 189744
Subsystem Name: ipsec-tools / racoon(8)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
crypto/dist/ipsec-tools
share/examples/racoon
usr.sbin/racoon
usr.sbin/racoonctl
File system objects:
/usr/sbin/racoon
/usr/sbin/racoonctl
Estimated Size in K (i386): 563791
Subsystem Name: sl(4)/ppp(4)/pppoe(4)
Userland: Yes
Kernel: Yes (Not Affected)
Source tree objects:
sbin/slattach
sbin/pppoectl
usr.sbin/sliplogin
usr.sbin/pppd
dist/pppd
share/man/man4/*ppp*
File system objects:
/sbin/pppoectl
/sbin/slattach
/usr/sbin/pppd
/usr/sbin/pppdump
/usr/sbin/pppstats
/usr/sbin/sliplogin
/usr/sbin/slstats
Estimated Size in K (i386): 346654
Subsystem Name: lp(4) and friends
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/lpr
usr.sbin/lpctl
File system objects:
/usr/bin/lp
/usr/bin/lpq
/usr/bin/lpr
/usr/bin/lprm
/usr/bin/pr
/usr/sbin/lpc
/usr/sbin/lpd
/usr/sbin/lptest
Estimated Size in K (i386): 216203
Subsystem Name: sup(8) / supservers(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/sup/
share/examples/supfiles
File system objects:
/usr/sbin/sup
/usr/sbin/supfilesrv
/usr/sbin/supscan
/usr/share/examples/supfiles/sup.jp.netbsd.org
/usr/share/examples/supfiles/sup.netbsd.org
/usr/share/examples/supfiles/sup2.fr.netbsd.org
Estimated Size in K (i386): 151497
Subsystem Name: quota(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/quotaon
usr.sbin/repquota
usr.sbin/quotacheck
usr.sbin/edquota
usr.bin/quota
libexec/rpc.rquotad/
File system objects:
/usr/libexec/rpc.rquotad
/usr/sbin/edquota
/usr/sbin/quotacheck
/usr/sbin/quotaoff
/usr/sbin/quotaon
/usr/sbin/repquota
Estimated Size in K (i386): 90902
Subsystem Name: amd(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/amd
share/examples/amd
File system objects:
/usr/sbin/amd
Estimated Size in K (i386): 146519
Subsystem Name: amd(8)
Userland: Yes
Kernel: No
Source tree objects:
usr.sbin/amd
share/examples/amd
File system objects:
/usr/sbin/amd
Estimated Size in K (i386): 146519
Subsystem Name: iscsi-target(8) / libiscsi
Userland: Yes
Kernel: Yes (Not affected)
Source tree objects:
dist/iscsi/
lib/libiscsi
usr.sbin/iscsi
File system objects:
/usr/lib/libiscsi.a
/usr/lib/libiscsi.so
/usr/lib/libiscsi.so.0
/usr/lib/libiscsi.so.0.0
/usr/lib/libiscsi_p.a
/usr/lib/libiscsi_pic.a
/usr/sbin/iscsi-target
Estimated Size in K (i386): 618672
7~
l8*
-lava (Brian A. Seklecki - Pittsburgh, PA, USA)
http://www.spiritual-machines.org/