Since cgd allowed us to forward this message, I'll move this discussion
to tech-kern.

------- Forwarded Message

To: Michael Richardson <mcr@sandelman.ottawa.on.ca>
Subject: Re: PR#4094
References: <200008081650.MAA02161@pzero.sandelman.ottawa.on.ca>
From: cgd@netbsd.org (Chris G. Demetriou)
Date: 09 Aug 2000 23:18:52 -0700
In-Reply-To: Michael Richardson's message of "Tue, 08 Aug 2000 12:50:52 -0400"
Message-ID: <87k8dp63lf.fsf@mail.netbsd.org>

Michael Richardson <mcr@sandelman.ottawa.on.ca> writes:
>   A very frustrating thing for me is that installation of our (for
> several different definitions of "our") device drivers often requires that we
> modify files like conf.c, etc. in a build tree.

Are there more files than that, which typically cause you problems?

Things which apparently cause people doing NetBSD development
problems:

	malloc types (M_*, kmemstats)

	file systems (MOUNT_*, mountcompatnames -- wtf do people keep
	adding crap to this anyway?!, it looks like at least the
	collection of a list of initialization is done by config but
	that could be done better in my opinion)

	devices -- cdevsw, bdevsw, chrtoblk (and a corresponding
	blktochr might be nice 8-), devnametobdevmaj.

(asking people "what problems did you have using NetBSD for ..." is an
especially fun interview question for people who put NetBSD
development experience on their resume, esp. if they know who they're
talking to. 8-)

the devices thing is extra special, actually: there's the
[bc]dev.*_{init,decl} crap that is scattered all over when there's
really no reason it has to be that way, there's always the fact that
new devices are added and nobody bothers add them to conf.c files (and
that when they do get added the chartoblktbl is always screwed up),
and i've even seen stuff like mem_no screwed up.

If there are more sticking points, i'd love to hear more about them
(and then we can see about making them better 8-).


> For revision control reasons,
> I'd prefer to be able to give customers a NetBSD source CD, tell them to
> mount it as /usr/src, and then using -b and -s options to config, they
> build our source code in a seperate directory.

"yes."  (that is, after all, much of the point. 8-)


>   PR #4094 was submitted by me some time ago. Please take a look.
>  
>   http://www.NetBSD.org/cgi-bin/query-pr-single.pl?number=4094 
>
>   The trick is easy to hacking conf.c. Just have it include a file in two
> places, controlled by an #ifdef, and put come padding in the character
> device table so that one can track minor releases (and even -current) without
> having to mknod again each time.

i'd say that this is the wrong solution, since it's fairly fragile,
makes the maintenance problem worse (hey, new port, new magic you
need), and attempts to dodge many of the related problems which should
be solved by a sane solution.


I think there is a reasonable solution to this issue (which happens to
be inspired by some stuff in FreeBSD, but i've not actually directly
studied their bits, only seen the outlines).

anyway, it goes something like this:

every module (where module is "group of .c files which are related,"
not necessarily single source file) defines a structure (once, in one
of the .c files) describing something it thinks belongs in a global
table, and declares it (extern) in headers as appropriate.

a pointer to the structure is placed in a global list of like
structure pointers, which is run through at kernel init time to set up
lists and tables as appropriate.


something like:

struct devdesc {
	TAILQ_ENTRY(devdesc) link;
	const char *name; /* useful for root device stuff, at least */
	int bdevnum, cdevnum;
	struct bdevsw *bdev;
	struct cdevsw *cdev;
}

then something like:

	static struct devdesc sd_dd = {
		{}, "sd", BDEV_MAJOR_SD, CDEV_MAJOR_SD, &..., &...
	};
	ADD_DEVDESC(&sd_dd);

where [BC]DEV_MAJOR_* are defined by an MD header file which can be
included by drivers.  ADD_DEVDESC() would add the pointer to a
ctor/dtor-like list which would be processed at kernel init time using
a simple loop.  (This is doable without difficulty on both a.out and
ELF, and simply requires a file to be added each as first and last file in
the kernel link.  With a bit of care, a hack could even be done for
new ports which don't use GNU tools or ELF to have the lists generated
by the kernel build Makefile rather than the toolchain.)

for malloc types if i recall my investigation correctly the new types
could just be added to a linked list.  for file systems, the list
could be used in place of the config-generated list, removing a bit of
cruft from config.

for devices, at least without lots of kernel mods, the easiest thing
to do is dynamically allocate (and potentially reallocate) kernel
bdevsw and cdevsw tables.  mem_no and related functions could be
turned into simple macros defined by the MD header.  chartoblk and
other infrequently used functions could be handled two ways: create a
cross-reference table to provide the lookup, or just traverse the list.

this also means that devices who don't have conf.c-equivalent entries
(they become #defines in the header) get flagged as compile-time
errors.  it also means that the kernel could be probed to find major
and minor number information for devices, if desired.  finally, it's a
step in the right direction for a devfs.

your problem has a solution in this scheme, as well:  in a custom
header, include the MD device major decl header.  have each port
define a "maximum" number that it knows about, and do something like
(extra lines elided)

#define	MY_CDEV_BASE	100
#if MAX_CDEV >= MY_CDEV_BASE
#error those hogs have done it again!
#endif
#define CDEV_MAJOR_MYDEV1	(MY_CDEV_BASE + 0)
#define	CDEV_MAJOR_MYDEV2	(MY_CDEV_BASE + 1)

and then use your header in your drivers.  simple.  it may waste a bit
of space in your dynamically allocated tables, but i believe that's
probably worth the convenience for you (and, best of all, you can
trade convenience for space by tuning your base).


This discussion belongs on tech-kern.  Feel free to forward this
message in its entirety there. I don't have too much time to
participate in the discussion, but will certainly attempt to read any
threads that pop up and comment.

Various details should be ironed out, e.g. extensibility, exact
structure contents, how to handle versioning, etc., and of course
somebody has to code the diffs and will undoubtedly break some eggs
while implementing it.  However, i believe those are simply details
and the basic concepts are the right ones.  I'm sure that if people
don't let quibbling over the exact details get in the way of doing the
right thing, the right thing can happen.  8-)


chris
- -- 
Chris Demetriou - cgd@netbsd.org - http://www.netbsd.org/People/Pages/cgd.html
Disclaimer: Not speaking for NetBSD, just expressing my own opinion.

------- End of Forwarded Message