On Sun, Mar 07, 2004 at 11:38:10PM -0500, Matthew Orgass wrote:
> On 2004-03-07 tlaronde@polynum.com wrote:
> 
> > Is there a practical way to segregate part I from part II, that is to
> > compile the "hardware bootstrapping", some drivers, hardware
> > configuration in order to create a primitive kernel used as a bootloader
> > (meaning the ability to use NetBSD sources to compile code to put in
> > network cards EPROM, and the ability to achieve more than is actually
> > done with other bootloaders)?
> 
>   How big is the boot ROM?  

My wording was not clear. The goal is to be able to achieve this stuff
with using as much as possible of the source tree without adding a lot
of derivation and so. If such a scheme (being able with a reasonable
amount of additionnal code to generate boot + driver +config) is
achievable, this means it could be used, too, to generate boot + 1
driver (net) + UDP/IP (DHCP discover, tftp retrieve) that is to generate
code that could be used in a network card EEPROM.

Say it will be a side effect ;) even if it is of no use for me at the
moment.

>[..] 
>   I have not investigated in detail what all the non-driver code is used
> for, but uvm and vfs are two big sections.  I don't know how much they can
> be reduced, but most likely not too much.  The kernel-userland interface
> does not take up that much of the kernel.  You might be able to save 75k
> by removing it, but I think that is about it.

Well, if I refer to the red daemon book (I should have started by
this...), chapter 14 (System Startup), we find this (14.3):

	The initialization process is roughly divided into three stages. The
	first stage is written entirely in assembly language and does the
	work necessary for non-assembly-language code to operate. The second
	stage does machine-dependent operations, including the configuration
	and initialization of the I/O devices on the machine. The third
	stage does machine-independent operations, completing its work by
	starting up the system-resident processes that compose the
	foundation for the normal 4.4BSD runtime environment.

At first glance, it could seem that boostrap + stage 1 + machine
dependent operations is what I'm after. But it's not so simple, since
the machine-dependent initialization phase does things that are clearly
"too much" for a bootloader [parts of the cpu_startup()], mainly the
initialization of the system data structures.
	=> One of the possibilities (in the perspective of "use as much as
	the current code for the bootloader") would be to decrease
	significantly the memory used for these (by the bootloader) via
	proper configuration

	=> For i386, reusing the code means the bootloader will escape real
	mode rapidly, meaning that it will be unable to use the firmware
	(BIOS) if the virtual mode is not unabled. That's a pain by itself.
	And that's i386 too...


>   If you didn't catch it before, see the "A potential step toward
> modularisation" thread last month on this list for some of the LKM issues.
> I am not a fan of LKMs, but some of the options for device detection could
> also make it possible to automatically detect hardware for static kernel
> configuration without including all possible drivers.  I would prefer
> separate probe functions to do this which could be statically linked into
> the kernel or separately loaded and unloaded in bulk (per bus).  I don't
> know what the current issues are with NetBSD LKMs.

Thanks for the tip. I have read the messages in the thread but I need to
look more closely to the docs and code to understand clearly the issues.

One thing (theory) is clear is that in the bootloader there is only one
device that needs to be accessed: the one where it can have access to
supplementary resources. If and only if the bootloader can probe to
detect what is here, it can them load (by accessing the supplementary
resources) additionnal drivers.

In §14.4 Autoconfiguration:

	Autoconfiguration is the procedure carried out by the system to
	recognize and enable the hardware devices present in a system.
	Autoconfiguration works by systematically probing the possible I/O
	buses on the machine.[...]For each I/O bus that is found, the type
	of device attached to it is interpreted and, depending on this type,
	the necessary actions are taken to initialize and configure the
	device.

In the case I'm looking for the "necessary actions" will mean trying to
add/load the correct driver if it is not present and, at a minimum,
print the identifier of the hardware in order to be able to compile a
matching kernel.

Alternatively, since as written (p498, same section) "Devices that are
present but not recognized remain unavailable until the system is
rebooted", add (the file is present in FreeBSD if I'm not mistaken) the
possibility by the bootloader to write an autoconf file (hence, there
will be static configuration, and the dynamic autoconfiguration will be
the in kernel one + the mask provided by the autoconf file
(enable/disable) --- this is already what is made but by hand when
entering autoconf (boot -c)).

The advantage to put the hardware detection in a bootloader will be:

1) To create the correct description of the machine
2) To allow an interruptable probe of the hardware. Example:
	
	I've just purchased a laptop and installed NetBSD 1.6.2(release) on it.
	The stocked laptop kernel has APM. Problem, the laptop, as specified
	in the motherboard doc, implements only partly APM and when the
	kernel tries to initialize it it freezes. I have the obligation to
	make a cold reboot, enter interactive autoconf and disable APM. 

> 
> > 4) The users interact with the primitive kernel via the kernel shell
> > (autoconf prompt is something like that). And the "programs" are not
> > external ones but in kernel ones (this simplifies the ABI).
> 
>   This would work for a basic interface, but of you assume network
> availability anyway you might want to put this in userland on NFS (or just
> in MFS in the kernel if you have the space).  Crunched simple userland
> code doesn't take up that much space, and there is a lot of code that you
> might potentially want to load a system (unfortunately most of it not so
> simple).  Use of userland would make it easier to use much more code if
> you need it with less changes.

I think that if the kernel is stripped done to the strict minimum of
drivers, then one can use the full power of NetBSD in the minimal
place, that's why I think, I may be wrong, that the bootloader ability
must be spartan.

For NFS, since it involves bigger programs, is there the ability to
change the rootfs at runtime? I mean, if the kernel is loaded with a
minimal MFS which has NFS userland programs, is it possible to load the
NFSclient to access the ressources and mount the NFS accessed fs as the
root fs overriding the memory image?

Thanks a lot for your comments :)

And if it was not clear in my first mail, if I eventually make something
(over than the present noise) and if I'm not too ashamed about the code
written, it will be of course contributed back. But if people with more
knowledge find that this is totally useless and silly, I could perhaps
stop wasting your and my time...

Regards,
-- 
Thierry Laronde (Alceste) <tlaronde@polynum.org>
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C