[patch] improved multiboot support in kernel and boot(8)

[Grégoire Sutre <gregoire.sutre%gmail.com@localhost>]

Hi,

The multiboot code in the kernel and in boot(8) does not implement
several features of the multiboot specification, in particular:

- multiboot in boot(8):
  . the ELF section headers are not passed to the kernel, hence
    /dev/ksyms and kernel modules cannot work.
  . the booted device is not passed to the kernel, hence
    x86_autoconf cannot guess the root device.

- multiboot in kernel:
  . there is no support for kernel modules.

I've been working on a patch to implement these features as well as some
others.  The patch is available here:

netbsd-5: http://pastebin.com/m6be0b13b
-current: http://pastebin.com/m21ff1ff2

I tested this in qemu and on real i386 hardware, both with netbsd-5 and
-current.  Obviously, I welcome your comments, test reports, etc. :-)

The rest of this email gives an overview of the changes, which are
mostly in the multiboot-specific parts of the sources, but in some cases
it seemed better to change files that are not multiboot-specific.

Multiboot in boot(8)
--------------------
This concerns sys/arch/i386/stand/lib/exec.c (function exec_multiboot).

A valid multiboot header is now searched in the first 8192 bytes of the
selected file, and booting is aborted if there is none.

The multiboot information structure (MBI) passed to the booted kernel
now contains the following additional information: booted device, ELF
section headers, memory map, BIOS drives, and loader name.  This
information was already available in the bootinfo data structures.  A
few files in sys/arch/i386/stand/lib have been modified in order to
directly access the bootinfo information.

To properly set the boot_device field of the MBI, the DOS partition
number is now stored in the biosdisk structure in biosdisk.c.

The file sys/lib/libsa/loadfile_elf32.c has also been slightly changed
to set the sh_addr fields of the loaded section headers to the correct
absolute physical addresses.  This makes the sh_addr fields consistent
with the sh_offset ones (which are used when booting with the native
boot protocol).

Bootinfo console options are now automatically added on the multiboot
command line (to mimic the behavior of the boot command).


Multiboot in kernel
-------------------
The current code (src/sys/arch/i386/i386/multiboot.c) copies the symbol
and string tables after the kernel image in early boot, and uses a
number of global variables to safely copy the rest of the MBI.  These
two tables may be placed anywhere in memory, for instance one of them
could already be after the end of the kernel image.  So the code checks
the placement of these tables and appends them to the kernel image in
the right order.

The situation is more complex with modules as we may have a bunch of
them.  Thus, a generic approach for moving chunks of data has been
implemented in the patch, and this scheme is used to move the whole MBI
(including modules, symbol and string tables, memory map, etc.) after
the kernel image.  The implemented scheme takes alignment constraints
into account.

Multiboot command-line processing in the kernel has also been modified.
This topic has been discussed in [1], and I believe that the solution
implemented in this patch accounts for the concerns expressed there.  A
preparsing procedure now locates the first argument and the rest of the
command-line.  The boot path is extracted from the first argument, and
the rest is used as options.  In particular, boothowto switches and
options of the form key=val are considered to be of the same nature,
hence they can be in any order.  For instance the following command-line
is recognized (in that case, the bootpath is set to /netbsd.gz):

hd1a:/netbsd.gz console=com -z root=wd0a -c

The function optstr_get(9) now uses the last value of a key if that key
occurs several times in the passed string.  This is required to override
console options set automatically by the boot loader (as mentioned above).

There are also other small changes (e.g. multiboot_print_info uses
aprint_* functions and prints the full multiboot information in debug
mode).

Since the file src/sys/arch/i386/i386/multiboot.c was heavily modified,
the patch may be difficult to read.  A copy of the new file is at:

netbsd-5: http://pastebin.com/m355f7b6f
-current: http://pastebin.com/m4294c5aa

Thanks for reading all of this! :-)

Grégoire

p.s. This patch obsoletes the ones that I previously sent in PRs #42599
     and #42640.

[1] http://mail-index.netbsd.org/port-i386/2010/01/09/msg001747.html