Re: booting from gpt/raid?

[Robert Elz <kre%munnari.OZ.AU@localhost>]

    Date:        Sun, 1 Jul 2018 16:55:05 -0400 (EDT)
    From:        mlh%goathill.org@localhost (MLH)
    Message-ID:  <20180701205505.AF9D712AAD%chopper.goathill.org@localhost>

  | Things are generally going well but before I actually install a
  | system, one question I have is how to make the configuration
  | bootable.

The first question to answer is whether you want a BIOS or UEFI
boot setup, you can do either with GPT.

The firmware in your system might only work with BIOS booting
(if it is a bit old) - if it supports UEFI it will probably also allow switching
to BIOS boot mode (most do these days,. eventually I'd suspect
that firmware makers will drop BIOS support.)

  | - GPT(8) says to make a bootable partition using gpt but doesn't
  |   mention doing that with raid

The raid part really makes no difference, it works the same way
for MBR and GPT booting (just different partition setup tools, and
so different commands to run.)

That said, I have a system configured very much like you describe
(it has more, bigger, drives, but that makes no difference) - and what
I'd do is make 3 (at least) GPT partitions on each drive (identical
sizes).   The first should be type EFI probably only needs to be about
10 - 20 MB (yes, megabytes) and will be used for UEFI booting, if
you ever set that up.   If you are going to use BIOS booting now,
you can just leave it empty, otherwise make a FAT filesystem on it
(wasting 20MB should not be a concern even if it is never used.)

The second partition (on each) should be big enough for a half dozen
(or so - depends how many older backup copies you want to keep)
netbsd kernels (which can be up to about 8MB each these days. so
make it at least 50MB - 128MB would be better - if you want to use
modules, rather than just embedding everything, you'll want space for
those as well, currently that's about 20MB per kernel version, so that
way I'd make this partition about 200MB.  This one should be type
ffs, and will be the boot parftition - set that one up for booting just
the way the NetBSD guide tells you to (regardless of whether you're
going to use MBR or UEFI booting).

All that should be there is /boot /boot.cfg and as many /netbsd* files
as you need (and /stand/*/*/modules if you want them - note only
modules to be loaded at boot time are actually needed there, the
modules will need to be duplicated on the root filesystem in order
to load them after booting, including via module autoload.)

Then the third should be whatever space is left (399G  and a bit
I guess) should have a PT type for NetBSD raid.   On that you
put either another GPT partition table, or a disklabel, to divide
the raid up however you want it partitioned (which can be as just
one big chunk if you like - it still should have some kind of label.)
Configure the raid with
	raidctl -A forceroot raidN
so it becomes auto configured, and will be the root filesystem
(even though you will have booted from the other partition).
(softroot might work, that's newer than my setup, so I have
never tried it.)

Then simply put a complete netbsd installation into the raid
partition(s). (root /usr /var /home .... either all on the one
partition, or on partitions of their own, whatever you prefer.)

When you boot, the MBR or UEFI boot will load the kernel from
the ffs (2nd) GPT partition, when that kernel starts, it will notice
the raid partitions (so they do need to be the correct type, in the
GPT table, or in the disklabel if using MBR) and also notice that
they are to be autoconfigured, so do that, and then move the root
filesystem there (the boot partition, where /netbsd came from,
will just be forgotten for now.)

You can mount the boot partitions (one from each drive)
(and the EFI partitions if you are actually using them) if you
want (I mostly keep mine unmounted, and mount them only
when I want to install a new kernel).

When I do that, I typically put the new kernel only on the one
that is actually used for booting (the drive the firmware is
configured to attempt to boot from first) and leave the other
untouched.  That way if I somehow botch things, I can always
just tell the firmware to swap the order of the boot drives,
and the older system just boots.  When I am happy it all
works, I just rsync the updated boot partition to the other one
(essentially mirroring manually).  This is usually only needed
rarely...

This all works, and is easy to configure, and almost impossible
to get wrong - the raidframe autoroot mechanism makes almost
all of the issues that people otherwise have simply vanish.

  | - NetBSD Raid guide says to install the bootloader onto one
  |   (preferably both) of the raid units but isn't using gpt on large
  |   disks.

Definitely put it all on both, if you don't, the redundancy you
get from the raid filesystems will be wasted, as the drive that
dies will be the one that you have setup to boot from (the other
one will have all of the data just fine, but without a way to
boot to get to it).

Aside you aren't using gpt on large disks either, you're
using gpt on small ones - but either way the MBR/GPT
difference really doesn't matter here, the mechanism to
make a partition bootable differs, but not in the way it
interacts with raidframe.

That said, as above, I'd avoid the raidframe boot hackery
and just use separate (manually sync'd) boot partitions.

  | using-large-disks apparently uses a separate boot drive

This would be relevant only if you really had large disks, and
an old (pre UEFI) firmware system which cannot boot from
anything bigger than 2T (because the MBR runs out of gas
there, as do disklabels - at least they do on drives that
advertise 512byte sectors).   On a system like that, using
a drive small enough for the firmware to cope with can be
the easy way to avoid issues.    But 400G is not going to
cause that kind of problem (unless, I suppose, your firmware
is so old that it only supports CHS addressing, and not
LBA - but if you have a still working system like that, you
should probably find a museum and donate it.)

kre