Re: GPT, wedges and RAID-1 on new server with NetBSD 7

[Tobias Nygren <tnn%NetBSD.org@localhost>]

On Mon, 7 Mar 2016 00:47:55 +0000
Gerard Lally <gerard.lally%gmail.com@localhost> wrote:

> If it's not possible to do this with sysinst is it at least possible
> to do it by dropping to a shell? Ideally I would like to use GPT with
> the RAID-1 setup as well, since I will be on 2 x 2TB disks and I
> anticipate this getting bigger, not smaller in years to come. I have
> successfully set up NetBSD with GPT by dropping to a shell but I don't
> know where to add RAID into that mix.

It is possible but a bit tricky. The stage 2 boot blocks are capable of
reading "disklabel on raidframe on gpt", but not "gpt on raidframe on
gpt". It would be nice to get the latter working.

Here's how I did it, by having a tiny boot mirror and large root
mirror. This method is also applicable if you want to have the root
fs on a huge raid5, which also cannot be read by the boot blocks.

# set up disks

gpt destroy wd0
gpt destroy wd1
gpt create wd0
gpt create wd1
gpt add -b 64 -s 256M -l boot_raid_col0 -i 1 -t raid wd0 # dk0
gpt add -b 524352 -l root_raid_col0 -i 2 -t raid wd0     # dk1
gpt add -b 64 -s 256M -l boot_raid_col1 -i 1 -t raid wd1 # dk2
gpt add -b 524352 -l root_raid_col1 -i 2 -t raid wd1     # dk3

# set up raids

cat > /tmp/raid0.conf << EOF
...
START disks
/dev/dk0
/dev/dk2
...
EOF

cat > /tmp/raid1.conf << EOF
...
START disks
/dev/dk1
/dev/dk3
...
EOF

raidctl -C /tmp/raid0.conf raid0
raidctl -C /tmp/raid1.conf raid1
raidctl -I 1234 raid0
raidctl -I 5678 raid1
raidctl -P raid0
raidctl -P raid1
raidctl -A yes raid0
raidctl -A yes raid1

# set up labels on raids

disklabel -i -I -r raid0
W
Q
gpt create raid1
gpt add -t ffs -b 64 -i 1 raid1   # dk4

# create filesystems

newfs -O2 /dev/rraid0a
newfs -O2 /dev/rdk4

# install kernel and boot loader

mount -t ffs /dev/dk4 /targetroot
mkdir /targetroot/bootfs
mount -t ffs /dev/raid0a /targetroot/bootfs
cp /usr/mdec/boot /targetroot/bootfs
cp netbsd /targetroot/bootfs
installboot /dev/rdk0 /usr/mdec/bootxx_ffsv2
installboot /dev/rdk1 /usr/mdec/bootxx_ffsv2
gpt biosboot -i 1 wd0
gpt biosboot -i 1 wd1

Now install your system on /targetroot.

Caveats:

sysinst will not understand how to install on this layout, so you'll
have to unpack the sets and do the installation steps manually.
(Unpack sets, copy /boot and /netbsd in place, run MAKEDEV)

Also, the kernel won't be able to autodetect the root device to use, so
you'll have to force it to dk4.

Alternatively, you could opt to make your raid0a a bit larger, put the
entire root file system on there and use dk4 as the /usr partition.
Then you can use "raidctl -A root raid0" and the kernel will be able to
find the root device automatically.

Hope that helps.

Cheers,
-Tobias