[HOW-TO] enlarge a RAIDFrame RAID-1

[Emmanuel Dreyfus <manu%netbsd.org@localhost>]

Sometime one has to resize a filesystem. For FFS, we have resize_fss(8),
but how to proceed if the FFS filesystem is inside a RAID-1 RAIDframe set?
Here you will find some procedures that I sucessfully tested. I just post
it here for future reference, but it would be nice if I (or someone else)
finds time to write a documentation web page on the topic.

My use case is a RAID-1 RAIDframe set of two 6 TB hard disks that were
formatted the old way, using MBR and disklabel. This only enables using
the first 2 TB of the disks, and migrating to GPT is required to use the
full 6 TB. We will also need to enlarge the RAID, a tricky operation 
that is documented here.

Our setup has two disks, wd0 and wd1, and raid0 is a RAID-1 set based on
wd0a and wd1a. Here is what disklabel says about our two disks  before 
the migration:
 a: 4261410815      2048       RAID
 b:  33554432 4261412863       swapo

After migration we will have
# gpt show -l wd0 
        start         size  index  contents
            0            1         PMBR (active)
            1            1         Pri GPT header
            2           32         Pri GPT table
           34         1980      1  GPT part - efi@wd0
         2014  11687488689      2  GPT part - raid0@wd0
  11687490703     33554432      3  GPT part - swap@wd0
  11721045135           32         Sec GPT table
  11721045167            1         Sec GPT header

# gpt show -l raid0
        start         size  index  contents
            0            1         PMBR
            1            1         Pri GPT header
            2           32         Pri GPT table
           34  11687488445      1  GPT part - root@raid0
  11687488479           32         Sec GPT table
  11687488511            1         Sec GPT header


There are two ways of resizing the RAIDframe. First here is 
Greg Osters' recommanded method:
- make sure you have a backup of your valuable data
- fail wd1 in raid0 using raidctl -f /dev/wd0a raid0
- migrate wd1 to GPT and resize the RAID partition
- create a new raid1 with compenents NAME=raid0@wd1 and "absent"
- creata a GPT and a FFS partition inside raid1
- format the FFS filesystem inside raid1
- copy data from raid0 FFS partiton to raid1 FFS partition
- unconfigure raid0
- migrate wd0 to GPT and resize the RAID partition
- add NAME=raid@wd0 as a spare to raid1
- reconstruct composent absent on spare NAME=raid@wd0
- optionally renamae raid1 to raidà using raictl -U 0 raid1

This approach is long, as it requires copying all the data.
It requires a few reboots, but it has the advantage that the 
data is always available during the migration. 

A faster but less secure approach will be documented here, 
all the operations are done booting on an INSTALL kernel, 
using the ramdisk's shell. It takes aboit 10 minutes to 
complete, most of the time being spent in fsck and resize_ffs.
- make sure you have a backup of your valuable data
- unconfigure raid0
- migrate wd0 and wd1 to GPT and enlarge the RAID partiton,
  lower the RAID partition start of 34 blocks, so that once 
  a GPT is added in the RAID, the FFS partition star block
  in unchanged. 
- configure a new raid0 with components NAME=raid0@wd0 and NAME=raid0@wd1 
- create a GPT inside raidà with an FFS partition
- DO NOT format the FFS partition, as you created it at the exact 
  place where the pre-migration FFS partition was. You data is 
  still there, unchanged.
- You can now remount the FFS partition. It works without a hitch if you 
  did not screw up things.
- Rewrite the new RAID parity.
- unmount the FFS partition and run fsck and resize_ffs to enlarge it.

The approach is fast, but make a mistake on a partition start
block and you data is gone. Note that rewriting RAID parity is 
fast, because it just reads the disks to check data is in sync. 
No copy is done. The longest operations are fsck and resize_ffs.

Here are the commands. Note that if you have other GPT-enabled 
disks, the dk devices unit numbers way vary. We install both 
EFI and BIOS boostraps. The latter requires a primary bootstrap
from 2023-07-01 or later (nebsd-9 and netbsd-10 branches) to 
boot a GPT/RAID-1/GPT/FFS setup.

raidctl -G raid0 > raid0.conf.orig
raidctl -u raid0

gpt create -Af wd0
gpt create -Af wd1

gpt add -t efi -l efi@wd0 -b 34 -s $(( 2048 - 34 - 34 )) wd0
gpt add -t raid -l raid0@wd0 -b $(( 2048 - 34 )) -s 11687488689 wd0
gpt add -t swap -l swap@wd0 wd0
gpt biosboot -A -i 2 wd0

gpt add -t efi -l efi@wd1 -b 34 -s $(( 2048 - 34 - 34 )) wd1
gpt add -t raid -l raid0@wd1 -b $(( 2048 - 34 )) -s 11687488689 wd1
gpt add -t swap -l swap@wd1 wd1
gpt biosboot -A -i 2 wd1

# Read the dk unit numbers from dmesg
efi0=dk0
efi1=dk3
raid0=dk1
raid1=dk4

newfs_msdos /dev/r${efi0}
mount -t msdos /dev/${efi0} /mnt && mkdir -p /mnt/EFI/BOOT && \
        cp /usr/mdec/bootx64.efi /mnt/EFI/BOOT && umount /mnt

newfs_msdos /dev/r${efi1}
mount -t msdos /dev/${efi1} /mnt && mkdir -p /mnt/EFI/BOOT && \
        cp /usr/mdec/bootx64.efi /mnt/EFI/BOOT && umount /mnt

sed "s/wd0a/${raid0}/; s/wd1a/${raid1}/;" raid0.conf.orig > raid0.conf
raidctl -C raid0.conf raid0
raidctl -A root raid0
raidctl -I 20230710 raid0   # Unique serial number is advised!
raidctl -i raid0

gpt create -Af raid0
gpt add -t ffs -l root@raid0 raid0
gpt set -a bootme -i 1 raid0

# check the dk unit numbe from dmesg
root=dk6

# INSTALL ramdisk lacks resize_ffs, copy it from the target filesystem
# This is the opportunity to check that eveyrthing went fine.
mount -o rw,log  /dev/${root} /mnt
cp /mnt/libexec/ld.elf_so /libexec/
cp /mnt/sbin/resize_ffs /tmp/
cp /mnt/lib/libc.so.12 /tmp/

# Copy secondary bootstrap
cp /usr/mdec/boot /mnt/boot

# Adjust fstab
cp /mnt/etc/fstab  /mnt/etc/fstab.orig
sed '  
        s|/dev/raid0a|NAME=root@raid0|;
        s|/dev/wd0b|NAME=swap@wd0|;
        s|/dev/wd1b|NAME=swap@wd1|;
' /mnt/etc/fstab.orig > /mnt/etc/fstab

umount /mnt

# Adjust console settings, this is for 115200 bps serial console
# and FFSv2 filesystem. As noted earlier, you need a recent bootxx_ffsv2
installboot -o console=com0,speed=115200 /dev/r${raid0} /usr/mdec/bootxx_ffsv2
installboot -o console=com0,speed=115200 /dev/r${raid1} /usr/mdec/bootxx_ffsv2

fsck -fy /dev/r${root}
# Make sure resize_ffs finds libc.so.12
export LD_LIBRARY_PATH=/tmp/
/tmp/resize_ffs -y /dev/r${root}

# Check new size
mount /dev/${root} /mnt
df -h /mnt
umount /mnt

# Reboot into normal operations
reboot



-- 
Emmanuel Dreyfus
manu%netbsd.org@localhost