ZFS - mounting filesystems (Was CVS commit: src/etc)

[Simon Burge <simonb%NetBSD.org@localhost>]

[ Moving this from source-changes-d to tech-userlevel and combining
a couple of messages with one rambling reply. ]

Greg Troxel wrote:

> Simon Burge <simonb%NetBSD.org@localhost> writes:
>
> > I'm running with a complete ZFS-only setup with no legacy mounts.  This
> > is my basic ZFS layout (leaving out a few mounts that don't add any more
> > value to this discussion):
>
> [I am guessing you have a boot partition that isn't zfs, just to load
> the kernel, but I haven't tried to read about that, and it seems a
> separate topic.]
>
> I will assume that the / (zfs as you say above) is the only mounted fs
> as init starts.

I'm using ZFS as my root filesystem, with the EFI boot loader reading
the kernel from the ZFS root filesystem.  This is based on the the
FreeBSD libsa ZFS code. https://github.com/snarkophilus/src/tree/zfsboot
is this work, and I'm planning on merging into main NetBSD sources soon.


Later, Greg Troxel wrote:

> It makes sense to have a boolean "critical" property (the name you
> suggested is fine) for zfs volumes that specify a mount point, so that
> such volumes would be mounted in mountcritlocal.  I am 100% happy for
> someone to add that and see no big problems.

I will look into doing more on this, depending on the outcome of
discussion on just "mountlocal" below.

> I think this is tricky if there are multiple pools and some don't come
> up.  But I think it's ok if the main path is that one should have all
> critical zfs filesytems on the same pool as root, with root on zfs.
> With root not on zfs but say /usr and /var on zfs, there needs to be
> some way for the boot to fail if they aren't mountable, just like if
> they were in fstab, if the pool doesn't attach and thus the critical
> variable aren't readable.  That might mean "if root is not zfs, and
> something in critical_fileystems_{local,remote} is in zfs, then those
> things have to use zfs legacy mounts".  It might mean having
> "critical_zfs_pools_{local,remote}" which will fail the boot if they
> are not attached at the mountcritlocal/mountcritremote stage, so that
> the critical properties are reliably there.

Multiple pools is something I haven't really thought about yet.  I see
that FreeBSD tries to import the pools in the ZFS pool cache which we
(NetBSD) don't try to do yet.  Probably more investigation needed there.

> I am opposed to deciding that all zfs filesystems should be treated as
> critical (in a world where we have not abolished the notion).
>
> We could have a discussion about why we even have the concept of
> critical filesystems, but if so that should not be about zfs and
> should be in a new thread on tech-userlevel.  And, I think it really
> isn't strongly releated to this discussion.

And Brad Spencer wrote:

> Ya, I suspect that the notion of precedence needs to be developed
> somehow and not abuse the notion of critical too much.  That is,
> filesystem mounting by type needs to happen in a particular order.  The
> default would have to choose something, either making not ZFS happen
> first followed by normal and usual ZFS pools, or the other way around.

Another idea that came up in off-list discussion with Taylor is in the
same vein as this - the whole notion of "critical".  So changing from

        mountcritlocal
        mountcritremote
        mountall

to

        mountlocal
        mountcritremote
        mountremote

or even just a simpler

        mountlocal
        mountremote

Do we have any valid need to have non-critical local filesystems?


For what it's worth, FreeBSD appears to do things in this order:

        /etc/rc.d/zpool
                load all cached zpools
        /etc/rc.d/zvol
                load zvols for swapping onto
        /etc/rc.d/root
                mount -uw /
        /etc/rc.d/mountcritlocal
                mount all non-NFS filesystems (except non-legacy ZFS)
        /etc/rc.d/zfs
                zfs mount -a (and zfs share -a)
        [ ... much later on ... ]
        /etc/rc.d/mountcritremote
                mount all NFS filesystems
        [ ... nearly last thing ... ]
        /etc/rc.d/mountlate
                mount filesystems marked as "late" in /etc/fstab

The "crit" in mountcritlocal and mountcritremote don't seem to have any
current meaning, so is similar to just the mountlocal/mountremote idea
above.

> I suspect that someone loses out as no default can handle every way that
> ZFS may be used when other filesystem types are present.  For example,
> in the cases I have, a FFS is the root filesystem along with other FFS
> file systems.  Under these are ZFS normal pools, so the order I would
> prefer in this case is mount FFS first and then import the pools, as the
> places that the pools would mount under would not exist the other way
> around.  However, I very much understand that the other way around could
> exist where ZFS normal pools are imported before anything else.

I think if you have complex ordering requirements with a mix of ZFS
and other filesystem types, then you are probably better off using ZFS
legacy mounts?





Brad Spencer wrote:

> Greg Troxel <gdt%lexort.com@localhost> writes:
>
> [snip]
>
> >
> >   I am opposed to deciding that all zfs filesystems should be treated as
> >   critical (in a world where we have not abolished the notion).
> >
> >   We could have a discussion about why we even have the concept of
> >   critical filesystems, but if so that should not be about zfs and
> >   should be in a new thread on tech-userlevel.  And, I think it really
> >   isn't strongly releated to this discussion.
>
> [snip]
>
> Ya, I suspect that the notion of precedence needs to be developed
> somehow and not abuse the notion of critical too much.  That is,
> filesystem mounting by type needs to happen in a particular order.  The
> default would have to choose something, either making not ZFS happen
> first followed by normal and usual ZFS pools, or the other way around.
> I suspect that someone loses out as no default can handle every way that
> ZFS may be used when other filesystem types are present.  For example,
> in the cases I have, a FFS is the root filesystem along with other FFS
> file systems.  Under these are ZFS normal pools, so the order I would
> prefer in this case is mount FFS first and then import the pools, as the
> places that the pools would mount under would not exist the other way
> around.  However, I very much understand that the other way around could
> exist where ZFS normal pools are imported before anything else.  I
> honestly don't remember what Solaris did, but I can't see any ordering
> default with NetBSD handling every case in the correct manor.  Even if
> it were possible to know the root filesystem type in a simple manor like
> one can know the root filesystem device and partition, I don't think
> that it would be possible to guess all of the cases correctly.
>
>
> -- 
> Brad Spencer - brad%anduin.eldar.org@localhost - KC8VKS - http://anduin.eldar.org
>

Cheers,
Simon.