Subject: ufs tool: replies
To: None <current-users@NetBSD.ORG>
From: der Mouse <mouse@Collatz.McRCIM.McGill.EDU>
List: current-users
Date: 08/23/1995 16:44:22
I wrote
> I've just finished a little tool for UFS filesystems.  [...]

I've gotten a somewhat astonishing reaction; it seems a lot of people
are interested in this.  I've got it up for ftp now (on 132.206.78.1
(collatz.mcrcim.mcgill.edu) in /mouse/fsresize.{c,8})...but note that
what's presently there will not compile on a stock system.  I believe
very little work would be needed to make it do so (maybe even just
fixing argvec), which is why I've made it available even in its current
state.  I hope that tomorrow sometime I'll get it cleaned up and ready
to distribute.

Some people have asked questions and/or made comments that I want to
address, which is why this second letter.  (Further quotes are from
mail I've received; quotes of my message will be double-quoted.)

> How do you deal with partitions, especially when you might be going
> over paritition boundaries?

I don't deal with partitions.  Before you use fsresize, you have to
ensure the device the filesystem resides on has enough unused space for
the expansion to take place.  This may mean concatenating a new disk to
a ccd virtual partition, or enlarging a file for a vnd partition, or
repartitioning a disk for a "traditional" partition.

> Can this do arbitrary overlapping file system moves?  If I've decided
> that I want to shuffle around the partitions on my disk and assuming
> that I've dealt with making sure any data in the partition I'm
> partially shifting into is safe, can this handle it?

No.  It makes no attempt to address what it sounds as though you want
to do.

> (I hope there is a manual :-)

I've written a rudimentary manpage for it.  Better documentation would
be welcomed, though I'm not sure how much more user-level docs there
can be - it really is a very simple program from the point of view of
its user.

> This sounds great, particularly if it could work in conjunction with
> the new dynamically updateable ccd driver.

As I understand the new ccd (which is close to your understanding of
it, though I haven't looked closely), it will "work in conjunction
with" it only to the extent that a human can update the ccd and then
use fsresize to grow the filesystem into the new space.  (Of course, as
someone else pointed out, this works only for non-interleaved ccd
configurations.)

>> (It makes no attempt to optimize choice of places to put the moved
>> stuff; this might be a good project if someone's feeling ambitious.)
> Actually, a more general tool which defragments and re-optimizes ufs
> filesystems might be more useful in that regard.

Agreed.

> How do you calculate the max possible size of the filesystem?

I don't.  You have to tell fsresize, when you run it, how large you
want the filesystem to be.  fsresize will, when growing, check that the
size you give is in fact accessible, same as newfs will, by writing the
last block in it.  It does not support automatically determining the
new filesystem size from anything, though this could probably be added.

> If it could be made to work for things like including excluding disks
> in a ccd device, I'd very much like to see it in the tree.  This of
> course requires not only resizing, but also removing block ranges
> from the middle of a fs, not only at the end.  Would this be hard?

If I understand you right, this would amount to doing exactly what it
does now, then shifting a certain portion further down in the (virtual)
partition, so that a disk could be removed from the middle of the ccd
and the rest concatenated to form a working filesystem.  It doesn't do
this at present; I think it would be better to address this with a
separate program to do the bulk shifting operation.

> Perhaps the resizer could also become a re-interleaver?

Perhaps it could, but it would be somewhat ugly.  Since some of the ufs
structures live at more or less fixed positions, re-interleaving could
be done only after adding the new disk, and in that case, interpreting
the old fs structures is messy because filesystem disk block numbers
need a nontrivial mapping.  I would prefer to create a new ccd-aware
utility one could run after adding or before removing a disk, one that
would just shuffle blocks so as to preserve (the initial segment of)
the virtual partition.

> How well does this work with the ccd driver?

> Can you take an existing partition, make it the first chunk of a
> non-interleaved ccd, and grow the filesystem in place?

I haven't tried.  I see no reason why it wouldn't work, assuming the
"existing partition" doesn't begin at the beginning of a disk (per the
warnings in the ccd-related manpages about using such partitions).

> As I see it, at the moment, the only way to re-interleave a ccd is to
> backup your data, reconfigure the ccd, newfs, and restore.

Which is why a re-interleaving tool could be useful.  Perhaps that will
be my next quixotic project. :-)

> How failsafe is it?  Will a power failure or a kernel crash in during
> a reorg trash the FS?  If not failsafe, have you any ideas on how to
> make it so?

I didn't consider that when writing it, and it doesn't do anything like
fsync() to make sure writes happen in the correct order.  Without
detailed inspection of the code, all I can do is make guesses.  I would
guess that when growing, a crash may leave a damaged filesystem, but
not damaged beyond fsck's ability to repair.  When shrinking, there is
no question that there are windows during which a crash could leave a
badly damaged filesystem, probably so severely damaged as to be
irreparable.  This may be fixable at the price of more code complexity;
just how much more code complexity I couldn't say without trying to
implement it.

					der Mouse

			    mouse@collatz.mcrcim.mcgill.edu