Re: Improving RAIDframe Parity Handling: The Diff

[Jed Davis <jld%panix.com@localhost>]

matthew green <mrg%eterna.com.au@localhost> writes:

> - newfs tends to dirty a huge portion of zones.  for my 250GiB filesystem,
>   newfs dirtied 1491 out of 4096 zones, which is a few more than the total
>   number of cyl groups:
>
>     using 1425 cylinder groups of 184.30MB, 11795 blks, 23296 inodes.
>
>   these zones cleared up a few minutes later, without syncing 1491 * 64MB,
>   so this will only be a problem with a crash in the minutes after a newfs

Indeed; they were written to, which usually means that there might be
more writes nearby soon (but happens to not to mean that in this case).
Note that the "few minutes later" is adjustable via raidctl(8); see
under "-M set"...

> - with 10 extractors of pkgsrc and one 'cvs co src xsrc' (and rm -rf's for
>   the both) running all in parallel, i ended up with about 250 dirty zones
>   out of 4096, which seems pretty high.  i haven't seen it go beyond 514
>   except for the newfs case.. but >1/8th seems a lot.

...meaning that, if one wanted to have fewer dirty zones at the cost of
a bit more overhead from parity map updates, that can be accomplished.

At one point I'd had thoughts of making the parameters automagically
adjust based on load, but that seemed... delicate, at best, and the
fixed settings do still beat the status quo by quite a bit.

> - (nit) "raidctl -s" output is confusing for parity reconstruction.  the
>   percentage done doesn't seem to make sense for me now.  from a guess, it
>   is not considering in-sync but beyond the current sync-point as being
>   in-sync so that the percentage done number grows at strange speeds, slow
>   while in a dirty zone, and rapidly while skipping clean zones.

It's reporting how far through the RAID set it is, just like before;
it's just that some parts of the disk are now "checked" without doing
any I/O, so they should complete more or less instantaneously.

It looks like this can actually be fixed without too much work, or
doing anything to userland.

> - might be nice to add a comment to the RAIDFRAME_SET_COMPONENT_LABEL ioctl
>   that the new #if 0'ed code is not well tested?

Might be.  I suspect that the old #if 0'ed code may not have been well
tested, either.

-- 
(let ((C call-with-current-continuation)) (apply (lambda (x y) (x y)) (map
((lambda (r) ((C C) (lambda (s) (r (lambda l (apply (s s) l))))))  (lambda
(f) (lambda (l) (if (null? l) C (lambda (k) (display (car l)) ((f (cdr l))
(C k)))))))    '((#\J #\d #\D #\v #\s) (#\e #\space #\a #\i #\newline)))))