Re: FFS: wrong superblock check ~> crash

[Mindaugas Rasiukevicius <rmind%netbsd.org@localhost>]

mlelstv%serpens.de@localhost (Michael van Elst) wrote:
> rmind%netbsd.org@localhost (Mindaugas Rasiukevicius) writes:
> 
> >If I have a system with an array of disks and one of them fails, a crash
> >would take down the whole node.  When many terabytes of data suddenly
> >disappear people get unhappy;
> 
> When a single node going down makes people unhappy, you may want to have
> more than one :)

There are convergence and recovery times which may have an extra cost
for you.  Sure, the failure of a single node might merely result in an
increased latency somewhere.  However, there can be a significant
difference in recovering the whole node and just replacing/recovering
one segment of data in it.  That is especially the case with the caches,
you do not want to randomly blow and refill them - it takes some time.

> >it usually costs quite a bit of money too.
> >So, how about making only the *failed* segment of data unavailable?
> 
> It all depends on what you can or are willing to handle. Also what a
> crash (and reboot) would actually solve. It doesn't help if your
> boot partition is damaged and you force a reboot, converting a
> maybe half working system into a dead system.

Not really relevant to my example where you would have dedicated disks
for the data, but I think we agree on this - there are different use
cases and it depends.  If the boot partition is damaged, then it does
indeed most likely mean that the system is not in the state where it
can do its job.  I would probably still just mount the file system to
read-only mode and let it fail elsewhere, but it may as well panic; at
this point it is non-functioning and requires manual intervention anyway.

> In my use case, a broken filesystem is usually a sign of an unnoticed
> hardware or software error and the best reaction to recover is to
> panic (and throw the data away, the machines have only temporary
> data). Continuing with a read-only filesystem doesn't do any good,
> because you have no means to find out wether the data you can read
> is complete or correct.

Why not?  It seems to be a question of how do you communicate the errors
back to the applications or administrator.  There are applications which
gracefully handle EIOs exactly for this purpose.  The fact that they are
very rare does not mean they do not exist. :)

> Most clustered systems also handle complete outages better than
> a degraded mode. That's why you have things like STONITH.

It really depends on the application or service; you often have to take
design considerations for both though.

-- 
Mindaugas