Re: External disk (umass) still writing: how to tell?

[tlaronde%polynum.com@localhost]

Thank you to both of you.

Things are clearer. And since it's a spare disk and I want to test for
example between eSata and USB connections, I will test various
things.

And if I find something of value (would it be only the differences of
throughput between USB and eSata or the differences of the OS load while
reading [resp. writing] in the two cases to the same disk), I would report
to the list.

Best regards,

T. Laronde

On Fri, Jul 26, 2019 at 03:23:22PM +0200, Johnny Billquist wrote:
> Hi.
> 
> On 2019-07-26 13:41, tlaronde%polynum.com@localhost wrote:
> >Hello,
> >
> >On Fri, Jul 26, 2019 at 01:23:34PM +0200, Johnny Billquist wrote:
> >>I think it's a serious mistake to use the physical action inside the disk as
> >>any kind of indication that there is any actual work going on, from the
> >>perspective of system data transfers.
> >>
> >>The disk can internally be doing various stuff at any point, which is
> >>totally irrelevant from this perspective. And disks usually also have the
> >>capacity to complete whatever operation is in progress and then move the
> >>heads to a safe area in case of power loss.
> >>
> >>So stop thinking that there is data being written out at a later point after
> >>the disk have been unmounted. If the disk is internally caching things, it
> >>is still safe to disconnect it. All transfer of data from the OS to the disk
> >>have been completed. The disk is free to handle this any way it want
> >>internally. But it has to ensure that all data are retained. If it didn't,
> >>then essentially it would be useless, as you could not even shut down the
> >>system.
> >>
> >
> >I'm not an english native speaker so just to be sure: do you mean it is
> >safe to unplug the board/device connection (this is not what is
> >bothering me) or do you mean it is safe to cut the power on the disk
> >(this is bothering me: if there is activity on the disk, since I can't
> >know what exactly it is doing, it must not be put out of power).
> 
> Don't worry about language. Keep asking if anything is unclear. I'm not a
> native English speaker either.
> 
> Yes, I do mean it is safe. As soon as umount have returned, the retention of
> data is the disks responsibility. If the disk can drop data it has been
> entrusted to keep, then the disk is broken. There is no way you can protect
> yourself against a broken device, so let's ignore that possibility for now.
> If the disk is not broken, then it will retain data that has been entrusted
> to it.
> 
> >That the OS has done its work and transfer all the data was my
> >assumption.
> 
> Right. So then there is nothing more the OS can do. It's all in the hands of
> the disk. You have to assume that the manufacturer of the disk have made
> sure that data that has been written to the disk are retained by the disk.
> 
> >My question was: does it mean that the OS can ensure that, when
> >the umount has been done, all the data it has transfered has been
> >written finally to the disk (physically) or could the disk still
> >have the data in cache (for me: memory)---and this is how I interpret
> >what you write: as far as the system is concerned, the data has been
> >transferred; what the device does with it is another problem, meaning
> >that a loss of power will be a loss of data erasing a cache not written
> >to disk.
> 
> You don't know, and you should not care.
> What you do know is that the disk have received the data, and the disk
> guarantees that the data is not lost.
> If the disk have a cache with a 100 year battery backup, or if the data was
> committed to the spinning rust is irrelevant. If you unplug the disk, and
> later plug it in again, the data that you wrote will still be there, and
> will be read out the same way you wrote it.
> 
> The disk internally is actually free to do all kind of things, as long as it
> fulfills the implicit and explicit requirements and expectations of the
> protocol between the controller and disk.
> Among those are that if data have been written to the disk, and the disk
> have acknowledged this, then the data will be there, even if power is lost.
> 
> It is not acceptable for the disk to acknowledge data as written, and loose
> it if power is lost after that.
> 
> Down that path lies madness. Because then you would never be able to trust
> the disk, and you would never be able to even shut the system down and power
> it off in an orderly fashion.
> 
> >Specially: if mounting with "sync" does the OS has the mean to
> >ensure that the data is written physically to the disk (that could
> >be powered off), waiting to return from umount until it is actually
> >done---while without sync, it can return once the data is transferred
> >as far as it (the OS) is concerned, but not waiting for the device
> >to tell it has been "committed" actually to disk.
> 
> No. The OS have no way of fully control what a disk does internally. And it
> wouldn't even make sense to try and have that kind of control.
> 
> But you can be sure that the data you wrote will be possible to read back,
> even if you have a power loss immediately after the write, no matter what
> kind of caches and optimizations the disk do internally.
> 
>   Johnny
> 
> -- 
> Johnny Billquist                  || "I'm on a bus
>                                   ||  on a psychedelic trip
> email: bqt%softjar.se@localhost             ||  Reading murder books
> pdp is alive!                     ||  tryin' to stay hip" - B. Idol

-- 
        Thierry Laronde <tlaronde +AT+ polynum +dot+ com>
                     http://www.kergis.com/
                       http://www.sbfa.fr/
Key fingerprint = 0FF7 E906 FBAF FE95 FD89  250D 52B1 AE95 6006 F40C