Re: External disk (umass) still writing: how to tell?

[Johnny Billquist <bqt%update.uu.se@localhost>]

Hi.

On 2019-07-26 13:41, tlaronde%polynum.com@localhost wrote:
> Hello,
>
> On Fri, Jul 26, 2019 at 01:23:34PM +0200, Johnny Billquist wrote:
> > I think it's a serious mistake to use the physical action inside the disk as
> > any kind of indication that there is any actual work going on, from the
> > perspective of system data transfers.
> >
> > The disk can internally be doing various stuff at any point, which is
> > totally irrelevant from this perspective. And disks usually also have the
> > capacity to complete whatever operation is in progress and then move the
> > heads to a safe area in case of power loss.
> >
> > So stop thinking that there is data being written out at a later point after
> > the disk have been unmounted. If the disk is internally caching things, it
> > is still safe to disconnect it. All transfer of data from the OS to the disk
> > have been completed. The disk is free to handle this any way it want
> > internally. But it has to ensure that all data are retained. If it didn't,
> > then essentially it would be useless, as you could not even shut down the
> > system.
>
> I'm not an english native speaker so just to be sure: do you mean it is
> safe to unplug the board/device connection (this is not what is
> bothering me) or do you mean it is safe to cut the power on the disk
> (this is bothering me: if there is activity on the disk, since I can't
> know what exactly it is doing, it must not be put out of power).

Don't worry about language. Keep asking if anything is unclear. I'm not 
a native English speaker either.

Yes, I do mean it is safe. As soon as umount have returned, the 
retention of data is the disks responsibility. If the disk can drop data 
it has been entrusted to keep, then the disk is broken. There is no way 
you can protect yourself against a broken device, so let's ignore that 
possibility for now.
If the disk is not broken, then it will retain data that has been 
entrusted to it.

> That the OS has done its work and transfer all the data was my
> assumption.

Right. So then there is nothing more the OS can do. It's all in the 
hands of the disk. You have to assume that the manufacturer of the disk 
have made sure that data that has been written to the disk are retained 
by the disk.

> My question was: does it mean that the OS can ensure that, when
> the umount has been done, all the data it has transfered has been
> written finally to the disk (physically) or could the disk still
> have the data in cache (for me: memory)---and this is how I interpret
> what you write: as far as the system is concerned, the data has been
> transferred; what the device does with it is another problem, meaning
> that a loss of power will be a loss of data erasing a cache not written
> to disk.

You don't know, and you should not care.
What you do know is that the disk have received the data, and the disk 
guarantees that the data is not lost.
If the disk have a cache with a 100 year battery backup, or if the data 
was committed to the spinning rust is irrelevant. If you unplug the 
disk, and later plug it in again, the data that you wrote will still be 
there, and will be read out the same way you wrote it.

The disk internally is actually free to do all kind of things, as long 
as it fulfills the implicit and explicit requirements and expectations 
of the protocol between the controller and disk.
Among those are that if data have been written to the disk, and the disk 
have acknowledged this, then the data will be there, even if power is lost.

It is not acceptable for the disk to acknowledge data as written, and 
loose it if power is lost after that.

Down that path lies madness. Because then you would never be able to 
trust the disk, and you would never be able to even shut the system down 
and power it off in an orderly fashion.

> Specially: if mounting with "sync" does the OS has the mean to
> ensure that the data is written physically to the disk (that could
> be powered off), waiting to return from umount until it is actually
> done---while without sync, it can return once the data is transferred
> as far as it (the OS) is concerned, but not waiting for the device
> to tell it has been "committed" actually to disk.

No. The OS have no way of fully control what a disk does internally. And 
it wouldn't even make sense to try and have that kind of control.

But you can be sure that the data you wrote will be possible to read 
back, even if you have a power loss immediately after the write, no 
matter what kind of caches and optimizations the disk do internally.

  Johnny

--
Johnny Billquist                  || "I'm on a bus
                                  ||  on a psychedelic trip
email: bqt%softjar.se@localhost             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol