Re: fsync, rdiff-backup, wapbl, and WD Elements 1T drive

[Matthew Mondor <mm_lists%pulsar-zone.net@localhost>]

On Fri, 28 Oct 2011 20:33:29 -0400
Greg Troxel <gdt%ir.bbn.com@localhost> wrote:

> So, I'm inclined to patch rdiff-backup not to fsync, since it seems
> excessive, and the backup is toast if the machine crashes before it is
> finished -- in that case rdiff-backup just rolls back.  Opinions?

I also wonder why fsync would be used for every file, especially if you
consider a whole run a single "transaction", even more so if using
snapshots (although you don't mention using them).  In which case it
simply should report failure and abort on any open/write/rename/close
error, and at the end, fsync once, also checking for error.  If at
that point everything was successful, the "transaction" is commited (as
far as software is concerned, of course, hardware buffers might still
need flushing), otherwise everything should be rolled back, unless an
inconsistent state is allowed (where the next full backup might fix
that).

I'm however wondering if the excessive fsync(2)s weren't eventually
added because of issues with ext4, as I somehow remember unix semantic
exceptions with it, and know that some have lost files using it as
they'd normally safely use other file systems (and I haven't followed
progress to know if it's since fixed).

But if rdiff-backup cannot optionally avoid those, adding an option to
tell it not to fsync at every file as you suggested would be very sane
IMO (it still could default to sync mode, in case there's upstream
resistence)...

I can understand the need for some transaction-logging applications to
call fdatasync(2) regularily, but that's another matter (and even then
it's usually configurable after how many bytes or seconds to call it to
allow the administrator to tweak performance).
-- 
Matt