Re: Draft draft of coming sftp revision...

[Simon Tatham <anakin%pobox.com@localhost>]

Joseph Galbraith <galb-list%vandyke.com@localhost> wrote:
> I am not a FTP guru, but I just talked to our FTP guru, and he tells
> me that stock FTP, at least, doesn't do any charset translation.
> There might be an extension for it.

Oh, OK. Fair enough.

> [charset translation can be done with an extension]

That's fair enough too.

> Okay; that's fair.  My first inclination is to say that anything not
> matching the servers newline sequence exactly (either the default
> \r\n, or newline as specified by the server) SHOULD NOT be
> translated. In particular, if the newline convention is \r\n, and
> the client sees a lone \r or lone \n, it should simply write the
> data through without modification.
> 
> What do you think?

Yes, that seems as good as anything to me. Realistically the only
instances of this I can easily bring to mind are

 (a) text files being stored on the server in the client's format
     already, in which case this algorithm will preserve whichever
     of \r and \n was the client's line-ending character, so it will
     end up looking plausible.

 (b) DOS text files containing CRs for purposes of visual weirdness,
     in which case this algorithm will transfer them correctly to
     Unix. It'll fail horribly when transferring them to a Mac, but
     they were doomed anyway in that situation.

So I think this is a plausible solution, yes.

[resuming transfers]
> But, usually the only thing we have client side is the file data.

Yes, I was thinking that too. Even if you can seek in server-side
text files, you'd have to scan right through the client-side
partially transferred file to work out where to seek _to_.

> The trouble with the fpos_t solution is that in 1a, and 2, and at
> least for us, sometimes 1 as well, the fpos_t would not be available
> because it was not written to disk (not file data) and the
> application has restarted.

Irritating, but probably true in most cases with current software.

However, software might change. I can easily imagine a text-aware
SFTP client continuously updating a hidden temporary file (much like
CVS's `.#' files) which would contain a server-side file offset
(fpos_t or an ordinary file offset, according to taste) and a local
file size. Then a transfer could be resumed by seeking to the given
remote position on the server, and to the given local position on
the client, and continuing to copy data. The temp file would be
removed when the file was fully transferred.

> The other thing I would note is that if a server wants to support
> resume, it only needs to support the offset field on the first write
> or read.  And it doesn't need to support that if it doesn't want to
> support resume.  I wrote the text to require clients to expect
> failure during non-sequential operation.

I'd recommend supporting the offset field at least to some extent,
for purposes of multiple parallel sequential reads. Or would you
expect a client to open the same file several times to do this?

Cheers,
Simon
-- 
Simon Tatham         "loop, infinite _see_ infinite loop"
<anakin%pobox.com@localhost>     - Index, Borland Pascal Language Guide