Re: Draft draft of coming sftp revision...

["Joseph Galbraith" <galb-list%vandyke.com@localhost>]

> > The trouble with the fpos_t solution is that in 1a, and 2, and at
> > least for us, sometimes 1 as well, the fpos_t would not be available
> > because it was not written to disk (not file data) and the
> > application has restarted.
> 
> Irritating, but probably true in most cases with current software.
> 
> However, software might change. I can easily imagine a text-aware
> SFTP client continuously updating a hidden temporary file (much like
> CVS's `.#' files) which would contain a server-side file offset
> (fpos_t or an ordinary file offset, according to taste) and a local
> file size. Then a transfer could be resumed by seeking to the given
> remote position on the server, and to the given local position on
> the client, and continuing to copy data. The temp file would be
> removed when the file was fully transferred.

But, consider the situation when the file wa
fully transferred, but has grown since the last
time I transfered it.  I just want to transfer
the new data.  A log file is a good example of this.
This looks just like a transfer that didn't complete.

Or, consider the solution where another protocol or a
different client preformed the first part of the transfer.
(I admit this is an edge case.)

Neither of these cases will work if additional metadata is
required to resume the transfer.

> I'd recommend supporting the offset field at least to some extent,
> for purposes of multiple parallel sequential reads. Or would you
> expect a client to open the same file several times to do this?

Okay-- I think I see what you are saying.  I'm not sure
how to change things to be clear about how this should
work.

I don't think the client should open the same file several
times for this-- in fact, I think the correct behavior for
the client is that if the server returns SSH_FX_OP_UNSUPPORTED
for a parallel sequential read, the client should fall
back to non-parallel reads.

But I think it is now obvious we need specify more
clearly about how the offset field should be used
when FXF_TEXT is used, and what the server SHOULD
do and what it MAY do.

I'll see if I can't rework the section and then probably
repost the draft again.

Okay-- I thought about this for a couple of minutes;
here are my first and second ideas... let me what
you think.

----- 1 -----
"When FXF_TEXT|FXF_APPEND are specified, the offset field
 is ignored and all writes will be appended to the file.

"If FXF_TEXT|FXF_APPEND|FXF_READ are specified, the
 read offset field has special behavior.  An offset of
 0 is considered a request for the next sequential
 data in the file.  Any other read offset is considered
 a seek and read operation.

"It is not possible to seek to position zero in the file.
 Alternatively, clients can close and reopen the file to
 seek position 0.

"Servers SHOULD support seek-and-read in order to support
 resume.  However, a client MUST be prepared to receive
 SSH_FX_OP_UNSUPPORTED when attempting a seek-and-read
 operation.  The client can then try a fall-back
 strategy, if it has one."

"Servers SHOULD correctly process multiple parallel
 reads correctly in this mode.  However, if the client
 recieves SSH_FX_OP_UNSUPPORTED when attempting multiple
 parallel reads, it SHOULD fall back to serial reads."

----- or #2 -----

"Servers SHOULD correctly process multiple parallel
 reads correctly in this mode.  However, if the client
 recieves SSH_FX_OP_UNSUPPORTED when attempting multiple
 parallel reads, it SHOULD fall back to serial reads."

"Clients SHOULD use the SSH_FXF_APPEND flag to append
 data to a text file rather then using write with a
 calculated offset.

"When a file is opened with the FXF_TEXT flag, the
 offset field in both the read and write function
 are ignored.

"To support seeks on text file the following
 SSH_FXP_EXTENDED packet is defined"

"string 'text-seek'
 uint64 line-number

"line-number is the index of the line number to
 seek to, where byte 0 in the file is line number 0,
 and the byte directly following the first newline
 sequence in the file is line number 1 and so on.

"Servers SHOULD support at least on 'text-seek' in
 order to support resume.  However, a client MUST
 be prepared to receive SSH_FX_OP_UNSUPPORTED when
 attempting a 'text-seek' operation.  The client
 can then try a fall-back strategy, if it has one."

So what do people think?  I think I'm leaning towards
#2-- it doesn't involve overloading the meaning
of the FXF_APPEND flag, which I think is a cleaner
design.  It also doesn't have the restriction about
seeking position 0.  Additionally, I think it makes
the calculation of where to seek less error prone.

Thanks again for your feedback.

- Joseph