Re: Sftp rename: 2 proposals

[mbp%ozlabs.org@localhost (Martin Pool)]

On Fri, Sep 05, 2003 at 11:04:19AM -0600, Joseph Galbraith wrote:
> I made this proposal, but after not being able to find it, I realized
> I screwed up sending it and it didn't ever make it to the list:
> 
> All right, how about the following change:
> 
>    Files (and directories) can be renamed using the SSH_FXP_RENAME
>    message.  Its data is as follows:
> 
>         uint32     id
>         string     oldpath [UTF-8]
>    	string     newpath [UTF-8]
>         uint32     flags
> 
>    where `id' is the request identifier, `oldpath' is the name of an
>    existing file or directory, and `newpath' is the new name for the
>    file or directory.

I would be happy to see this proposal accepted rather than mine.  I do
think it is important the problem be addressed in some way.

>    flags is 0 or a combination of
> 
>      SSH_FXP_RENAME_OVERWRITE  0x00000001
>      SSH_FXP_RENAME_ATOMIC     0x00000002
>      SSH_FXP_RENAME_NATIVE     0x00000004
>      SSH_FXP_RENAME_COPY_OK    0x00000008
> 
>    If flags does not include SSH_FXP_RENAME_OVERWRITE,
>    and there already exists a file with the name
>    specified by newpath, the server MUST respond
>    with SSH_FX_FILE_ALREADY_EXISTS.
> 
>    If flags includes SSH_FXP_RENAME_ATOMIC, and
>    the destination file already exists, it is
>    replaced in an atomic fashion.  I.e., there
>    is no observable instance in time where the
>    destination file does not exist.  SSH_FXP_RENAME_ATOMIC
>    implies SSH_FXP_RENAME_OVERWRITE.

s/instance/instant/?

Perhaps rather than "does not exist" say something like "where the
name does not refer to either the old or the new file", as the unix
rename operation is described.  Having a moment where the file appears
empty or incomplete would be wrong, but this might happen for a
copy/delete operation.

Perhaps it should mention that "observable" includes other processes
or SSH clients, not just this client.

>    There may, however, be some span of time where
>    both the source and destination file exist.
>    SSH_FXP_RENAME_ATOMIC has no bearing on the
>    source file.

I'm not sure about this last sentence.  If you just mean that both
files may exist for a moment, it is redundant with the previous
sentence.

>    If flags includes SSH_FXP_RENAME_ATOMIC and the
>    server cannot replace the destination in an
>    atomic fashion, then the server MUST respond
>    with SSH_FX_OP_UNSUPPORTED.
> 
>    Because some servers cannot provide atomic rename,
>    clients should only specify atomic rename if correct
>    operation requires it.  If SSH_FXP_RENAME_OVERWRITE
>    is specified, the server MAY preform an atomic
>    rename even if it is not requested.
> 
>    If flags includes SSH_FXP_RENAME_NATIVE, the server
>    is free to do the rename operation in whatever
>    fashion it deems appropriate.  Other flag values
>    are considered hints as to desired behavior, but
>    not requirements.
> 
>    If flags includes SSH_FXP_RENAME_COPY_OK, the
>    server MAY preform a copy / delete operation
>    in order to complete the operation.  If an atomic
>    rename is also requested, the server may only
>    preform this sequence if it can do so in an
>    atomic fashion for the destination file.

s/preform/perform/

I know what copy/delete is, but I don't understand the meaning or
intention of this flag.  Do you mean that e.g. cross-filesystem moves
would be allowed only if this flag was specified, and they would fail
otherwise?  

So presumably most clients would turn SSH_FXP_RENAME_COPY_OK on?  Who
would turn it off?  Clients worried about the time it would take to
copy the file?

>    The server may fail rename requests in other
>    situations, for example if `oldpath' and `newpath'
>    point to different file systems on the server.
> 
>    The server will respond to this request with a
>    SSH_FXP_STATUS message.

[on my proposal]

> o It is not possible to know if the server is unix (and I
>   think, in general, it is a bad idea to make decisions
>   based on OS type.)

I wasn't suggesting that the client ought to send "are you unix?".
However, if a human is using sftp interactively they *may* know what
the remote filesystem is, and want to take advantage of that.  

> o If the client does _not_ wish to have the file replaced,
>   it must stat the file and wait for the response before
>   preceeding.

That is a drawback.  And there is a small race here as well: some
other process might create a new file in the interim.

> o As mentioned above, if the client wishes the file to
>   always be overwritten, it must delete the destination,
>   and then do the rename, but even that may _still_ fail
>   if something comes along and recreates the file in the
>   interim.  (Imagine a system where the file must exist,
>   and so there is a monitor program which recreates the
>   file if it is deleted.  Farfetched?  Maybe.  Possible?
>   Definitely.)

But this problem also exists on server systems where the native rename
does not overwrite.  Rename-and-replace on Windows (I think) has to be
implemented by deleting the file and then renaming.  So there is still
a race window, although since it does not include a network round trip
it is smaller.

> I think it is far better for the client to be able to specify
> the behavior they want on the rename.  Most clients are probably
> happy with whatever the server does natively.

Yes, I agree.  I like that your proposal makes the "don't care" value
obvious.

-- 
Martin