Re: Stateless SFTP server and READDIR race condition.

[Nicolas Williams <Nicolas.Williams%sun.com@localhost>]

On Tue, Apr 18, 2006 at 12:01:13PM -0600, Michael Closson wrote:
> On Thu, 2006-04-13 at 21:24 -0500, Nicolas Williams wrote:
> > SFTP needs to decide whether it's a filesystem protocol or a file
> > transfer protocol.  Then we can figure out what to do about state
> > recovery across reboots, partitions, etc.  A file transfer protocol
> > shouldn't really care about such failures, whereas a filesystem protocol
> > kinda has to.
> 
> Yes, I see your point.
> 
> Readers of this group may be interested in what people are doing with
> ssh and sftp.
> 
> My research group at the U of Alberta is involved in a Grid-like project
> (i.e., Globus/OGSA).  Basically, we want to be able to easily aggregate
> multiple HPC (High Performance Computing) systems at the user-level
> (also non-root level).  A typical use case goes something like this:  A
> researcher has ssh access to multiple clusters (probably administered by
> different groups and there fore no shared file system), and they want to
> load-balance some jobs across these clusters.  So, we need a nice way to
> ship job data back and forth.  We have an overlay file system that
> presents global view of these multiple systems to application programs.
> This overlay file system uses SFTP on the back end.  On the front end
> (i.e., the interposition agent) we have NFS, Samba, FUSE and a library
> that exports a POSIX api (open(), read(), etc.)

Just because NFS is typically implemented in kernel-land doesn't mean
you can't have one in user-land.  In fact, I think there may one or two
user-land NFSv4 implementations out there...

The same applies to CIFS.  And the reverse applies to FTP, and probably
to SFTP.

The point here is: don't get stuck with the wrong protocol because of
any misperceptions you may have.  Code availability probably means that
it is easier for you to use protocols traditionally implemented for your
purposes, but not necessarily, since one of the things you need is
thread-safety, and I doubt much off-the-shelf SFTP code is...

Nico
--