Re: solving the SFTP text mode issue

["Joseph Galbraith" <galb-list%vandyke.com@localhost>]

> > The solution proposed works extremely well in the
> > fixed length record cases -- in fact, I would say
> > it is ideal.  It provides, fast, efficient, random
> > record based access.
>
> The question is: is there a need for random access to record-based files?
>
> After some consideration, I think that the READ_RECORD/WRITE_RECORD
proposal
> is probably too enthusiastic. Implementing random access to files where
> records are lines of text will be difficult on most platforms. Most
> applications don't currently do random access in the first place, so it's
> hard to imagine it being done much with records.
>
> But if we dismiss random access, than the only remaining advantage of the
> readrecord/writerecord proposal is that it allows LF to be embedded in
> lines.
>
> I think that Wei's proposal is the most balanced of all I've considered
> today:
>
> - It does seem to support all kinds of text files reasonably well,
including
> record-based text files. The latter is true as long as individual records
> don't contain LF characters, which I assume is reasonable to expect from a
> text file.
>
> - It is simple to implement on Windows and Unix. It resembles the approach
> taken by C/C++ standard libraries to solve the same problem. Translation
> between CRLF and LF is simple. No convoluted algorithms to support random
> access need to be implemented.

In terms of solving the text transfer problem,
Wei's proposal is sufficient I believe, although
it does not provide for resuming a large text
transfer if it is interupted.  (Something our
customers want to be able to do.)

I'll talk more about Wei's solution in a response
directly to it.

I must say that the solution I proposed was
not aimed at solving the text transfer problem,
per say, but at solving the record file access
problem that VMS folks have (and probably
mainframe folks too.)

It just happens that the text access problem
is a subset of the record access problem.

Now, if the random access thing bothers your
too much, we might say something like:

If the SSH_FILEXFER_ATTR_RECORD_ACCESS
flag is not set, the file can not be
opened for RECORD access.  Any attempt
to do so will result in a
SSH_FX_INVALID_ACCESS_MODE error.

If the SSH_FILEXFER_ATTR_RANDOM_ACCESS
flag is not set, the offset and / or
record number field is ignore in
READ, READ_RECORD, WRITE, and WRITE_RECORD
operations and the file is read / written
in sequential order.

Now, VMS (or any other operating system that
natively supports random access record oriented
files) can natively support this, and NT or
unix, for example,

Of course, on the other hand, I'm not
really attached to this solution -- only
proposing that it solves more of the proposed
problem then other solutions proposed.

Personally, if we want to solve just the
text access solution, we have the server
advertise what line delimiter it will
use, and the MacOS and VMS folks who
deal with different kinds on a per file
basis can do conversions on the server side.

I.e., the VMS server would convert a
variable length counted file into \r\n
delimited, and always advertise \r\n
as its line delimiter.

As I said before, we already implement
this solution in our servers, and if
other servers implemented it, it would
solve 99% of our customers problems,
which is a lot more than we currently have.

And it doesn't really prevent implementors
working under VMS from using the solution --
they just have to work harder.  They have
to do the server side translation work
the rest of us are avoiding by not specifying
the canonical line termination in the draft,
but rather allowing the server to specify it's
own cananical line termination.

- Joseph