Re: Why SFTP performance sucks, and how to fix it

[Markus Friedl <markus%openbsd.org@localhost>]

On Tue, Jul 08, 2003 at 04:01:02PM +1200, Peter Gutmann wrote:
> What this means is that no matter how fast
> the link, every (say) 16K the transmission stops for 1 RTT until the other
> side has sent its Ack (referred to as a window adjust in SSHv2 terminology).

Well, the problem here is the window size, not the packet size.

> In addition to the protocol-level handbrake, the SFTP protocol that runs on
> top of SSH contains its own handbrake.

That's not true.  While early SFTP implementations did wait for the
per-packet ACK, most recent implemenations (both ssh.com and OpenSSH)
send multiple concurrent requests.  For example, when running with
2 concurrent requests (sftp -R 2) the throughput improves from
~600K/s to ~1.9M/s for simple tests.

> The fix for this problem is obvious: Remove the handbrake.  This is no good
> reason for the per-packet Ack, and certainly other protocols such as SSHv1 and
> SSL/TLS function perfectly without it (the absence of the handbrake in SSHv1

SSL/TLS does not do port-forwarding, and SSHv1's lack of flow
control can cause problems when multiplexing different
channels over SSH.

> is why SSH FAQs observe that the SSHv1 scp is so much faster than the SSHv2
> SFTP, even though SFTP is overall a better design).

With OpenSSH's default (16 concurrent requests) I almost get 
scp(1) like performance in sftp(1), so I doubt it's a SFTP protocol
issue.

> The SSH-level handbrake can therefore be provisionally removed by having
> implementations set the window size to INT_MAX, and permanently removed by

This would mean that the implementatios have to buffer up to INT_MAX
bytes per channel.

> deprecating the Ack/window-based flow control and perhaps optionally providing
> Xon/Xoff-style flow control if absolutely necessary

SSH is more than just a file transfer protocol.

> (as was mentioned earlier,
> both SSHv1 and SSL/TLS function fine without requiring this).

Again SSHv1 does not function fine :)

> The SFTP-level
> handbrake can be removed by eliminating the maximum packet-size wording of the
> SFTP specification, and recommending that implementations read and write all
> data at once rather than engaging in additional redundant packetisation at the
> SFTP level.

Interleaving requests could be mentioned in the SFTP spec in an
implementation hint section.

> experimentation has shown that it's fairly safe to ignore the
> receiver's window size and send data at the maximum rate possible, discarding
> any window adjusts that arrive

I hope there are not many SSH implementations out there doing this :)

-markus