Subject: Re: Archie-Clients
To: VaX#n8 <vax@ccwf.cc.utexas.edu>
From: Bakul Shah <bakul@netcom.com>
List: current-users
Date: 01/28/1995 13:34:53
> One kind of has to ask the question, then, why it hasn't been done.
> I know many ftpd's now don't fork for "ls", but really, one server per
> client is a pretty bad idea on a big ftp site.

May be because each user has to have his own context?  Each
FTP user may have his own curren dir., may have used his own
login acct. etc.  Dealing with all that from a single ftpd
will be pretty messy and *slow*!  One server per client eats
some memory resources but I suspect the reason big ftp sites
bog down is probably due to file system traffic (how fast
N files, scattered all over a FS can be read) and network
pipe (how fast this data can be shovelled down the netpipe).

Somebody *must* have done performance analysis on a big FTP
site.  If not,it is worth doing.  Anyway, the gathered data
should be analyzed before changing ftpd in any major way.
If there is a definite knee in the graph (of number of files
being simulteneously transferred versus throughput) may be
the thing to do is provide `unfair' service -- service N
requests while blocking any more.

One thing that may be worth looking into doing is to pass
open file descriptors to a `transfer' daemon.  The idea is
that an ftpd opens the input file and a tcp connection for
transferring to/from it and passes both descriptors to a
transfer daemon.  A transfer daemon will accept a limited
number of such transfer requests and handle them all via a
select.  This will effectively throttle the requests.  (the
cutoff point may be dynamically adjusted by looking at
system load or fixed depending on known values of average FS
throughput and network throughput).

> You don't have to reconnect, and even better, if the
> server went down 90% through grabbing a file, you can
> continue from where you left off.
> (meaning less resends)

You can use the `reget' command of ftp for this.

Bakul