On 24 Dec 1999 20:52:14 -0000 
 explorer@flame.org wrote:

 > When is TCP buffer space allocated?  I thought things happened
 > as data flowed, but I thought someone said it happened at
 > socket creation time.

No actual buffer space is allocated when a socket is created.  Sockbufs
are chains of mbufs, which are allocated as the data flows in or out.

A "full" sockbuf is simply one where the total size of all mbufs in the
chain has reached a limit.

 > The reason I'm asking is that I have an application where literally
 > thousands (5k to 10k) TCP streams will be open at once.  I'd
 > be happy to limit the space on incoming data to 1k or 2k, and outgoing
 > to 2k or 4k.  Bandwidth isn't really an issue -- in fact, I don't
 > want any one client using much of it at all.  Small windows are
 > fine by me, with most data backed up on the remote end.
 > 
 > My questions are:
 > 
 > 	If I use setsockopt() on the socket I'm listening on,
 > 	would it make sense for sockets that I accept() from
 > 	it to aquire the same incoming/outgoing buffer space
 > 	limits?

That should work, yes.

 > 	If I set the incoming to something small, how small
 > 	can I get before I break, or seriously confuse TCP?

Well, the default is 16k for both (see net.inet.tcp.sendspace and
net.inet.tcp.rcvspace).  You certainly want it to be at least
as large as the default MSS, I would think, although I guess that's
not strictly necessary, either.  Certainly 1k or 2k as a receive window
size would not break TCP (and if it did, it's a bug).

 > 	I assume setting the incoming space to something small
 > 	will cause the TCP window to shut quickly, buffering data
 > 	on the client side.  This is fine by me, as I said.  But,
 > 	will I break much?

Yes, the receive socket buffer size is, in fact, the size of
the window when completely open (as data fills the sockbuf, the
window closes, and as the application reads data, the window opens
back up).

        -- Jason R. Thorpe <thorpej@nas.nasa.gov>