Subject: Re: TCP send buffer free space
To: None <gantose-work@bigfoot.com>
From: David Brownlee <abs@netbsd.org>
List: tech-net
Date: 07/09/2001 17:32:14
On Mon, 9 Jul 2001, Dave Gantose wrote:

> Yes, another reasonable suggestion. Unfortunately, I have no control over
> the client. The spec says to establish this one server socket. The master
> controller will then make a connection and accept and forward my messages
> to the (single) final destination.
>
> By the way, to the master controller, a playback packet looks the same as
> a realtime packet. So even if it were willing to help with flow control of
> playback data (which it is not), it couldn't tell the difference.
>
	I think it would help if you could get the master controller to ack:

	Example with completely arbitrary values:
	    - master controller sends ack every block
	    - you have around 10 blocks of buffer between your app and master
	    - 4 blocks write ahead keeps everything busy enough without
	      introducing too much latency for a realtime block.

	    If you have a realtime packet:
		- less than 10 blocks ahead, write it (if close to
		  10 blocks possibly log a warning)
		- at 10 blocks, log louder warning and try anyway
		  (on the assumption you can do nothing better)
	    If you have playback data:
		- less than 4 blocks ahead, write it

	I don't know if you every intermix live and playback data - the
	above should work either way.. but only if you can get some acks
	from the other end... :/

> I *could* just hardcode some slow-down factor into the playback data
> process. That is, send a little, sleep a little, send a little, sleep a
> little,... But that seems so inflexible--and inefficient if the bandwidth
> is not being used for other things.

	You could use setsockopt(...,SO_SNDBUF) to set a low output
	buffer size and rely on the fact you will only have that
	much in the local buffer, but you still have no control
	over buffering after it leaves your host.

	I'd really want to tweak the client (ideally :) or master
	controller...

	Can you put a small process on the other end to which the
	client connects and it makes the connection to the master
	controller (and handles the flow control). Not that that
	is anything like an ideal solution...

>
> Dave Gantose
> Zin Tech
>
> -- David Brownlee wrote:
>
> >     Have the client ACK every N units of playback data. Allows you to
> >     guarantee no more than N units of replay data in front of new
> >     realtime data... Tune N to taste :)
> >
> > --
> >         David/absolute    -- www.netbsd.org: No hype required --
> >
> >
> > On Mon, 9 Jul 2001, Dave Gantose wrote:
> >
> >> I would love to have two separate sockets, but this is a Space Station
> >> project and the single socket has been dictated to us from on high :(
> >>
> >> -- David Brownlee wrote:
> >>
> >> >     One option might be to have two sockets between the client and
> >> >     server - one for realtime and one for playback. Could simplify
> >> >     logic considerably :)
> >> >
> >> > --
> >> >         David/absolute    -- www.netbsd.org: No hype required --
> >> >
> >> >
> >> > On Mon, 9 Jul 2001, Dave Gantose wrote:
> >> >
> >> >> A few months ago, a question was posted here entitled "Determination of
> >> >> TCP send buffer queue", but I don't see a response in the archives, and I
> >> >> have a similar problem. To wit:
> >> >>
> >> >> SHORT DESCRIPTION
> >> >> I would like to be able to find out just how much free space is available
> >> >> in the Send buffer of my TCP socket connection at a particular time.
> >> >>
> >> >> LONG DESCRIPTION
> >> >> Alternatively, maybe someone wiser than me can tell me  what I should
> >> >> *really* do instead... I am writing a realtime data stream (from a data
> >> >> generator) to this connection much of the time. Occasionally, I "replay"
> >> >> an old data stream (from an archive file). These old messages are exactly
> >> >> the same format as the realtime ones, and they must be sent through the
> >> >> same socket connection, but since they are just being read from a file,
> >> >> they are produced much more quickly, and they tend to fill up the TCP Send
> >> >> buffer.
> >> >>
> >> >> Now I wouldn't mind some delay in the *replayed* data getting to its
> >> >> destination; using blocking I/O to write it to the socket, and letting TCP
> >> >> manage the data flow would be just fine.
> >> >>
> >> >> BUT, it is important that the *realtime* data be transmitted in a timely
> >> >> manner, and I have not used blocking when writing that data to the socket
> >> >> because I am afraid that a backup could cause bad things to happen "up the
> >> >> line" (i.e., in the data generator itself).
> >> >>
> >> >> So my thought is to only allow a replayed data message to be written to
> >> >> the TCP socket if there is at least X amount of space left in the TCP Send
> >> >> buffer. That way, I can assure there will be enough space for at least the
> >> >> next N realtime messages to be written as they come in. (I realize that if
> >> >> processing at the other end of the socket connection--over which I have no
> >> >> control--is slow enough, then I'll eventually get into trouble at my end
> >> >> anyway. I'm just trying to provide an optimum level of protection.) But
> >> >> how do I find out if there is X free space left in the Send buffer?
> >> >>
> >> >> Or what should I do instead?
> >> >>
> >> >> Thanks in advance for any help and/or advice.
> >> >>
> >> >>
> >>
> >>
> >>
> >>
>
>
>
>