Re: TCP vs urgent data [was Re: poll(): IN/OUT vs {RD,WR}NORM]

[Johnny Billquist <bqt%softjar.se@localhost>]

On 2024-05-29 14:45, Mouse wrote:
> Should we maybe move this to tech-net?  It's no longer about poll().
>
> > > I question whether it actually works except by accident; see RFC
> > > 6093.
> > I hadn't seen that one before,
>
> Neither had I until Johnny Billquist mentioned it upthread.  (I tend to
> share your reaction to the modern IETF, though I have additional
> reasons.)

I became aware of it a bunch of years ago when I was doing my TCP 
implementation for RSX and needed to figure what, if anything I should 
do with urgent data. That RFC was nice to discover at that point. :-)

> > > But the facility it provides is of little-to-no use.  I can't recall
> > > anything other than TELNET that actually uses it,
> > TELNET and those protocols based upon it (SMTP and FTP command at
> > least).
>
> FTP command, yes.  SMTP I'm moderately sure doesn't do TELNET in any
> meaningful sense; for example, I'm fairly sure octet 0xff is not
> special.  I find no mention of TELNET in 5321.

I also have not seen anything suggesting that SMTP use the NVT model.
It's basically just the CRLF as end of line thing, and nothing more.
FTP on the other hand do use NVT (or if you want to call that telnet).
The only situation I know of when FTP uses urgent is when trying to 
abort an ongoing transfer. But it's primarily used in this case to just 
potentially wake the server up, as it might be ignoring the command 
channel while transferring data.

But I've also found this to be inconsistent, and behaving in odd ways in 
different implementations, so I would in general say that while it might 
not hurt to sent the ABOR with urgent, it's not something anyone should 
expect.

But reading RFC 959, there is no mention of using urgent data under any 
circumstances. What that RFC says about aborting is:

"Some servers may not be able to monitor the control and data connections
   simultaneously, in which case some special action will be necessary
   to get the server's attention.  The following ordered format is
   tentatively recommended:

      1. User system inserts the Telnet "Interrupt Process" (IP) signal
      in the Telnet stream.

      2. User system sends the Telnet "Synch" signal.

      3. User system inserts the command (e.g., ABOR) in the Telnet
      stream.

      4. Server PI, after receiving "IP", scans the Telnet stream for
      EXACTLY ONE FTP command.

   (For other servers this may not be necessary but the actions listed
   above should have no unusual effect.)"

Which is definitely different than for telnet, which suppsedly scans 
until a DM is found. Here instead you should start searching for the 
first command after IP. Exactly how you trigger the ftp server to even 
start this searching is left undefined, although urgent would seem the 
logical choice, if it worked.

But here again, it becomes the case that the actual urgent pointer 
information isn't really used. It appears that, just as with telnet, 
urgent is actually just used as a signal for the process to start 
reading through the stream of incoming data looking for something.

If you have an implementation that anyway is all the time accepting 
data, then urgent becomes superfluous.

So I would still argue that urgent, as described, isn't actually that 
useful as described. And I'm still left wondering what they actually 
intended with it, as it supposedly is used to mark which data the 
process receives (or sends) is intended to be urgent. But there is no 
strict correlation between what the sender then considered urgent data, 
and what the receiver reports as urgent data.

I do look at both telnet and ftp as implementations that use urgent in a 
way that isn't matching what appears to have been the intent, and is 
actually an implementation doing a workaround for a deficit in the tcp 
spec. In the end, a simpler solution would have sufficed. All that was 
needed was just a way to send a signal to the other end to wake the 
process up, informing it that it should start scanning through data that 
has been received.

> > SMTP has no actual use for urgent data, and never sends any, but FTP
> > can in some circumstances I believe (very ancient unreliable memory).
>
> Yes.  It should, according to the spec, be done when sending an ABOR to
> abort a transfer in progress.  But, unlike TELNET's specification that
> data is to be dropped while looking for IAC DM, the urgent bit can be
> completely ignored by an FTP server which is capable of paying
> attention to the control channel while a data transfer is in progress.

I don't even see any actual mention of urgent in the ftp spec. But there 
is an implication that the server needs to be notified somehow if it 
isn't usually servicing the command channel while a transfer is in progress.

> > > then botched it further by pointing the urgent sequence number to
> > > the wrong place,
> > In fairness, when that was done, it wasn't clear it was wrong - that
> > all long predated anyone even being aware that there were two
> > different meanings in the TCP spec, people just used whichever of
> > them was most convenient (in terms of how it was expressed, not which
> > is easier to implement) and ignored the other completely.   That's
> > why it took decades to get fixed - no-one knew that the spec was
> > broken for a long time.
>
> So...I guess next to nothing depended on it even then, or someone would
> have noticed the interoperability fail sooner than decades.

Nothing do depend on it (obviously). And as the rfc about urgent points 
out, most implementation tries to actually treat urgent data as OOB, and 
actually seem to keep a separate 1 byte buffer for the data provided by 
the urgent pointer. Obviously this is very broken i relation to the tcp 
spec, but that just shows even more how broken it is out in the real world.

In the end I still don't think it's clear how this was supposed to be used.

  Johnny

--
Johnny Billquist                  || "I'm on a bus
                                  ||  on a psychedelic trip
email: bqt%softjar.se@localhost             ||  Reading murder books
pdp is alive!                     ||  tryin' to stay hip" - B. Idol