Re: A new network interface send queue API

[Greg Troxel <gdt%lexort.com@localhost>]

Jason Thorpe <thorpej%me.com@localhost> writes:

>>> ==> void ifq_fini(struct ifqueue *);
>>> Finalize and tear down an interface queue.  For ifp->if_snd, this is done
>>> for you in if_detach().
>> 
>> Presumably this can sleep and involves deleting all packets and waiting
>> for locks.
>
> Yes, it can block to acquire locks and purges the queue.

would be good to note.  locking/sleeping is always hard.

>> This staging concept is a requirement on queueing disciplines to have
>> the ability to pick a next packet within the queue and maintain that
>> choice.  Clealry one can do this with a queue and an on-deck slot, but
>> if you're tols the queue is 10 packets, that's a 9 packet queue and the
>> on-deck slot.
>
> The current implementation has an on-deck slot, and that slot is not
> counted against the maximum queue length.  I could change that to make
> the on-deck slot have cost if people feel strongly about it.

I think it's important.   When setting up a queue with a max size, I
expect that to be the max number of packets that can be in the queue.  I
view the staging slot as in the queue, even if the "this is next" label
has been firmly affixed.

The alternative is to be clear that the API is for a queue with an
associated on-deck slot that is not part of the queue, and that the
normal path is to select an on-deck packet and transmit it immediately,
so that absent failures, the on-deck slot is almost always empty.

This requires if_stage to be expresed as "select a packet from the queue
for transmission, removing it from the queue and placing it in the
on-deck slot".

This reminds me that I didn't like _restage, as that implies "do the
staging process again", but it's totally different.  How about

  if_next_select (if_stage)
  if_next_replace (if_restage)

or ondeck instead of stage.

> You could call ifq_get() after ifq_stage(), but I wanted to have what I see as 2 distinct and clear patterns for usage:
>
> 1- stage + commit/abort
>
> 2- get

I would be inclined to say that you can do 1 or you can do 2, but get
while a packet is in the on-deck slot is an error.

> “Commit” transfers ownership of the packet to the caller, which is now
> responsible for freeing the packet when they’re done using it.
> “Abort” clears the staging area and frees the packet.  It would be
> used if there’s something fundamentally wrong with the packet that
> makes it non-transmittable, yes.

OK - would be good for that to be really obvious for people reading the
interface.  My overall view is that these things are harder than most
people realize.  Not you of course, but most people reading the API spec
and trying to use it correctly.