Subject: Bandwidth control with ALTQ on NetBSD/alpha
To: None <port-alpha@netbsd.org>
From: Paul Mather <paul@gromit.dlib.vt.edu>
List: port-alpha
Date: 05/12/2003 11:27:50
Yesterday, I began experimenting with using ALTQ for bandwidth control
on my DEC 3000/300 running NetBSD/alpha 1.6R (-CURRENT).  Eventually,
I want to use some rules to limit outgoing BitTorrent traffic on an
AlphaServer 1000A, but I started by limiting active FTP transfers on
my 3000/300 to modem speed (as that is something that is easy for me
to test).

I found that outgoing traffic is limited correctly, but incoming
traffic---although limited to a degree---is still much higher than it
should be.  Here is the altq.conf file I used:

>>>>>
interface le0 bandwidth 10M cbq

class cbq le0 root NULL pbandwidth 100
class cbq le0 def root borrow pbandwidth 95 default

class cbq le0 ftp root exactbandwidth 33600 red
        filter le0 ftp 0 0 0 20 6
        filter le0 ftp 0 20 0 0 6
<<<<<

If I understand things correctly, this configuration should cap
traffic either to or from TCP port 20 on interface le0 to 33.6Kb/s.

What I observed was that using active FTP to another system on my LAN,
a "put" command correctly limits the transfer to < 4KB/s, but a "get"
transfers at ~118 KB/s.  Although this is capped to some degree, it is
still higher than the specified hard limit.  (Note that the ftp class
is not specified as being able to borrow bandwidth from the parent.)

Is this an artifact of ALTQ on Alpha, of ALTQ, or of my ruleset?

When I run altqd in verbose debug mode (-dv), I get the following message
output:

>>>>>
/home/paul root@hawkwind# altqd -dv -f /tmp/cbq 
ALTQ config file is /tmp/cbq
tbr is already installed on le0,
  using the current setting (rate:10.00M  size:5.86K).
cbq_flowspec: maxburst=16,minburst=2,pkt_size=1500
  nsPerByte=800.00 ns, link's nsPerByte=800.00, f=1.000
  packet time=1200 [us]

warning: maxburst smaller than timer granularity!
         maxburst=16, packet_time=1200 [us]
  maxidle=37.50 us
  offtime=0.00 us minidle=-1200.00 us
  maxq=30
cbq_flowspec: maxburst=16,minburst=2,pkt_size=1500
  nsPerByte=860.22 ns, link's nsPerByte=800.00, f=0.930
  packet time=1290 [us]

  maxidle=59.79 us
  offtime=183.56 us minidle=-1290.32 us
  maxq=30
cbq_flowspec: maxburst=4,minburst=2,pkt_size=1500
  nsPerByte=40000.00 ns, link's nsPerByte=800.00, f=0.020
  packet time=60000 [us]

  maxidle=7962.17 us
  offtime=119496.77 us minidle=-60000.00 us
  maxq=30
le0: add a filter (null) to class ctl_class
 Filter Dest Addr: 0.0.0.0 (mask 0) Port: 0
        Src Addr: 0.0.0.0 (mask 0) Port: 0
        Protocol: 1 TOS 0 (mask 0)
le0: add a filter (null) to class ctl_class
 Filter Dest Addr: 0.0.0.0 (mask 0) Port: 0
        Src Addr: 0.0.0.0 (mask 0) Port: 0
        Protocol: 2 TOS 0 (mask 0)
le0: add a filter (null) to class ctl_class
 Filter Dest Addr: 0.0.0.0 (mask 0) Port: 0
        Src Addr: 0.0.0.0 (mask 0) Port: 0
        Protocol: 46 TOS 0 (mask 0)
syscall error: add filter failed!: Invalid argument
can't add ctl class IPv6 filter on interface 'le0'
cbq_flowspec: maxburst=4,minburst=2,pkt_size=1500
  nsPerByte=238095.24 ns, link's nsPerByte=800.00, f=0.003
  packet time=357142 [us]

  maxidle=48198.62 us
  offtime=723367.74 us minidle=-357142.86 us
  maxq=60
le0: add a filter (null) to class ftp
 Filter Dest Addr: 0.0.0.0 (mask 0xffffffff) Port: 0
        Src Addr: 0.0.0.0 (mask 0xffffffff) Port: 20
        Protocol: 6 TOS 0 (mask 0)
le0: add a filter (null) to class ftp
 Filter Dest Addr: 0.0.0.0 (mask 0xffffffff) Port: 20
        Src Addr: 0.0.0.0 (mask 0xffffffff) Port: 0
        Protocol: 6 TOS 0 (mask 0)
cbq enabled on interface le0 (mtu:1500)

Enter ? or command:
altqd le0> 
<<<<<

Is the "warning: maxburst smaller than timer granularity!" significant
here?  I increased maxburst to 32 (and maxdelay to 48) on the root
class (getting rid of the warning), but still see the same behaviour.

Note that I don't have the INET6 option in my kernel config file,
which is why I get the "can't add ctl class IPv6 filter on interface
'le0'" error.

Here are the ALTQ kernel options I have in my kernel config file:

>>>>>
options         ALTQ
options         ALTQ_CBQ
options         ALTQ_HFSC
options         ALTQ_PRIQ
options         ALTQ_WFQ
options         ALTQ_FIFOQ
options         ALTQ_RED
options         ALTQ_BLUE
options         ALTQ_LOCALQ
options         ALTQ_FLOWVALVE
options         ALTQ_RIO
options         ALTQ_CDNR
<<<<<

Did I miss any I need?

Cheers,

Paul.

e-mail: paul@gromit.dlib.vt.edu

"Without music to decorate it, time is just a bunch of boring production
 deadlines or dates by which bills must be paid."
        --- Frank Vincent Zappa