limiting upload with tc

i have a linux router for a LAN that has an interface named "external" which is the public-facing address. I am trying to set a global max upload rate for a client with local ip 172.16.16.11 to 2mbit, and if the upload link is congested, have it slow down to a minimum of 900kbps.

I do this:

tc qdisc add dev external root handle 1: htb default 1
tc class add dev external parent 1: classid 1:2 htb rate 905kbit ceil 2.15mbit

and this:

iptables -t mangle -A POSTROUTING -o external -s 172.16.16.11 -j CLASSIFY --set-class 1:2

Except the client ALWAYS has 900kbps upload - the speed never reaches the ceil value, even when the upload link is free from all other traffic.

Why isn't this working as I think it should? or am i (most likely) misunderstanding this...

thanks

You've got a rate of 905K which is the maximum rate this class and all its children are guaranteed. You have a ceiling rate of 2.15M which is the maximum rate at which a class can send, if its parent has bandwidth to spare.

What you also need to define is the amount of bytes that can be burst at ceil speed, in excess of the configured rate. The docs don't say what the default is if you don't specify it, but if the burst defaults to 0, you would get this behaviour.

Try adding a burst size of 905K and see what that does.

So adding that burst value produced a very strange result when doing a speed test via speedtest.net. The upload test took a very long time to begin, immediately jumped up to 6 megabits (???) and, finally, roughly 75% of the way through the test, dropped down to 900k. WTF??? I have a ~10mbps upload at this site, and nowhere in my rules am i specifying anything about 6 megs...

I gotta say, the man page for tc is sorely lacking, at least on an "understandability" front. I can't even say that I could explain what the "burst" value exactly does....

Actually, reading your post again, htb is not actually the thing you want, it is basically a traffic-shaper. It is unconditional in relation to other traffic.

What you need is cbq. This allows you to manually define traffic classes, then guarantee how much each gets. This means that under congestion conditions bandwidth is allocated as you specify, but under non-congestion conditions either class can use more.

Have a look at the tc cbq man page and see of it makes sense.

I gotta say, the man page for tc is sorely lacking, at least on an "understandability" front. I can't even say that I could explain what the "burst" value exactly does....

right. much of the linux documentation is still a mish-mash of howtos and half-intelligible man pages.
your issue was not in the burst value, but rather in setting up a container class which would allow for
bandwidth borrowing. try this.

IF_INET=external

# upload bandwidth limit for interface
BW_MAX=2000

# upload bandwidth limit for 172.16.16.11
BW_CLIENT=900


# first, clear previous settings
tc qdisc del dev ${IF_INET} root

# top-level htb queue discipline; send unclassified data into class 1:10
tc qdisc add dev ${IF_INET} root handle 1: htb default 10

# parent class (wrap everything in this class to allow bandwidth borrowing)
tc class add dev externel parent 1: classid 1:1 htb \
  rate ${BW_MAX}kbit ceil ${BW_MAX}kbit

# two child classes
#

# the default child class
tc class add dev ${IF_INET} parent 1:1 \
  classid 1:10 htb rate $((${BW_MAX} - ${BW_CLIENT}))kbit ceil ${BW_MAX}kbit

# the child class for traffic from 172.16.16.11
tc class add dev ${IF_INET} parent 1:1 \
  classid 1:20 htb rate ${BW_CLIENT}kbit ceil ${BW_MAX}kbit

# classify traffic
tc filter add dev ${IF_INET} parent 1:0 protocol ip prio 1 u32 \
  match ip src 172.16.16.11/32 flowid 1:20

awesome thanks for that script, i'll try messing with that when I get home tonight. one thing, you classified the traffic with tc filter and not iptables; does that make a difference? i wouldn't think so...

i used tc for the classification because it's easier to keep track of the traffic shaping configuration when everything is in one place. i don't know the pros and cons of using iptables, instead, for the classification. There may be times when classifying with tc is not an option, e.g. I believe (someone please correct me if I'm wrong) that classifying based on the presence of a MARK (which is logical rather than something written in the packet itself) is not possible with tc, but is possible with iptables.

that sounds right, seeing as MARK is an iptables option and, AFAIK, not a native tc option.