traffic shaping help

monohouse · 10-31-2008, 02:24 PM

where can I ask questions about traffic shaping in linux ? the LARTC mailing list is down and does not allow signup anymore

estabroo · 10-31-2008, 07:53 PM

What kind of traffic shaping do you want to do? Looking at the tc man page is a good start to seeing different traffic shaping ideas.

monohouse · 11-01-2008, 02:43 AM

whatever traffic shaping I want to do I believe i have already done it, what I need now is help in determining why it isn't working, you can take a look at it maybe, I believe I have finished with the man pages and kernel configuration and searching but maybe someone out there can find me the part that I missed, I posted here 3 additional topics about traffic shaping and got no replies so I figured nobody here knows anything about it, so I thought I should ask if someone knows who does

estabroo · 11-01-2008, 10:43 AM

what is it that you want to accomplish with this? It looks like in your script you have all the incoming traffic shaping commented out which seems like it would defeat the purpose of that script.

monohouse · 11-01-2008, 06:23 PM

the purpose of this script and of any script related to traffic shaping in linux is to shape the outgoing traffic, incoming traffic, at least according to the LARTC howto is not worth doing at all, and also according to common sense, are you saying that the script has no outgoing traffic shaping at all ?

what I am trying to (actually) accomplish, is to prioritize outgoing traffic only on interface ppp0 which is a pptp-client GRE connection on top of physical dhcp connection eth0 which goes to the cable modem, the script, which I wrote, that uses HTB and SFQ (I wrote at least 2 more scripts, one of them uses PRIO as well, that also fail).

from what i was able to gather so far, the classes seem to get their packets according to the rules, you can see it with

Code:

tc -s class ls dev ppp0

you can of corse also see

Code:

tc -s qdisc ls dev ppp0

and

Code:

tc -s filter ls dev ppp0

you can obviously see that iptables does mark the packets with

Code:

iptables -t mangle -L -n -v

, and as you can see in case one of the marking methods might not work for whatever reason, I have made sure to add CLASSIFY targets on the PORTROUTING to make sure that it does, so far so good.

here is the problem: no traffic shaping whatsoever, not on the shaping system and not on the masqueraded systems, I have made sure that ICMP and some UDP (as you can see in the script) traffic gets first priority by specifying prio parameter to HTB and in case there was a problem with the script I have created additional scripts which use different methods: script1 script2

if you know or see something that i have missed tell me, because I have been analyzing many other scripts like "the ultimate shaper", "myshaper for ADSL modes", "wondershaper", and I can't seem to find what is wrong, everything reports to be ok, but shaping is just not happening.

estabroo · 11-02-2008, 08:30 AM

How do you know its not traffic shaping? The script looked reasonable. Are the 1.0.0.2 and 1.0.0.4 correct (seems like maybe the were meant to be 10.0.0.2 and 10.0.0.4)?

estabroo · 11-02-2008, 08:36 AM

Since this is a tunnel over your cable channel do you have access to the far endpoint? If so you might be able to apply shaping on the incoming as well. That's actually what I do, my isp lets me colocate a small embedded linux box, so I just tunnel all my traffic to it and shape on both sides. It works great for making sure my voip phones always have bandwidth. Currently I'm using a tbf in conjunction with pfifos but your htb looks intriguing and actually nicer since it allows full bandwidth if nothing else is using it, whereas mine reserves the voip bandwidth regardless of use.

monohouse · 11-02-2008, 09:41 AM

1.0.0.2 and 1.0.0.4 are private addresses, but I think that is not necessary at all to specify addresses manually.

the first reason I know that shaping is not working is because when I do a ping on a semi-congested connection (download at full, upload more or less free or up to 50% used) I get values that are indifferent when I do a ping without the shaping.

the second, the software that uses the specified UDP ports (8452, 2074) does not indicate any improvement (the same as if without).

the third, when I had a winxp-based server, I was using cfos shaper, and the performance of both the above tests was visible (to say the least) on both the shaping system and the masqueraded connections (what's more, the connection was *FULLY* congested, not just download).

the 4th reason, is this: (from http://lartc.org/howto/lartc.cookbook.ultimate-tc.html )

Quote:

If we do all this we get the following measurements using an excellent ADSL connection from xs4all in the Netherlands:

Baseline latency:
round-trip min/avg/max = 14.4/17.1/21.7 ms

Without traffic conditioner, while downloading:
round-trip min/avg/max = 560.9/573.6/586.4 ms

Without traffic conditioner, while uploading:
round-trip min/avg/max = 2041.4/2332.1/2427.6 ms

With conditioner, during 220kbit/s upload:
round-trip min/avg/max = 15.7/51.8/79.9 ms

With conditioner, during 850kbit/s download:
round-trip min/avg/max = 20.4/46.9/74.0 ms

the 5th reason, is that there should be *SOME* change, ANY change at all.

I can do tests again to show you, but I have already posted those before

btw, at one time, I had the ENTIRE mangle table along with all the chains marked the same way as the POSTROUTING table, and that also had no effect.

I have made significant changes to the kernel, and recompiled at least 40 times in the last month to get it working, but no matter what I do in the kernel, it has no effect, according to some manuals I have read so far, all the *NECESSARY* options that need to be in the kernel for shaping to work are already present, but I was afraid something too obvious might have been missed that noone mentioned in any of the guides (for example the advanced linux traffic control and routing howto), but recently I am beginning to think that is not the case, and that the kernel is fine, I also thought that ppp filtering has to be enabled, but that had no effect either.

as for incoming shaping, it's too early for that because nothing is working at the moment at all, when I see that something does I will see what I can do about that, but considering that I am not a company with phones and $$$ I doubt they will care, and if i did have the $$$ I would go for a business connection in the first place, and in that case I would have warmer relations with the ISP considering the $$$ they get. but that will not be necessary, because if it worked fine with cfos it should work better with linux.

estabroo · 11-02-2008, 10:36 AM

Do you have the data being routed over the ppp interface? Move the icmp traffic to 20, move the dns traffic to 25, that'll help make sure you aren't going over the rate for a given section. In myshaper.sh you are splitting the bandwidth equally, you might want to change that distribution give more bandwidth to 21 which has your special traffic and reduce the rest to guarantee its minimum. If you have other traffic on happening on the cable line that is not over ppp then it can make the ppp traffic shaping useless since it can use all the bandwidth without regard to the ppp.

monohouse · 11-02-2008, 02:53 PM

Quote:

Move the icmp traffic to 20, move the dns traffic to 25

I will do that, and see what happens, however, it shouldn't be necessary for 2 reasons: the first is the prio parameter given to the HTB, it means priority, I would suggest that the priority value represents what it should, and that the lower is the priority value, the sonner that traffic gets out, the other reason, is the other script(s), as you can see one of the other scripts I wrote uses the PRIO algorithm, which means ANYTHING that is located in a higher priority BAND will be sent out first regardless of any rules, that is the original and real algorithms I actually intend to use, I switch to the HTB only because I found the PRIO script to not work.

for what you are suggesting, I am going to take it a abit further, im gonna move ALL traffic to 26 and icmp to 20, but i am almost certain that is futile.

as for the cable interface, if that were the case (as it surely is not, there is NO other traffic on it) it doesn't matter, because in that case the entire ppp connection would have the problem and the same rules would apply, because the more important packets are still sent first.

the data being routed over the ppp interface, is in the POSTROUTING, and everything there is either CLASSIFY or MARK, and both do not work.

estabroo · 11-02-2008, 03:14 PM

Can you verify that you are sending it over the ppp interface? It could be its not working because no data is actually going out the ppp interface.

monohouse · 11-02-2008, 04:38 PM

that's not possible, it's the only way - and the connection works, nothing works without the tunnel and tc -s class ls dev ppp0 says that it's all there, as well as iptables -t mangle -L -n -v in the POSTROUTING chain

it's something else, I just can't figure out what, do you have a script ? and did you have to do anything special to get it working ? maybe show me your script maybe there is something there, I am using 2.6.26.5 kernel, soon the new version, but I have doubts that this is really the problem

estabroo · 11-02-2008, 05:42 PM

Mine is much simpler and doesn't use any iptables marking

Code:

#!/bin/sh

/sbin/tc qdisc add dev $1 root handle 1: prio priomap 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 0
/sbin/tc qdisc add dev $1 parent 1:1 handle 10: pfifo
/sbin/tc qdisc add dev $1 parent 1:2 handle 20: pfifo
/sbin/tc qdisc add dev $1 parent 1:3 handle 30: tbf rate 500kbit burst 5kb latency 200ms
/sbin/tc filter add dev $1 protocol ip parent 1: prio 1 u32 match ip dst <ip_of_voip_server1> flowid 1:1
/sbin/tc filter add dev $1 protocol ip parent 1: prio 1 u32 match ip dst <ip_of_vpn_server> flowid 1:1
/sbin/tc filter add dev $1 protocol ip parent 1: prio 1 u32 match ip dst <ip_of_voip_server2> flowid 1:1
/sbin/tc filter add dev $1 protocol ip parent 1: prio 1 u32 match ip dst <ip_of_voip_server3> flowid 1:1

I have something similar on the incoming side but it does a src match instead of a dst match.

I'll put together one using htb similar to your myshaper script and see if it works for me and I'll post it here if it does (might not get to it until tomorrow evening though)

estabroo · 11-02-2008, 05:51 PM

If you do a iptables -L -v -n -t mangle do the counters show your marking rules being hit?

monohouse · 11-02-2008, 07:18 PM

yes, of corse