Linux - NetworkingThis forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
I'm a home user with a Laptop running Fedora and a Desktop running Debian.
The laptop is running 24/7 and I'm always downloading something from legaltorrents.com using rTorrent which is set to use all the available bandwidth. I don't wanna set a permanent limit on this.
The problem is that my connection is 512Kbps so it makes browsing difficult when torrents are sucking up all the bandwidth (that's normal, I know) but I'm not browsing 24/7.
Is there some sort of realtime bandwidth management apps (free or commercial) that will put a limit on rTorrent only when I'm browsing and then set it to use max bandwidth again when I finish browsing. It would be great if I could make it use all my available bandwidth when I load a youtube link and half the bandwidth when I visit other sites. The rest of the time, it can allow rtorrent to suck all the bandwidth.
It should also prioritize DNS (I'm already running dnsmasq for local caching)
In your case, I think, only "traffic contol = tc" "man tc" can help.
It is not dynamic, but I believe can give to some traffic flow more priority.
You need to find good manual.
Yes, tc is the thing, along with maybe the CFQ queueing discipline (a kernel module).
The man page for tc was rather sparse last time I looked, and really good documentation was rare too, but there are some examples of using it, floating around the net.
As to it being "dynamic", I'm not sure what that might mean precisely in this context, but, like nimnull suggests, it can allow you to give priority to one traffic or another, for example:
: you have 512 units to deal with (kbits, mbits, whatever...)
: you allot a max of 512 kbits to bittorrent
: you allot a max of 256 kbits to http/https/whatever you use for browsing.
:: you give the browsing protocols priority
The above pseudo-example says:
A) "if I'm browsing, give me my <=256 kbits FIRST, before allotting whatever is left, to bittorrent"
B) "If I'm NOT browsing, bittorrent can have all the bandwidth."
This allows bittorrent to suck the network dry if it's the only thing using it, BUT once you start browsing, you get priority over the bandwidth, up to 256<units>, so bittorrent will still run, just not as fast.
Sorry for the vague pseudo code but if you want a script example, which I have <somewhere> just ask, then give me some time to locate it.
You can install wondershaper, which uses tc, and is pre-configured. You can edit the config file to optimize for various dynamic traffic scenarios. http://lartc.org/wondershaper/
Thanks for the tip, everyone. I'm running wshaper.hfsc right now. I'll report back after a day or two of using it and let you guys know how it's working out for me. I can notice the improvement on DNS already.
My rtorrent accepts connections on port 33333, but when it makes outgoing connections to other peers, it will connect to whatever port they're using. So it won't set the lowest priority on those connections but instead categorize it as DATA, and incoming connections categorized as P2P right? It's still an improvement, though.
@GrapefruiTgirl: Yes, please. It'll be great to look at your script too.
OK, since you ask, I'll dig it up and post it for you. I think it's on my firewall machine which currently has no net connection, so be patient and I'll post it a bit later on. Remember, it's not very sophisticated -- but you can examine it to at least see the syntax of how to use tc.
I've left in the /proc stuff I also had this script doing, as well as the qdisc setup.
NOTE: I earlier mentioned the CFQ (Completely Fair Queueing) discipline, but I was mistaken (like I said (I think) I haven't used this in some time ) -- the qdisc (Queueing Discipline) I actually was using is the HTB module (Hierarchical Token Bucket) instead.
The way this script works, is I would have my firewall call it after the firewall was up. This script takes two arguments:
$1 is the external internet-connected interface on which to place the qdiscs, and...
$2 is a start/stop/restart argument.
So like:
shell# ./scriptname <nic> [start|stop|restart]
I used this with a dial-up connection mostly (ppp0) which had such low bandwidth that using this was almost useless -- keyword being "almost" -- because it DID actually help the two machines sharing the dial-up connection, to not starve each other.
Also, another key benefit of this setup, is that by regulating the consumed bandwidth on MY end such that it stayed *just* below the maximum possible bandwidth that our connection from the ISP could handle, it caused downloads (file transfers) to go at a nice steady rate from start to end, rather than that effect we often see where the transfer starts FAST, then gets slow-slow-slow, stops for a second, than goes FAST again..... It does this be 'teaching' the ISP that you only want traffic to arrive at a specified MAX rate and not to exceed that rate. This results in the ISP NOT overloading the connection, pausing, resending dropped packets, repeat....
Here it is:
OH, and P.S. for clarity -- the 192.168.0.88 address was my desktop machine, connected to eth0 on the firewall box on which this script ran, and 192.168.2.88 was for my roommates machine, connected to eth1 on the firewall box.
Code:
#!/bin/sh
# /etc/rc.d/rc.tcpip_proc_customize
#!/bin/bash
#
# to execute:
# <scriptname> <dev> <start|stop|restart>
# example: bash# ./script ppp0 start
#
configuration_start() {
echo "Setting /proc/ipv4 custom configuration, activating qdiscs."
if [ -f /proc/sys/net/ipv4/ip_forward ]; then
echo "Activating IPv4 packet forwarding."
echo 1 > /proc/sys/net/ipv4/ip_forward
fi
if [ -f /proc/sys/net/ipv4/ip_dynaddr ]; then
echo "Activating IPv4 dynamic ip address."
echo 1 > /proc/sys/net/ipv4/ip_dynaddr
fi
if [ -f /proc/sys/net/ipv4/tcp_sack ]; then
echo "Activating IPv4 selective ACK."
echo 1 > /proc/sys/net/ipv4/tcp_sack
fi
if [ -f /proc/sys/net/ipv4/tcp_keepalive_time ]; then
echo "Setting IPv4 TCP-keepalive sends to every 30 minutes."
echo 1800 > /proc/sys/net/ipv4/tcp_keepalive_time
fi
if [ -f /proc/sys/net/ipv4/tcp_keepalive_intvl ]; then
echo "Setting IPv4 TCP-keepalive re-send unACKeds to 30 second intervals."
echo 30 > /proc/sys/net/ipv4/tcp_keepalive_intvl
fi
if [ -f /proc/sys/net/ipv4/tcp_keepalive_probes ]; then
echo "Setting IPv4 TCP-keepalive repeat re-send unACKeds to 20 retries."
echo 20 > /proc/sys/net/ipv4/tcp_keepalive_probes
fi
if [ -f /proc/sys/net/ipv4/tcp_slow_start_after_idle ]; then
echo "Setting IPv4 TCP-slow start after idle, to false."
echo 0 > /proc/sys/net/ipv4/tcp_slow_start_after_idle
fi
# Begin HTB qdisc configuration:
# first define our qdiscs
# here is the main one, the root qdisk which is on ppp0
# the handle is 1:
tc qdisc add dev $1 root handle 1: htb
# now, the parent class (handle 1:1) is the main class on qdisc 1: above
# and has available the full bandwidth
tc class add dev $1 parent 1: classid 1:1 htb rate 2.5kbps burst 4k
# the 4 classids below are supposed to share the bw of the parent class
# eth0 classes:
# classid 1:10 has a lower priority 2 and lower avail bw; will be for incoming ftp
# classid 1:11 has higher priority 1, and higher avail bw; will be for other (http etc..) incoming
tc class add dev $1 parent 1:1 classid 1:10 htb rate 1.26kbps ceil 2.5kbps prio 3
tc class add dev $1 parent 1:1 classid 1:11 htb rate 1.26kbps ceil 2.5kbps prio 2
# eth1 classes, same idea as above on eth0
tc class add dev $1 parent 1:1 classid 1:12 htb rate 1.26kbps ceil 2.5kbps prio 3
tc class add dev $1 parent 1:1 classid 1:13 htb rate 1.26kbps ceil 2.5kbps prio 2
tc qdisc add dev $1 handle ffff: ingress
tc filter add dev $1 parent ffff: protocol ip prio 1 u32 \
match ip src 0.0.0.0/0 police rate 2.5kbps burst 4k drop flowid :1
# all routes get stochastic fairness: (what does that mean??)
tc qdisc add dev $1 parent 1:10 handle 10: sfq perturb 10
tc qdisc add dev $1 parent 1:11 handle 11: sfq perturb 10
tc qdisc add dev $1 parent 1:12 handle 12: sfq perturb 10
tc qdisc add dev $1 parent 1:13 handle 13: sfq perturb 10
# prioritize ACK packets: put ACKs into the 'interactive' class
# which means they are given high priority; this increases responsiveness
# when using www or similar services. ACK traffic gets bw directly from the main parent
# class (classid 1:1) hence the "flowid 1:1"
tc filter add dev $1 parent 1: protocol ip prio 1 u32 \
match ip protocol 6 0xff \
match u8 0x05 0x0f at 0 \
match u16 0x0000 0xffc0 at 2 \
match u8 0x10 0xff at 33 \
flowid 1:1
# put eth0 ftp traffic traffic into lower priority eth0 category
tc filter add dev $1 protocol ip parent 1:0 prio 3 u32 \
match ip dst 192.168.0.88 \
match ip protocol 6 0xff \
flowid 10:
# all other eth0 traffic into higher priority
tc filter add dev $1 protocol ip parent 1:0 prio 2 u32 \
match ip dst 192.168.0.88 \
flowid 11:
# and repeat same for eth1:
tc filter add dev $1 protocol ip parent 1:0 prio 3 u32 \
match ip dst 192.168.2.88 \
match ip protocol 6 0xff \
flowid 12:
tc filter add dev $1 protocol ip parent 1:0 prio 2 u32 \
match ip dst 192.168.2.88 \
flowid 13:
# Ingress policing; This here accepts packets at near the maximum possible
# rate that our ppp0 connection can handle, and drops everything it cannot handle;
# meanwhile it tells the other end (the ISP) that we can only receive data at
# *this* rate, and so they should slow down their transmission and not try to queue
# so much data and jam it over the line to us because we will only drop it anyway.
# This polices ALL inbound traffic on ppp0 and keeps it at or near our max
# receiving rate.
}
configuration_stop() {
echo "rc.optimize_tcpip: restoring default /proc/ipv4 settings, removing qdiscs."
if [ -f /proc/sys/net/ipv4/ip_forward ]; then
echo 0 > /proc/sys/net/ipv4/ip_forward
fi
if [ -f /proc/sys/net/ipv4/ip_dynaddr ]; then
echo 0 > /proc/sys/net/ipv4/ip_dynaddr
fi
if [ -f /proc/sys/net/ipv4/tcp_sack ]; then
echo 0 > /proc/sys/net/ipv4/tcp_sack
fi
if [ -f /proc/sys/net/ipv4/tcp_keepalive_time ]; then
echo 7200 > /proc/sys/net/ipv4/tcp_keepalive_time
fi
if [ -f /proc/sys/net/ipv4/tcp_keepalive_intvl ]; then
echo 75 > /proc/sys/net/ipv4/tcp_keepalive_intvl
fi
if [ -f /proc/sys/net/ipv4/tcp_keepalive_probes ]; then
echo 9 > /proc/sys/net/ipv4/tcp_keepalive_probes
fi
if [ -f /proc/sys/net/ipv4/tcp_slow_start_after_idle ]; then
echo 1 > /proc/sys/net/ipv4/tcp_slow_start_after_idle
fi
# clean existing down- and uplink qdiscs, hide errors
tc qdisc del dev $1 root 2>&1 1> /dev/null
tc qdisc del dev $1 ingress 2>&1 1> /dev/null
}
# Reconfig the configuration:
configuration_restart() {
configuration_stop $1
sleep 1
configuration_start $1
}
if [ ! "$1" -o ! "$2" ]; then
echo "rc.optimize_tcpip: missing argument-- please specify <dev> [start | stop | restart]"
exit 1
fi
case "$2" in
'start')
configuration_start $1
;;
'stop')
configuration_stop $1
;;
'restart')
configuration_restart $1
;;
*)
echo "rc.optimize_tcpip: please specify $1 [ start | stop | restart ]"
;;
esac
#EOF
If you have questions, I'll do my best to answer them, but again, I haven't messed with this in quite a while, so I'm kinda rusty.
Sasha!
Last edited by GrapefruiTgirl; 01-10-2010 at 09:17 PM.
Cigarette, regarding the wondershaper or wshaper.hfsc scripts, they need to be tweaked in order to work. The basic procedure is that you need to determine your download and upload speeds, so that you can enter them into the script. This is important, particularly so that the upload queue in your cable/DSL modem is never filled. If it is, it will exhibit the characteristics that Sasha describes. The way that you want to determine this is by NOT yet using the script, go to a download/upload internet speed test, and test your upload and download speeds. When you get this well-identified, then enter those into the scripts (or in the case of wondershaper, you need to include it on the command line). If you want to see the latency improvement that the scripts will provide, then before the scripts are active, open another shell and ping somewhere while the upload speed test is running. You will see horrible results. Now start up the wshaper.hfsc script. Run the upload speed test again; the upload speed, if you tweaked it correctly, will be slightly slower (maybe 80-90%), but the pings will be MUCH faster and more consistent. The purpose of the scripts is to slightly slow down your regular speeds, so that the modem's buffers are not filled, and so that you have control over which types of traffic/services in your LAN have access to the bandwidth. As you have already done, look for the ports, etc. that your services are running on and figure out which priority these services will get. If you want a bit more of a description of the hfsc discipline (hierarchical fair service curve), a good description is at: http://linux-ip.net/articles/hfsc.en/ and a description of the various queueing approaches is at: http://linux-ip.net/articles/Traffic...ul-qdiscs.html
Cigarette, even though you don't have control over the destination ports for your torrent peers, do you have control over the source port? If so, I believe you can specify that so that you can get the torrent traffic in the lowest priority category.
I got interested in implementing traffic control on my LAN when my son, who is an avid gamer, was complaining about the latency of his games, especially if I was doing a lot of downloading. Using hfsc with the proper tweaks, he now consistently hosts games (the host is the one that the internet game server determines has the best latency among the participants in an online game). There is no slowdown while several others in the family are surfing the net, or even if I am downloading a large file, or running torrents. It seems to run as advertised (believe me, I hear about it if the latency goes up).
Regards, Jeff
Cigarette, I did a little looking around at posts, and this post is from 2006, so I don't know if rtorrent has upgraded its capability to specify source ports, but:
http://osdir.com/ml/network.bit-torr.../msg00037.html
quoting the post:
"I don't think rtorrent has an option to do this. You can restrict which
port it _listens_ on for incoming connections by using the -p option
Viktor referred to, or port_range in the configuration file.
You can also have rtorrent bind _outgoing_ (as well as listening)
connections to a specific IP (-b, bind), so you might be able to
simulate the behaviour you're after by adding an additional IP to your
local system and getting rtorrent to use that; then, allow that IP to
make connections to the internet.
The Linux netfilter suite (iptables) can also permit/deny traffic based
on the local process ID amongst other factors; so you could potentially
set up the local firewall to allow the rtorrent process to connect to
whatever it wants."
If you want, you can specify a different IP address (for example, bind eth0:1 to another LAN address) and then add rules in wshaper.hfsc to lower the priority of that IP address.
If you want to, you can tag packets associated with the rtorrent PID using iptables for a specific TOS value, and the wshaper script can be amended to deal with packets of a given TOS value differently. You need the iptables OWNER match for that, described in the document located here: http://www.frozentux.net/iptables-tu...tml#OWNERMATCH
The first question to answer is, is getting the rtorrent upload traffic moved from the data category to the lowest priority category important enough to work one of the above solutions? You still have general surfing and any required low latency applications getting priority now.
...
The Linux netfilter suite (iptables) can also permit/deny traffic based
on the local process ID amongst other factors; so you could potentially...
That gave me an idea.. Running rtorrent as a unique user (create a user/group account for it) would allow iptables to match traffic specific to that user.
If you want to, you can tag packets associated with the rtorrent PID using iptables for a specific TOS value, and the wshaper script can be amended to deal with packets of a given TOS value differently. You need the iptables OWNER match for that, described in the document located here:
No, that was removed from IPTABLES, all you can do is:
owner
This module attempts to match various characteristics of the packet creator, for locally gener-
ated packets. This match is only valid in the OUTPUT and POSTROUTING chains. Forwarded packets
do not have any socket associated with them. Packets from kernel threads do have a socket, but
usually no owner.
[!] --uid-owner username
[!] --uid-owner userid[-userid]
Matches if the packet socket's file structure (if it has one) is owned by the given user.
You may also specify a numerical UID, or an UID range.
[!] --gid-owner groupname
[!] --gid-owner groupid[-groupid]
Matches if the packet socket's file structure is owned by the given group. You may also
specify a numerical GID, or a GID range.
[!] --socket-exists
Matches if the packet is associated with a socket.
It looks like kernel developers do not really think|care about security from "INSIDE".
I wasn't aware that the OWNER match had been amended. Thanks
To summarize a couple of approaches that can work:
- have the rtorrent traffic run from a separate IP address, and within the wshaper script specify that traffic from this address gets lower priority;
- have the rtorrent processes run using a different UID/GID and then using the iptables OWNER match, tag the packets with a specific TOS value that can be handled as lower priority traffic within the script.
Last edited by jeff_k; 01-14-2010 at 05:28 PM.
Reason: correction of spelling
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.