iptables can do some shaping by accepting up to some maximum of connections ( see
http://www.debian-administration.org/articles/187 and
http://iptables-tutorial.frozentux.n...imit-match.txt ) using hitcount and update. There is also a limit parameter usually used for controlling logging. see man iptables
That is not very smooth because there is no proportionality. You could try writing a little programme to run periodically that would adjust the limits according to how busy the system was or you could come up with limits that would work in your particular situation, e.g. limit everyone to something reasonable and the packages that pay the bills being at a much higher limit.
Of course, someone has figured out a better way to do this, see man tc
and
http://lartc.org/