could be a number of issues:
1. Do you have a firewall and is this set up for NAT and correctly?
2. Could be slow net works?
3. look in /proc/sys/net/ipv4/tcp_rmem amd tcp_wmem. I had to reduce the default values to about 25000 (in the middle value ) as it was send packets more than once
4 When you say you have set MTU, have you set this on the router AND the linux box. Should be the same. I use a value of 1430
5. Look at www.dslreports.org/tweaks
fro more linux info
6. use traceroute to look at the route to the server
Hope that is a starter