LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 01-29-2014, 01:32 AM   #1
srijivdimri
Member
 
Registered: Jun 2013
Location: India.
Posts: 35

Rep: Reputation: Disabled
Latency increases when we increase the packet size.


Hi Experts,

I am into a very weird issue. We have 10Mbps lease line connecting our servers in China and Chicago. When I do a normal ping to my Chicago server from China or Vice versa, the RTT is 161ms.

ping 10.30.49.52
PING 10.30.49.52 (10.30.49.52) 56(84) bytes of data.
64 bytes from 10.30.49.52: icmp_seq=1 ttl=64 time=161 ms
64 bytes from 10.30.49.52: icmp_seq=2 ttl=64 time=161 ms
64 bytes from 10.30.49.52: icmp_seq=3 ttl=64 time=161 ms
^C

But when I increase the packet size, the latency increases from 161 to 165ms as you can see below

ping 10.30.49.52 -s 500
PING 10.30.49.52 (10.30.49.52) 500(528) bytes of data.
508 bytes from 10.30.49.52: icmp_seq=1 ttl=64 time=165 ms
508 bytes from 10.30.49.52: icmp_seq=2 ttl=64 time=165 ms
508 bytes from 10.30.49.52: icmp_seq=3 ttl=64 time=165 ms

I have asked the Provider to increase the MTU to 1500 and still there is no improvement. The total packet size will be somewhere equal to 546 bytes(20 bytes of IP header, 8 bytes of ICMP header + 500 data 18 bytes of min ethernet frame) which is much less that 1500 bytes of MTU that we have set on out network devices like routers, switches and firewalls. I am completely stuck as to why the latency increases when we increase the packet size to 500. As we keep on increasing the packet size, the latency keeps on increasing.
Is there a way on Linux servers to discover MTU along the path (PMTU) ?

Appreciate all your help in advance.
 
Old 01-29-2014, 01:39 AM   #2
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,448

Rep: Reputation: Disabled
Of course latency increases with packet size. It has to.

Every router and switch along the path has to receive the entire packet/frame before it can forward it. The latency introduced at each point thus equals the speed of the inbound link in bps divided by the frame size in bits. Larger frames = increased latency.
 
Old 01-29-2014, 01:44 AM   #3
srijivdimri
Member
 
Registered: Jun 2013
Location: India.
Posts: 35

Original Poster
Rep: Reputation: Disabled
Thanks for looking into this. The latency should increase only when the frame exceeds the MTU configured on the network device. This is because then fragmentation will occur and reassembling of those packets will take some time. But what if the frame size is much less that the MTU configured ? In that case the complete packet will be sent, no fragmentation and no additional delays should be added. MTU stands for Maximum Transmission Unit that I can pass via Data-Link.

So if the MTU is 1500 bytes and the total frame size is say 1000, then the latency should not be there as at that point the complete frame can pass through the network device.

Is there a command to find the Path MTU on Linux ?
 
Old 01-29-2014, 01:49 AM   #4
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,448

Rep: Reputation: Disabled
Quote:
Originally Posted by srijivdimri View Post
Thanks for looking into this. The latency should increase only when the frame exceeds the MTU configured on the network device.
No, it shouldn't. Latency will increase with packet size in a linear fashion until the packet reaches the MTU limit, and then there will be a noticable jump as the packet has to be fragmented, which introduces a new header. An MTU of 1500 bytes does NOT mean that every packet is padded to that exact size.

This is why it pays off to limit the size of UDP datagrams (which as you know are never subject to fragmentation at layer 4) in VoIP applications. It reduces latency at the expense of line utilization, as the relative overhead due to IP and UDP headers increase.

Last edited by Ser Olmy; 01-29-2014 at 01:57 AM.
 
Old 01-29-2014, 04:14 PM   #5
jefro
Moderator
 
Registered: Mar 2008
Posts: 16,676

Rep: Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454
There is a ping command that tells you the max mtu. I forget what it is but it will tell you how far you can go. I think it involved using wireshark and look for split packets or something.

Unfortunately ping is a very poor test for that distance for use in speed (yes fragment still). You say leased line but the isn't a dedicated line. You have maybe a few hundred devices in between. Any one of which could change how it handles icmp.

Looking at hardware at your end or trying to switch how they handle settings may help. Many wirespeed devices could have a way to handle how packets are held until complete packet is sent or some can be switched to start sending at first bit.

Look also at your system settings for nic and interior devices. Look very closely at vpn.

Last edited by jefro; 01-31-2014 at 03:44 PM.
 
Old 01-31-2014, 01:09 AM   #6
srijivdimri
Member
 
Registered: Jun 2013
Location: India.
Posts: 35

Original Poster
Rep: Reputation: Disabled
Hi Jefro,

Thanks for looking into this. We have multiple vendor devices like Cisco Switches, Juniper and HP and all have an MTU set to 1500. I double-checked this settings. Also on my RedHat Server NIC, the MTU is set to 1500, I can confirm that.

So, from here, I have 1 question- Is it always that increasing the packet size will increase the RTT for the ping packets ? ( Assuming that the packet is much smaller than the MTU defined on the network devices ).
 
Old 01-31-2014, 08:54 AM   #7
Ser Olmy
Senior Member
 
Registered: Jan 2012
Distribution: Slackware
Posts: 2,448

Rep: Reputation: Disabled
Since packets are transmitted as a sequence of bits, and since a larger packet represents a longer stream of bits which will take longer to transmit, AND since routers (and just about all switches) operate in store-and-forward mode, the answer is a resounding "yes". If not, it would mean that big data packets could be transmitted just as fast as smaller data packets, which obviously makes no sense.

As for how much round trip time will increase if the packet size is increased by X bits, that depends on the number of hops between endpoints and the bandwidth between routers.
 
Old 01-31-2014, 09:17 AM   #8
srijivdimri
Member
 
Registered: Jun 2013
Location: India.
Posts: 35

Original Poster
Rep: Reputation: Disabled
Got it...thank you so much for all your expert advice. Appreciate that.
 
Old 01-31-2014, 03:46 PM   #9
jefro
Moderator
 
Registered: Mar 2008
Posts: 16,676

Rep: Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454Reputation: 2454
https://secure.dslreports.com/faq/5793


Thought 1500 was wrong. Might have to go lower on vpn.
 
Old 01-31-2014, 08:05 PM   #10
Shadow_7
Senior Member
 
Registered: Feb 2003
Distribution: debian
Posts: 3,073
Blog Entries: 1

Rep: Reputation: 669Reputation: 669Reputation: 669Reputation: 669Reputation: 669Reputation: 669
576 was the old standard dialup packet size. Smaller packets have better queing of packets. Since you don't have to wait for big packets to clear before yours can be sent. For those networks with many users hogging all the available resources. If you exceed the MTU of the entire journey end to end your packets will fragment and be slower since you'll have more headers.

Last edited by Shadow_7; 01-31-2014 at 08:07 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] 'cd' command increases size of shell propmt thelink123 Linux - Newbie 8 09-11-2010 09:57 AM
Socket Programming, killing threads increases memory size gauravparihar Programming 1 03-27-2009 05:25 AM
Opening,Editing MS Word document in writer increases size tuxtuxtux Linux - Desktop 3 12-29-2008 03:04 PM
Smokeping shows latency increase with new kernel! thekillerbean Linux - Software 2 04-16-2007 02:34 AM
increase point size size when printing PDF files newlin Red Hat 0 09-27-2003 07:13 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:44 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration