ifconfig reports millions of dropped RX packets

b00n · 07-11-2009, 02:23 PM

Hello,

I've got a problem on my linux (web) server it reports millions of dropped RX packets
e.g.:

eth0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:794033 errors:0 dropped:1649413175 overruns:0 frame:0
TX packets:1844998 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:50672248 (48.3 MiB) TX bytes:2740995502 (2.5 GiB)
Interrupt:50 Base address:0x8000

The uptime is 44 minutes and a couple of seconds later the output says:

eth0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:815926 errors:0 dropped:770327332 overruns:0 frame:0
TX packets:1874908 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:52075806 (49.6 MiB) TX bytes:2784506627 (2.5 GiB)
Interrupt:50 Base address:0x8000

So the dropped rx packets keep truncating frequently...
Now after a day the ping times go up to 3 seconds and only a reboot will help it...
Since Im rather new to all of this I have no idea whats goin on...
I thought it may be the httpd but stopping it doesnt change anything.

It would appear that this started with the updated kernel 2.6.18-128.1.16.el5...

frankbell · 07-11-2009, 02:45 PM

I just checked my webserver:

Code:

eth0      Link encap:Ethernet  HWaddr 00:11:11:a9:7d:8d  
          inet addr:[masked]  Bcast:[masked]  Mask:[masked]
          inet6 addr: fe80::211:11ff:fea9:7d8d/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:392615 errors:0 dropped:0 overruns:0 frame:0
          TX packets:631561 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:116091537 (110.7 MiB)  TX bytes:385239874 (367.3 MiB

15:43:32 up 3 days,  4:38,  1 user,  load average: 0.14, 0.07, 0.01

A certain number of dropped packets and collisions is inherent in TCP/IP. The protocol is designed to deal with them, but the millions you report is certainly scewy.

Could you tell us some more about your computer's specs, your website (is it using a database like a blog or is it plain HTML), and your connection (I assume you are self-hosting).

b00n · 07-11-2009, 05:21 PM

22:16:25 up 3:47, 1 user, load average: 0.00, 0.00, 0.00

And now if I ping it I get
Minimum = 943ms, Maximum = 2403ms, Average = 1957ms

Its a dual core atom with 2gigs of ram running an apache + php + mysql on CentOS 5.3
It also has subversion and a trac integration in apache but thats not being used atm...

Code:

top - 22:19:52 up  3:51,  1 user,  load average: 0.14, 0.04, 0.01
Tasks: 122 total,   1 running, 121 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.2%us,  0.2%sy,  0.0%ni, 99.5%id,  0.2%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   2065732k total,  2004068k used,    61664k free,    18928k buffers
Swap:  2031608k total,        0k used,  2031608k free,  1794732k cached


             total       used       free     shared    buffers     cached
Mem:          2017       1957         59          0         18       1753
-/+ buffers/cache:        185       1831
Swap:         1983          0       1983

So everything is looking pretty fine from what I can tell...

lazlow · 07-11-2009, 05:59 PM

Fall back to the .14 kernel and see if the issue persists. It could be an issue with the .16 kernel or it could be a hardware issue (atom motherboards are known to have unusually high failure rate).

b00n · 07-11-2009, 06:34 PM

Okay so I did go back to 2.6.18-92.1.22.el5 and it's working fine again!
Thanks for your help!

lazlow · 07-11-2009, 06:50 PM

Ok, so we know it is not a hardware issue. Now the question is when (kernel wise) did it pop up. Obviously someplace between -92.1.22 and -128.1.16(-128,-128.1.1, -128.1.6, -128.1.10, -128.1.14). As soon as you can figure that out you can file a bug report and hopefully get the issue addressed.

b00n · 07-12-2009, 07:07 AM

Actually 2.6.18-128.1.16.el5 is the kernel I got after 2.6.18-92.1.22.el5. Didn't have anything in between so I can't really tell...

lazlow · 07-12-2009, 01:06 PM

All the other kernels are still in the repo.