LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
Reload this Page [SOLVED] How do I diagnose a Network problem? CentOS 6.3
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
Page 1 of 2 1 2
  Search this Thread
Old 03-17-2014, 10:26 AM   #1
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Rep: Reputation: 0
How do I diagnose a Network problem? CentOS 6.3

I've dabbled in CentOS for years, and have never figured out a number of things. Today, help me to understand how to diagnose Networking. I have a production server sitting in a server farm: epsilon.scubaboard.com. It's hosting a few not-so-important web sites, but I do need to get it up and running quickly. I was playing around on a non-production server beta.scubaboard.com which I could only ssh into on the local IP. (192.168.2.20) Anyway, I was reading about how to implement NetworkManager and yum installed the beast. Only I wasn't on beta. The connection had closed and I was on epsilon. Everything ground to a halt and ultimately I could only ssh into it from another local server. I played around with a number of settings, mostly leaving them intact. I did play with resolv.conf and I don't remember any others. It was about this time I realized that I had inadvertently installed NetworkManager and yum remove NetworkManager seemed to resolve everything. I could ssh in on a public IP, I could ping out and all the sites were back up: Yay! Then I rebooted the server and it all went to hell.

Right now, I want to figure out why I can't ping 8.8.8.8. Somehow I have stopped the ability of the server to see the rest of the interwebs. Here are some test results:

The non-production server has a similar problem. If I fix the one, the other issue should be obvious.

Don't talk to me of resolv.conf this is not a DNS issue. Here's my ifcnfg -all output:

Code:
eth0 Link encap:Ethernet HWaddr 00:30:48:35:2A:32
inet addr:192.168.2.70 Bcast:192.168.2.255 Mask:255.255.255.0
inet6 addr: fe80::230:48ff:fe35:2a32/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:29336 errors:0 dropped:0 overruns:0 frame:0
TX packets:1403 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:2216036 (2.1 MiB) TX bytes:234825 (229.3 KiB)
Interrupt:18 Memory:d8000000-d8020000

eth1 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:69.28.64.172 Bcast:69.28.64.175 Mask:255.255.255.240
inet6 addr: fe80::230:48ff:fe35:2a33/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:17242 errors:0 dropped:0 overruns:0 frame:0
TX packets:17116 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1497034 (1.4 MiB) TX bytes:719088 (702.2 KiB)
Interrupt:19 Memory:d8020000-d8040000

eth1:1 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.25.243 Bcast:209.208.25.247 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:2 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.25.244 Bcast:209.208.25.247 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:3 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.35 Bcast:209.208.92.63 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:4 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.25.246 Bcast:209.208.25.247 Mask:255.255.255.248
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:6 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.37 Bcast:209.208.92.63 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:7 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.38 Bcast:209.208.92.63 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:8 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.39 Bcast:209.208.92.63 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:9 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.40 Bcast:209.208.92.63 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:10 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.36 Bcast:209.208.92.47 Mask:255.255.255.240
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

eth1:11 Link encap:Ethernet HWaddr 00:30:48:35:2A:33
inet addr:209.208.92.41 Bcast:209.208.92.63 Mask:255.255.255.224
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
Interrupt:19 Memory:d8020000-d8040000

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:99933 errors:0 dropped:0 overruns:0 frame:0
TX packets:99933 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8304064 (7.9 MiB) TX bytes:8304064 (7.9 MiB)
Code:
Shutting down interface eth0: [ OK ]
Shutting down interface eth1: [ OK ]
Shutting down loopback interface: [ OK ]
Bringing up loopback interface: [ OK ]
Bringing up interface eth0: Determining if ip address 192.168.2.70 is already in use for device eth0...
[ OK ]
Bringing up interface eth1: Determining if ip address 69.28.64.172 is already in use for device eth1...
[ OK ]
 
Old 03-17-2014, 10:33 AM   #2
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
One more thing:

Code:
 netstat -ntulp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name
tcp        0      0 0.0.0.0:587                 0.0.0.0:*                   LISTEN      1678/master
tcp        0      0 0.0.0.0:110                 0.0.0.0:*                   LISTEN      1574/dovecot
tcp        0      0 127.0.0.1:3310              0.0.0.0:*                   LISTEN      1379/clamd
tcp        0      0 127.0.0.1:783               0.0.0.0:*                   LISTEN      1600/spamd.pid
tcp        0      0 0.0.0.0:143                 0.0.0.0:*                   LISTEN      1574/dovecot
tcp        0      0 69.28.64.172:10000          0.0.0.0:*                   LISTEN      1796/perl
tcp        0      0 209.208.92.40:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.92.39:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.92.38:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.92.37:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.25.246:53           0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.92.35:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.25.244:53           0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.92.41:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.92.36:53            0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 209.208.25.243:53           0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 69.28.64.172:53             0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 192.168.2.70:53             0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 127.0.0.1:53                0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      1371/sshd
tcp        0      0 127.0.0.1:11000             0.0.0.0:*                   LISTEN      1782/lookup-domain-
tcp        0      0 0.0.0.0:25                  0.0.0.0:*                   LISTEN      1678/master
tcp        0      0 127.0.0.1:953               0.0.0.0:*                   LISTEN      1322/named
tcp        0      0 0.0.0.0:20000               0.0.0.0:*                   LISTEN      1789/perl
tcp        0      0 0.0.0.0:993                 0.0.0.0:*                   LISTEN      1574/dovecot
tcp        0      0 0.0.0.0:995                 0.0.0.0:*                   LISTEN      1574/dovecot
tcp        0      0 0.0.0.0:5672                0.0.0.0:*                   LISTEN      1707/qpidd
tcp        0      0 0.0.0.0:3306                0.0.0.0:*                   LISTEN      1550/mysqld
tcp        0      0 :::587                      :::*                        LISTEN      1678/master
tcp        0      0 :::110                      :::*                        LISTEN      1574/dovecot
tcp        0      0 :::143                      :::*                        LISTEN      1574/dovecot
tcp        0      0 :::80                       :::*                        LISTEN      1695/httpd
tcp        0      0 :::53                       :::*                        LISTEN      1322/named
tcp        0      0 :::22                       :::*                        LISTEN      1371/sshd
tcp        0      0 :::25                       :::*                        LISTEN      1678/master
tcp        0      0 ::1:953                     :::*                        LISTEN      1322/named
tcp        0      0 :::443                      :::*                        LISTEN      1695/httpd
tcp        0      0 :::993                      :::*                        LISTEN      1574/dovecot
tcp        0      0 :::995                      :::*                        LISTEN      1574/dovecot
tcp        0      0 :::5672                     :::*                        LISTEN      1707/qpidd
udp        0      0 0.0.0.0:10000               0.0.0.0:*                               1796/perl
udp        0      0 0.0.0.0:20000               0.0.0.0:*                               1789/perl
udp        0      0 209.208.92.40:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.92.39:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.92.38:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.92.37:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.25.246:53           0.0.0.0:*                               1322/named
udp        0      0 209.208.92.35:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.25.244:53           0.0.0.0:*                               1322/named
udp        0      0 209.208.92.41:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.92.36:53            0.0.0.0:*                               1322/named
udp        0      0 209.208.25.243:53           0.0.0.0:*                               1322/named
udp        0      0 69.28.64.172:53             0.0.0.0:*                               1322/named
udp        0      0 192.168.2.70:53             0.0.0.0:*                               1322/named
udp        0      0 127.0.0.1:53                0.0.0.0:*                               1322/named
udp        0      0 :::53                       :::*                                    1322/named
 
Old 03-17-2014, 11:20 AM   #3
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,624

Rep: Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651
first have a look at what release of cent you are using
CentOS 6.3 is two versions out of support

Cent is ONLY supporting the current minor versions
CentOS 5.10
and
CentOS 6.5

Code:
cat /etc/redhat-release
 
Old 03-17-2014, 12:05 PM   #4
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
How does that help me to diagnose any Networking problem? I posted in the title that it's CentOS 6.3. I can't do an update if I can't get to the interwebs, now can I?
 
Old 03-17-2014, 12:14 PM   #5
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
maybe iptables is "in the way"?
Nmap scan report for 69.28.64.172
Host is up.
PORT STATE SERVICE VERSION
22/tcp filtered ssh
80/tcp filtered http

Code:
service iptables save
service iptables stop
test.
 
Old 03-17-2014, 12:20 PM   #6
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Habitual View Post
maybe iptables is "in the way"?
Nmap scan report for 69.28.64.172
Host is up.
PORT STATE SERVICE VERSION
22/tcp filtered ssh
80/tcp filtered http

Code:
service iptables save
service iptables stop
test.
that doesn't seem to work...

Code:
[root@epsilon pjmu]# service iptables stop
iptables: Setting chains to policy ACCEPT: filter          [  OK  ]
iptables: Flushing firewall rules:                         [  OK  ]
iptables: Unloading modules:                               [  OK  ]
[root@epsilon pjmu]# ping -c 3 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
From 69.28.64.172 icmp_seq=2 Destination Host Unreachable
From 69.28.64.172 icmp_seq=3 Destination Host Unreachable

--- 8.8.8.8 ping statistics ---
3 packets transmitted, 0 received, +2 errors, 100% packet loss, time 11999ms
pipe 2
 
Old 03-17-2014, 12:25 PM   #7
John VV
LQ Muse
 
Registered: Aug 2005
Location: A2 area Mi.
Posts: 17,624

Rep: Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651Reputation: 2651
on a server you should not need the wireless and VPN dynamic "network manager"

also "network" and "network manager" can NOT !!! both be turned on

disable "network manager" and turn on "network"

also unless you really using ipv6 you might want to disable that

dhcp has a 5 min. time out and if it is looking for a ipv6 address that can not be looked up
as in 8.8.8.8 and 8.8.4.4 is googles ipv4 address
this will first try to use ipv6
time out
then try the ipv4 address
Last edited by John VV; 03-17-2014 at 12:26 PM.
 
Old 03-17-2014, 12:30 PM   #8
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
NetworkManager has been stopped and then removed as per the OP. How do I diagnose if Network is working? I even posted the output of service network restart in the OP.

Telling me to disable one and to turn on the other doesn't give me the direction I need. What commands I need to use to do this would be helpful.

Quote:
Originally Posted by John VV View Post
on a server you should not need the wireless and VPN dynamic "network manager"

also "network" and "network manager" can NOT !!! both be turned on

disable "network manager" and turn on "network"

also unless you really using ipv6 you might want to disable that

dhcp has a 5 min. time out and if it is looking for a ipv6 address that can not be looked up
as in 8.8.8.8 and 8.8.4.4 is googles ipv4 address
this will first try to use ipv6
time out
then try the ipv4 address
 
Old 03-17-2014, 12:34 PM   #9
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
I did a gig 8.8.8.8

Code:
[root@epsilon pjmu]# dig 8.8.8.8

; <<>> DiG 9.8.2rc1-RedHat-9.8.2-0.23.rc1.el6_5.1 <<>> 8.8.8.8
;; global options: +cmd
;; connection timed out; no servers could be reached
 
Old 03-17-2014, 12:42 PM   #10
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
I ran iptables -L

Code:
[root@epsilon pjmu]# iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
It's my opinion that there is a clear cut way to diagnose Network connectivity issues. Surely, someone knows how to diagnose why I can't ping out, even though I can ssh in on a private IP.
Last edited by NetDoc; 03-17-2014 at 12:44 PM.
 
Old 03-17-2014, 01:19 PM   #11
Habitual
LQ Veteran
 
Registered: Jan 2011
Location: Abingdon, VA
Distribution: Catalina
Posts: 9,374
Blog Entries: 37

Rep: Reputation: Disabled
contents of /etc/resolv.conf please...?
 
Old 03-17-2014, 01:27 PM   #12
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by Habitual View Post
contents of /etc/resolv.conf please...?
Sure why not? It's not a resolv.conf issue if I can't ping a simple IP address. But hey, here it is:

Code:
search scubaboard.com

nameserver 8.8.8.8
nameserver 8.8.4.4
 
Old 03-17-2014, 01:36 PM   #13
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694
Its unlikely to be resolv.conf

How about routing/network config?

Code:
ip a
ip route
 
1 members found this post helpful.
Old 03-17-2014, 01:59 PM   #14
NetDoc
Member
 
Registered: Sep 2007
Posts: 50

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by szboardstretcher View Post
Its unlikely to be resolv.conf

How about routing/network config?

Code:
ip a
ip route
I'm liking this direction...

ip a
Code:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:30:48:35:2a:32 brd ff:ff:ff:ff:ff:ff
    inet 192.168.2.70/24 brd 192.168.2.255 scope global eth0
    inet6 fe80::230:48ff:fe35:2a32/64 scope link
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:30:48:35:2a:33 brd ff:ff:ff:ff:ff:ff
    inet 69.28.64.172/28 brd 69.28.64.175 scope global eth1
    inet 209.208.25.243/29 brd 209.208.25.247 scope global eth1:1
    inet 209.208.92.36/28 brd 209.208.92.47 scope global eth1:10
    inet 209.208.92.41/27 brd 209.208.92.63 scope global eth1:11
    inet 209.208.25.244/29 brd 209.208.25.247 scope global secondary eth1:2
    inet 209.208.92.35/27 brd 209.208.92.63 scope global secondary eth1:3
    inet 209.208.25.246/29 brd 209.208.25.247 scope global secondary eth1:4
    inet 209.208.92.37/27 brd 209.208.92.63 scope global secondary eth1:6
    inet 209.208.92.38/27 brd 209.208.92.63 scope global secondary eth1:7
    inet 209.208.92.39/27 brd 209.208.92.63 scope global secondary eth1:8
    inet 209.208.92.40/27 brd 209.208.92.63 scope global secondary eth1:9
    inet6 fe80::230:48ff:fe35:2a33/64 scope link
       valid_lft forever preferred_lft forever
ip route
Code:
 209.208.25.240/29 dev eth1  proto kernel  scope link  src 209.208.25.243
209.208.92.32/28 dev eth1  proto kernel  scope link  src 209.208.92.36
69.28.64.160/28 dev eth1  proto kernel  scope link  src 69.28.64.172
209.208.92.32/27 dev eth1  proto kernel  scope link  src 209.208.92.41
192.168.2.0/24 dev eth0  proto kernel  scope link  src 192.168.2.70
169.254.0.0/16 dev eth0  scope link  metric 1002
169.254.0.0/16 dev eth1  scope link  metric 1003
default via 69.28.64.162 dev eth1
 
Old 03-17-2014, 02:13 PM   #15
szboardstretcher
Senior Member
 
Registered: Aug 2006
Location: Detroit, MI
Distribution: GNU/Linux systemd
Posts: 4,278

Rep: Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694Reputation: 1694
And can you ping your gateway?

Code:
ping -I eth1 69.28.64.162
Also, what is your traceroute:

Code:
traceroute 8.8.8.8
And what do you get:

Code:
ping -I eth0 8.8.8.8
ping -I eth1 8.8.8.8
 
1 members found this post helpful.
  


Reply
Page 1 of 2 1 2



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] Network Slowdown: difficult to diagnose Sum1 Linux - Networking 29 10-30-2011 03:14 PM
Help me diagnose why I can't use wireless (the notorious BCM4318 network card) veeruk101 Linux - Newbie 1 06-15-2011 03:00 AM
Problem with network and RAID on Centos 5.2 and Centos 5.4 thesirwolf Linux - Hardware 1 04-02-2010 11:47 AM
Problem in network in CentOS after restarting network sidkapoor2000 Linux - Networking 3 07-01-2009 12:27 AM
LXer: Diagnose and manage your network for free with Linux and open source tools LXer Syndicated Linux News 0 07-30-2007 01:16 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 11:31 AM.

Contact Us - Advertising Info - Rules - Privacy - LQ Merchandise - Donations - Contributing Member - LQ Sitemap -
Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration