LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 02-06-2015, 06:47 PM   #1
me4iex
LQ Newbie
 
Registered: Feb 2015
Posts: 3

Rep: Reputation: Disabled
Diagnose an intermittent network problem?


I experience an intermittent DNS problem where lookups become extremely slow or fails completely. I have configured the machine to use OpenDNS, but the problem is also present with Google DNS or any other DNS provider. When the problem arises I'm still able to ping hosts on their IP address including that of the OpenDNS server, without any latency or packet loss. File downloads, established SSH connections etc. aren't influenced either. I have tried to run tcpdump -i eth0 and ping google.com while the problem was present, but everything looked normal beside a long delay before recieving the DNS reply.

What strategy and Linux tools can I use to pinpoint the exact cause of this problem?
 
Old 02-07-2015, 04:51 AM   #2
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
Interesting. Let's start with some questions.
Quote:
Originally Posted by me4iex View Post
What strategy and Linux tools can I use to pinpoint the exact cause of this problem?
- When did this start? Can you link that to any change in network, equipment or configuration?
- How does this exhibit itself? Is it gradual, like during the day or during specific hours, is it after any n requests?
- Have you checked any machines on your network to see if they submit excessive amounts of DNS queries?
- Does this behaviour confine itself to certain domain names or TLDs or not?
- Do you use local DNS caching? (If not: see pdnsd as it uses a disk cache which is persistent and quite efficient.)
 
1 members found this post helpful.
Old 02-07-2015, 11:30 AM   #3
me4iex
LQ Newbie
 
Registered: Feb 2015
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
- When did this start? Can you link that to any change in network, equipment or configuration?
I believe it started a few months ago, but the nature of the problem makes it hard to say exactly. However, no changes to network equipment or configuration has been made in a year or more.
Quote:
- How does this exhibit itself? Is it gradual, like during the day or during specific hours, is it after any n requests?
I have not been able to find any pattern. Everything can be working fine for days, then suddenly the problem arises for as little as a few minutes to many hours. It can be any time of day at the first DNS request or after many thousands. While the problem is present it feels like I'm connected to the Internet by a flaky connection. A web page may fail to load after a long delay, but load instantly when trying again after a few seconds. However, at the same time I can have a file downloading at 10MB/s or an established SSH connection that is still responsive and working as normal.
Quote:
- Have you checked any machines on your network to see if they submit excessive amounts of DNS queries?
I can rule out that machines on the network are submitting excessive amounts of DNS queries.
Quote:
- Does this behaviour confine itself to certain domain names or TLDs or not?
As far as I can tell, the behaviour is not confined to specific domain names or TLDs.
Quote:
- Do you use local DNS caching? (If not: see pdnsd as it uses a disk cache which is persistent and quite efficient.)
I do not use any form of local DNS caching.
 
Old 02-08-2015, 05:14 AM   #4
unSpawn
Moderator
 
Registered: May 2001
Posts: 29,415
Blog Entries: 55

Rep: Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600Reputation: 3600
OK.
- Are you willing to try using DNS caching (Pdnsd) to see if this will help? Else
- when you experience slowness please run this script and post output:
Code:
#!/bin/bash
LANG=C; LC_ALL=C; export LANG LC_ALL; INPUT="$1"; test -z "${INPUT}" && exit 127
DIG_ARGS="+nocomments +noquestion +nostats +nocmd +noauth +noadditional +noedns +noanswer +stats +showsearch +time=10 +tries=1 +retry=1"
\dig $DIG_ARGS -t NS any "${INPUT}" 2>&1; \dig $DIG_ARGS -t NS any "${INPUT}" @8.8.8.8 2>&1; \dig $DIG_ARGS -t NS any "${INPUT}" @208.67.222.222 2>&1
exit 0
You only need to supply it with a domain name that seems to "stall". It's a bit of a stab in the dark but we have to start troubleshooting somewhere.
 
Old 03-20-2015, 10:47 AM   #5
me4iex
LQ Newbie
 
Registered: Feb 2015
Posts: 3

Original Poster
Rep: Reputation: Disabled
I had installed the script provided by unSpawn and was eagerly awaiting the next slowness. Many days went by and nothing happened, at least while I was sitting at the computer. I decided to write a small script, that could shed some light on how often the problem was present. The script did the following:

Code:
1. Select a random domain name from a list with 100 different.
2. Lookup the domain using a specified DNS server. 
3. Abort if the lookup is not completed after 7 seconds, and ping IP address of the DNS server 5 times. Log time and date if all ping packets are replied.
4. Sleep 5 seconds and start over.
Multiple instances of the script was run at the same time, each with a different DNS server:

ISP DNS server primary and secondary
OpenDNS primary and secondary
Google Public DNS primary and secondary
A DNS server installed by myself on a dedicated server

I could see that occasionally there would be a random period of time i.e. 12 minutes were the DNS queries sent to OpenDNS, GoogleDNS and my own DNS server were delayed. Queries sent to the ISP DNS servers were not delayed in these periods. I then configured a second instance of the DNS software on my dedicted server to listen on port 80. I discovered that in the periods with delay, the DNS queries sent to port 80 on my server was not affected.

I have contacted my ISP and explained my findings, but they refused to look further into it.

Is there anything I can do to investigate this further?
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
[SOLVED] How do I diagnose a Network problem? CentOS 6.3 NetDoc Linux - Networking 22 04-24-2015 07:11 AM
Network intermittent disconnect problem Westeros Linux - Networking 3 07-10-2014 08:55 AM
Intermittent connection problem - network/apache jakepa2001 Linux - General 4 06-05-2005 10:43 AM
Intermittent kernel panic - steps 2 diagnose? jago25_98 Linux - Software 2 01-05-2005 01:52 PM
Intermittent network connectivity problem... does not make sense!!! please help!! SiliconBadger Linux - Networking 0 05-24-2002 10:17 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 01:54 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration