LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Networking
User Name
Password
Linux - Networking This forum is for any issue related to networks or networking.
Routing, network cards, OSI, etc. Anything is fair game.

Notices


Reply
  Search this Thread
Old 09-29-2005, 10:13 PM   #1
longtex
Member
 
Registered: Aug 2004
Location: Texhuahua
Distribution: RedHat 6,7,9,EL; SuSE 9.2 Pro, Knoppix 3.4/.7/.9, Fedora Core 4
Posts: 87

Rep: Reputation: 15
ping fast, but ssh, ftp slow


This just started.. system's been running fine for a year.

Part of the application process is to ftp images to a LAN server. This normally takes nearly no time, since the images are on the order of 10-15K.

On one workstation, it's still more or less instant, but the other one takes 15-20 seconds. Trying to troubleshoot it, here's what I see: pings to the server are running about 0.1ms, pretty steady, on both systems; on the good one, a manual ftp (or ssh, or scp) gets a nearly immediate response (ftp asks for user, other two want password), but on the "bad" system, all three of those take 15-20 seconds before a response is received.

I thought of hardware problems, but would the pings be fast if bad network cable, or card, or switch?

Any ideas?
 
Old 09-30-2005, 01:46 AM   #2
btmiller
Senior Member
 
Registered: May 2004
Location: In the DC 'burbs
Distribution: Arch, Scientific Linux, Debian, Ubuntu
Posts: 4,290

Rep: Reputation: 378Reputation: 378Reputation: 378Reputation: 378
The pings could be fast because they're small amounts of data and perhaps your NIC is having an issue with large packets. I'd suggest sniffing the interface with ethereal and looking for signs of problems (i.e. excessive numbers of TCP retransmits). You can also looks at the ifconfig output and see if there are a lot of packet errors detected. if you perchance have a managed switch, you might be able to get some of this data from it too. Try moving whhich port on your hub/switch the machine is connected to, in case the problem is a bad hub/switch port.
 
Old 09-30-2005, 11:32 AM   #3
longtex
Member
 
Registered: Aug 2004
Location: Texhuahua
Distribution: RedHat 6,7,9,EL; SuSE 9.2 Pro, Knoppix 3.4/.7/.9, Fedora Core 4
Posts: 87

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by btmiller
The pings could be fast because they're small amounts of data and perhaps your NIC is having an issue with large packets. I'd suggest sniffing the interface with ethereal and looking for signs of problems (i.e. excessive numbers of TCP retransmits). You can also looks at the ifconfig output and see if there are a lot of packet errors detected. if you perchance have a managed switch, you might be able to get some of this data from it too. Try moving whhich port on your hub/switch the machine is connected to, in case the problem is a bad hub/switch port.
Yes - I looked at the ifconfig stats on both sides, and there are zero errors shown. The "bad" workstation did have iptables running for some reason (these are behind a pretty stout firewall, and none of the internal boxes have their firewalls running) but I killed that and it doesn't seem to have made the slightest difference.

We did change the cables/switch ports, also with no changes, so it's just about got to be a software issue of some sort... even if it's just bad or corrupted data being fed to something.

I'm thinking it's not specifically related to the ftp, since ssh and scp demonstrate the same behavior, i.e. not even responding for 15-20 seconds... as though they were trying (successfully, finally) to punch through a firewall.

I'm still looking through logs for some kind of clue...

Thanks for the response, I appreciate it!
 
Old 09-30-2005, 12:03 PM   #4
Snowbat
Member
 
Registered: Jun 2005
Location: q3dm7
Distribution: Mandriva 2010.0 x86_64
Posts: 338

Rep: Reputation: 31
Are you connecting by IP address or by hostname? It could be a DNS lookup failure (on either end).
Run tcpdump in a second shell on the affected machine and see what's happening on the wire when you use ftp.
 
Old 09-30-2005, 12:13 PM   #5
longtex
Member
 
Registered: Aug 2004
Location: Texhuahua
Distribution: RedHat 6,7,9,EL; SuSE 9.2 Pro, Knoppix 3.4/.7/.9, Fedora Core 4
Posts: 87

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by Snowbat
Are you connecting by IP address or by hostname? It could be a DNS lookup failure (on either end).
Run tcpdump in a second shell on the affected machine and see what's happening on the wire when you use ftp.
ip's.

The problem's got to be on the "bad" workstation, because I can ftp to the LAN server from my office (200 miles away) with immediate response... and the same is true from every other computer I've tried - I ssh to another computer and ftp from there to that LAN server, no problems, no slowth.

Tell me a bit about tcpdump, please? I'm thinking what I'll be doing is opening two ssh sessions on the "bad" w/s, and then start ftp on one and tcpdump on the other. No doubt tcpdump has a way to capture its output to a file, then we look at the output and see... what? I'm thinking we also want to do the same thing on the "good" w/s and compare the outputs? And, yes, as soon as I'm done responding here I'm going to go RTF tcpdump M, don'cha know...
 
Old 10-02-2005, 06:41 AM   #6
Snowbat
Member
 
Registered: Jun 2005
Location: q3dm7
Distribution: Mandriva 2010.0 x86_64
Posts: 338

Rep: Reputation: 31
I suggest you do this from the keyboard of the 'bad' workstation and look at the tcpdump output live - ssh adds extra network traffic to the mix that will need to be filtered out later. Open two sessions (preferably in a GUI so you can see both concurrently). In one session start tcpdump -i eth0. In the other session start a command line ftp connection to the server. Watch what happens in the tcpdump session: does the workstation send out the ftp connection request immediately... does the server reply immediately?

Alternatively you could do this using ssh and dump tcpdump's output to a file for analysis. Grep the file for lines containing the workstation and server IP addresses. You'll be looking for the same thing - does the workstation send out the ftp connection request immediately... does the server reply immediately?
 
Old 10-02-2005, 03:28 PM   #7
longtex
Member
 
Registered: Aug 2004
Location: Texhuahua
Distribution: RedHat 6,7,9,EL; SuSE 9.2 Pro, Knoppix 3.4/.7/.9, Fedora Core 4
Posts: 87

Original Poster
Rep: Reputation: 15
RESOLUTION:

It turned out to be DNS addresses.

The client's ISP had changed his service a few months (I think maybe 5 or 6) earlier, from a 206. to 12. and the DNS server changed as well. However, as it turned out, the old DNS servers continued to respond until a couple of weeks ago, when this problem manifested.

This is somewhat weird, because both workstations had "old" DNS servers listed, but only one workstation showed the problem.

Actually, it's doubly weird, because the ftp (as well as scp and ssh) connection was - as I said before - by IP address.

Setting the correct DNS address(es) in /etc/resolv.conf cured the problems.

I think perhaps the two workstations had installed slightly different versions of RH9. At the client's central office, where they build the systems, I found a set of CDs labeled RH9 which they got from their hardware supplier, and also a set of CDs with RH9 that were included with the book Red Hat 9 Bible... so I have a suspicion (not yet confirmed) that at least the ftp's were slightly different.

The real question in my mind is why on earth the "bad" ftp (and scp, ssh, and possibly others) would bother to attempt connection with a DNS server when it was given an IP address??? Is it perhaps a setup issue?
 
Old 10-02-2005, 05:24 PM   #8
sal_paradise42
Member
 
Registered: Jul 2003
Location: Utah
Distribution: Gentoo FreeBSD 5.4
Posts: 150

Rep: Reputation: 16
I have run into the same problem before.
It appears that a lot of programs will do reverse lookup on an IP to look for the host-name, if the DNS information is not correct this will make it hang until it times out and doesn't do it. I am not sure where this is configured, most likely in the programs conf file.

Another solution would have been to add a host entry for the machine you were trying to get to in your /etc/hosts file.
 
Old 10-02-2005, 05:49 PM   #9
longtex
Member
 
Registered: Aug 2004
Location: Texhuahua
Distribution: RedHat 6,7,9,EL; SuSE 9.2 Pro, Knoppix 3.4/.7/.9, Fedora Core 4
Posts: 87

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by sal_paradise42

Another solution would have been to add a host entry for the machine you were trying to get to in your /etc/hosts file.
I assume from this that you've tried it and it worked, without a DNS server available - that is , the reverse lookup?

Do these ill-behaved programs look in hosts before they look in resolv.conf? I hadn't thought about the possibilities of reverse lookups in those two...

I'll have to give that a try when I can do it without disrupting business...
 
Old 10-02-2005, 07:24 PM   #10
sal_paradise42
Member
 
Registered: Jul 2003
Location: Utah
Distribution: Gentoo FreeBSD 5.4
Posts: 150

Rep: Reputation: 16
yes /etc/hosts is queried first and then /etc/resolv.conf
I don't think is ill behavior in the application part (maybe unexpected) you just need accurate information for name resolution.
 
Old 10-02-2005, 09:42 PM   #11
longtex
Member
 
Registered: Aug 2004
Location: Texhuahua
Distribution: RedHat 6,7,9,EL; SuSE 9.2 Pro, Knoppix 3.4/.7/.9, Fedora Core 4
Posts: 87

Original Poster
Rep: Reputation: 15
Quote:
Originally posted by sal_paradise42
yes /etc/hosts is queried first and then /etc/resolv.conf
I don't think is ill behavior in the application part (maybe unexpected) you just need accurate information for name resolution.
Tomayto, tomahto. My pronunciation is "ill behavior".

When I give a program an IP address, I don't expect it to use a DNS to find out the associated name, unless finding the name is its job (or part of it, like logging with names).

I noticed that ntpupdate -u 10.10.10.10 did NOT go off to find the names.

If it's a conf matter, that's all well and good, but if the DEFAULT behavior is to do it, well, then... I have to say it's ill-behaved. And if that's not the default, but the distro's default installation includes conf that tells a program like ftp to do name lookup unless told not to... then it's ill-behaved on the part of whoever put it in the installation set.

It could well have been explicitly advertised as such and I missed it. Whatever.

All that said, I still think it's a Bad Thing for ftp to lookup an IP address as a matter of course.

I'll take a look at the "good" workstation after they close and see if it's a different version of ftp or the same ftp but different configurations.

Red Hat 9 - hmmm... what's the .conf for ftp? Would another parameter in a different configuration affect ftp? Lessee...

Last edited by longtex; 10-02-2005 at 09:44 PM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
slow NFS/SCP but fast HTTP/FTP darcon3k Linux - Networking 1 10-13-2005 06:22 AM
Can ping but can not telnet/ftp/ssh my box fbarre Linux - Networking 10 04-20-2004 01:11 AM
internet fast-slow-fast-slow big_and_fat_guy Mandriva 5 10-11-2003 06:44 PM
Can't ping/ssh my box, Shorewall seems to block all traffic except http / ftp tiduck Linux - Networking 10 05-22-2003 10:21 PM
FTP - Fast Downloads - SLOW UPLOADS? fed007 Linux - Newbie 2 08-19-2001 12:32 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Networking

All times are GMT -5. The time now is 03:15 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration