SSH hangs after username
 

I administer the network at a local non-profit, where they run a SuSE-based Proliant server that I setup.

This morning, I started getting alerts that WAN connectivity was down. Surely enough, their ISP had a widespread outage.

However, I also discovered some issues on the LAN. They run a few Windows machines, with client programs that access a MySQL DB on the Linux machine. They were all taking *much* longer than normal to get data from the DB - I wrote them, and even I assumed they had frozen before the data displayed - what is normally instantaneous took 1-2 minutes.

So, I figured I'd SSH into the server and see what's up. Big surprise. I open PuTTY from one of the Windows machines, open a connection to the server, and prompted for my username, enter it, and -- nothing. I wait. And wait. No password prompt. PuTTY times out almost every time.

Now, the internet is back up. I use Nagios for system checking from another network, and have it configured (with DSA key authentication) to SSH to the server in question and execute a script. Most of these SSH-based checks are timing out, even with private key authentication.

I tried to ssh into the server in question as the user that runs the SSH scripts (with public keys) and had, once again, an amazing delay.

Any ideas? Why would it hang after the username?

A common cause of logins hanging is offline shared filesystems (NFS/Samba). Even if you're not using quotas you are usually running a quotacheck at login (otherwise it wouldn't know you aren't running quotas :rolleyes:). This hangs because it tries to access filesystems that it thinks are still there because they weren't umounted.

A workaround is to temporarily mv quotacheck (/sbin/quotacheck) to another name (e.g. /sbin/quotacheck.save). Then a login won't find it. Of course long term you need to do something about the missing mounts - the easiest thing being to reboot to clear out what it thinks is there and then remount after the boot (assuming they're in /etc/fstab or automounter).

That or maybe its a hardware issue. If you are able to I would say go to the actual system and see if its slow.

I have no physical access to this system. Well, to be specific, I have physical access, but it's installed in an electrical closet on a shelf about seven feet off the floor, it's a *pain* to drag a monitor and keyboard in there. If it becomes an issue, I'll setup the system for a serial terminal and bring my laptop (when it gets back from being repaired).

At the moment, the problem seems to be resolved. SSH access from the LAN is back to normal speed - with no changes on my part. SSH via WAN is just as slow as it was before, but this doesn't seem to be a WAN issue - it serves pages as quickly as it did before.

Also, the SQL clients are back to normal speed. It's strange... the load average and memory usage during the problems was the same as normal...

Maybe its a network issue rather then a physical or software problem.

At the moment it seems to be a WAN-only issue... and only SSH. Web serving is fine in terms of speed...