SSH quits responding from requests outside local subnet.

kabars_edge · 02-05-2016, 10:30 AM

I've been beating my head against an issue for a while and cannot figure it out, so I'm hoping I can crowdsource an answer.

I have a server running Debian Wheezy (7.8). This machine responds to SSH without issue from other machines internally and externally when brought up. However, after a period of time, somewhere between 15 and 90 minutes, it just quits responding to SSH requests from outside the subnet. All I have to do to make it reachable via SSH is to SSH from another server on the same subnet, then it starts responding to external requests again. This server is on a Comcast business cable connection using one of their Netgear business class routers. I've actually tried everything from changing IP addresses to ports, and the behavior continues with this machine only. I have 3 other Debian servers sitting on this connection and none of them display this behavior.

Any thoughts, ideas, suggestions would be greatly appreciated.

/etc/network/interfaces

Code:

# The primary network interface
allow-hotplug eth0
iface eth0 inet static
	address 10.1.10.11
	netmask 255.255.255.0
	network 10.1.10.0
	broadcast 10.1.10.255
	gateway 10.1.10.1
	# dns-* options are implemented by the resolvconf package, if installed
	dns-nameservers 10.1.10.1

/sbin/ifconfig

Code:

eth0      Link encap:Ethernet  HWaddr 00:e0:81:bc:87:b8  
          inet addr:10.1.10.11  Bcast:10.1.10.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:82401 errors:0 dropped:0 overruns:0 frame:0
          TX packets:20122 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:57270807 (54.6 MiB)  TX bytes:1909057 (1.8 MiB)
          Interrupt:46 Base address:0x6000

unSpawn · 02-05-2016, 12:58 PM

Some things I'm missing here from your end point:
- what does "All I have to do to make it reachable via SSH is to SSH from another server on the same subnet" mean? What do you actually do?
- checking /var/log/messages and /var/log/secure or equivalent for clues,
- listing cron jobs,
- listing any defensive measures like fail2ban,
- ssh daemon in debug mode to see debug output,
- ssh daemon on another port to see if its port-specific,
- remote tcptraceroute to end point TCP/22 to see where the trace stops,
- firewall "-j LOG" rules to verify access.

kabars_edge · 02-05-2016, 03:34 PM

Quote:

Some things I'm missing here from your end point:
- what does "All I have to do to make it reachable via SSH is to SSH from another server on the same subnet" mean? What do you actually do?

I literally just SSH into the misbehaving box from any other machine on it's local subnet, as soon as I create an SSH connection, it starts working from external IP addresses, I don't even have to authenticate.

Quote:

- checking /var/log/messages and /var/log/secure or equivalent for clues,

No entries whatsoever related. I can be running a tail on the auth.log, messages, user.log, and syslog and see nothing until I log in from the local subnet. This makes me think it may be a networking problem with the Comcast router.

Quote:

- listing cron jobs,

No cron jobs.

Quote:

- listing any defensive measures like fail2ban,

None currently. I even uninstalled IPTables to be sure.

Quote:

- ssh daemon in debug mode to see debug output,

This I did, I see nothing until I connect from within the subnet, which then causes it to work perfectly.

Quote:

- ssh daemon on another port to see if its port-specific,

I've tried different ports/IPs as I said in my original post.

Quote:

- remote tcptraceroute to end point TCP/22 to see where the trace stops,

Good idea, I had not thought of TCPtraceroute, will perform and report back.

Quote:

- firewall "-j LOG" rules to verify access.

I removed all firewall/security early in the process to ensure that wasn't what was causing the problem.

Thanks for the thoughts. Next time I run into the problem, I will try the TCPtraceroute and post the results here. Unfortunately, I have to leave my home and makeshift data center to make this happen.