If someone could help me with this, it would be great! I have an issue where something on the server in question where it tries to make a connect to an external IP, and either it gets denied and tries again or just keeps trying and this locks up the server and makes it unresponsive for about 5 min intermittantly. This never happens the same time every day and all i find in the logs is what is below. A kernel upgrade was just done but this didnt help.

I' like to know what on the server is trying to make repeated connections so i can stop it or any other helpful suggestions would be great!

Jul 17 03:53:05 server01 kernel: ip4-fwh-net IN= OUT=eth0
SRC=172.xx.xx.xx DST=216.xxx.xxx.xxx LEN=52 TOS=0x00 PREC=0x00 TTL=64
ID=0 DF PROTO=TCP SPT=22 DPT=52930 WINDOW=5792 RES=0x00 ACK URGP=0
Jul 17 03:53:17 server01 kernel: ip4-fwh-net IN= OUT=eth0
SRC=172.xx.xx.xx DST=216.xxx.xxx.xxx LEN=52 TOS=0x00 PREC=0x00 TTL=64
ID=0 DF PROTO=TCP SPT=22 DPT=52930 WINDOW=5792 RES=0x00 ACK URGP=0
Jul 17 03:54:10 server01 kernel: ip4-fwh-net IN= OUT=eth0
SRC=172.xx.xx.xx DST=216.xxx.xxx.xxx LEN=52 TOS=0x00 PREC=0x00 TTL=64
ID=0 DF PROTO=TCP SPT=22 DPT=52930 WINDOW=5792 RES=0x00 ACK URGP=0
xx.xx DST=216.xxx.xxx.xxx LEN=52 TOS=0x00 PREC=0x00 TTL=64
ID=0 DF PROTO=TCP SPT=22 DPT=52930 WINDOW=5792 RES=0x00 ACK URGP=0
Jul 17 03:54:55 server01 kernel: ip4-fwh-net IN= OUT=eth0
SRC=172.xx.xx.xx DST=216.xxx.xxx.xxx LEN=52 TOS=0x00 PREC=0x00 TTL=64
ID=0 DF PROTO=TCP SPT=22 DPT=52930 WINDOW=5792 RES=0x00 ACK URGP=0
xx.xx DST=216.xxx.xxx.xxx LEN=52 TOS=0x00 PREC=0x00 TTL=64
ID=0 DF PROTO=TCP SPT=22 DPT=52930 WINDOW=5792 RES=0x00 ACK URGP=0

Difficult to say from the information that you have given, but let me clarify a few points:

• is the 172.xx.xx.xx address this server?
• the 216.xxx.xxx.xxx address, have you tried to see who owns that?

(One of them isn't your distro/their repositories and it is looking for updates, is it?)

You could try looking for a cron/anacron job or maybe something like dhcp/avahi, and you may want to try looking at your list of running processes to see if there is something odd there, but I think that I would hope that knowing who the two ends of the process are might give some clue as to the easiest way forwards.

I take it nothing odd is turning up in any of the log files?

Right, the 172 is the servers ip and the other IP it the server it keeps trying to connect to. It never happens at the same time, so no [attern there. The server is a backup server but the times this happens, backups dont run.

Im guessing is the 172. src server is trying to connect to the other ip and keeps trying to connect to the 216. ip and cant make a connection and keeps trying for approx. 15 min. If so, i would like to know how to track down what is trying to connect to the 216. ip.

If this isnt it, how can i make it stop?

Thanks

1.) Don't bother masking the IPs. If some script kiddie wants to root boxes, he's going to be running a bunch of automated scan/attack tools looking for thousands of machines, he's not going to be trawling forums looking for single machines here and there that probably aren't vulnerable to his automate scripts any way. On the other hand, masking IPs (like the external one trying to talk to your box) can prevent people from accurately identifying what's going on, and therefor make it harder (or impossible) to help you. Show the real IPs, please.

2.) It looks like that is actually an inbound connection, not an outbound. The strange thing is, from your log it doesn't look like the first packet in the ssh connection is blocked (the incoming one from 216.x.x.x), but iptables actually blocks the return packet from your ssh daemon.

3.) This really shouldn't cause an interruption to your server... perhaps there's a massive amount of incoming ssh connections and they just aren't being logged by iptables because the logging rules are setup strangely?

But doesnt the SRC=172.xx.xx.xx DST=216.xxx.xxx.xxx line mean that the src of the ssh connection is the server thats locking up (172.ip) and that its trying to reach the destination IP 216.?

So is it because the ssh is hanging cause the 216 ip is busy and cant accept the connection or is denying it altogether. But id like to find out this or the root of the problem, why the 172 server it trying to connect to 216 server anyways and stop that.

So help with finding why or why the sytem locks up for a few min while making this connection would be great!

That means it's a packet coming from the socket that sshd is listening on, going to an ephemeral port on another system (i.e. a port that a client would bind to when making an outbound connection). The iptables log isn't showing strictly connections, it's showing packets that are part of an (attempted) connection.

As for the rest of your questions, I already told you, it's inbound, not outbound. Post the real IPs. What's the output of iptables -L?

If you can get in there while it's doing this even for a second, you may be able to do:

netstat -anp

I don't know though, that's just my two cents.

It could be that 216.x.x.x has been connected to your port 22 (ssh) and after some time your ssh server is checking if it's still connected.
There is a firewall blocking icmp error so your sshd server keeps on retrying the number (ClientAliveCountMax etc.. in sshd_config)

You should identify this machine and you could also run a sniffer to see what's going on.

From this very few details we can see that 216 has been logged in (ACK from your machine)


That's strange that a failed connection locks the station... the last OS I've seen doing this was windows 3.11..

The other side hasn't actually been connected, notice the IP Id field is 0 in the response. It's the first packet going from the server back to the client. i.e. it's the syn/ack in the three-way handshake (weird I don't see the syn flag set, but maybe iptables doesn't show that on acks?).

I think it would be displayed as SYN/ACK.
And IPID 0 can be because of a older kernel (2.4.x) Especially as this is a small "DF" packet, there is no real need for IPid.
IpId have never been very clearly defined for their initial values.

Did you check your /var/log/auth.log? If its incoming ssh could be someone doing scans against it. And how are you measuring the unresponsiveness? Could be the firewall temporarily is blocking ssh to stop the repeated ssh connections.

Initial value of IP Id can be whatever, but it needs to increment with each following packet. There's nothing remarkable about DF (don't fragment), it simply means the data shouldn't be fragmented by intermediate routers (to maintain the packet size and efficiency). Pretty much everything on the Internet should be able to handle the default, which is 1500 minus ethernet headers. That's why DF is so common. It's also used for PMTU discovery.

that's the theory yes. But in practice this can be used to find out opened ports hidden behind a firewall. As the id changes it means that the kernel acked the packet.sending then to an open unfiltered port you will see id+2. This is known since nearly a decade as idle scan. Simply search for ip id zero df and you will see a kernel patch i am talking about.
Also less than a year ago all linux and bsd were found to use a not so perfect rng for ip id.

You should specify which BSD, because OpenBSD has had strong randomization in source ports and IDs for quite some time.

Maybe a SamSpade.org lookup on the other IP, or a better wiretrace utility on the packet flow, for more information?