Diagnosis with lsof command
 

We have RHEL 5.10 in our production setup. There is a process with PID 30670 whose source is becoming difficult to find. I used the losf command like:

lsof –i | grep 30670

and it gave the output with headings as:

COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME

I guess the last column of ‘NAME’ is important but I am not able to understand what it denotes, request a briefing on the other column headings also.

I hope my question is clear about details of column headings in lsof command.

Please revert with the reply to my query.

Regards

just look at the man page of lsof

Thanks for your answer. Man page of lsof is not at all user friendly in denoting what the column headings denote. Hope, you understand the problem in man pages sometimes.

Regards

would be nice to see what caused your confusion

I got some clue. The generic format of a man search is under the headings name, synopsis, description, options, output etc. I could find help under the output heading. I had killed 30670 as the source was unknown. The process has again started with a PID 15082. When I give,

lsof -i | grep 15082

the output under name heading is

TCP <host-name>:33797->hosted-for-minecraft.net:https (ESTABLISHED)

Could this mean that this process is opening the port 33797 on the host <host-name>? The host-name is the hostname of our server.

Request you to update with your inputs so that this issue can be resolved.

Regards

yes, on the host port 33797 is used to connect to hosted-for-minecraft.net on port https (443). But the port on the remote host is opened and waiting for connection and your host initiated that connection used the port 33797. This port is not opened. (opened means listening)

I needed some clarification in this. The port initiated on our host is 33797 but is that the port is not open? Is this deduced from the output,

TCP <host-name>:33797->hosted-for-minecraft.net:https (ESTABLISHED)

that the port is not open?

Request your update.

Regards

open means some app or daemon is running on that host and will listen on that port. Listening on a port means that app is waiting for incoming packages and will process any package arriving.
From the other side someone, another process/host must initiate the communication, send package to that host.
In your case hosted-for-minecraft.net is listening on port 443 and waiting for requests. Your own host initiates a communication, it sends a package to hosted-for-minecraft.net:https and port 33797 is used to send the package. Your host does not listen on port 33797 and does not wait any incoming packages, therefore this port is not opened. It waits only for responds from the other side, all the other requests will be dropped.

Thanks, there is a lot of clarity now. I wanted to check that as the source of process is unknown, can this port 33797 be blocked using iptables so that it could help in stopping this process?

Request your input.

Regards

yes, you can block that port but I think that app will use another port to connect to the remote server.
here you can find some tips about that: http://www.cyberciti.biz/faq/iptables-block-port/

I had a query, that can an established connection as in this case be used for flooding our network? That would mean that the port 33797 opened on our host is used for flooding from the link hosted-for-minecraft.net:https.

Request your valuable inputs on this.

Regards

I do not think so (but I do not really understand what you are speaking about). Would be nice to explain it better. Thanks.

Thanks for your reply, I would try to explain it better. We had a recent problem around 2 weeks back that there was a suspected broadcast from this server which choked up the lease line connectivity to our network operation center as they share the same lease line. You had mentioned in your earlier reply that the host waits for responds from the other side and all other requests would be dropped, so could these responds be sending so much traffic which effectively is an broadcast. There was another conceptual query that the states of a port are open, closed & listening. So is the transition between the ports from open->listening->closed as a life cycle of transitions? means that the port is first open, then listening & finally closed. Or is it that the port can be straight from listening to closed.

I hope, I have been able to give some clarity on this.

Regards

When your starting point is a PID then next time just supply the PID to 'lsof -Pwlnp'. That's quicker, easier, more efficient.

I would rather say open and listening is the same state. And a port can be either opened or closed (accept incoming connections or not)