I'm trying to debug a program using lsof. I've been seeing a file-descriptor leak in /proc/$PID/fd from sockets that eventually keeps me from being able to create new network connections when I hit the resource limit.

lsof shows me that the socket handles that are being leaked are identified with "TYPE=sock 0,0 ...can't identify protocol" I've performed an strace and there are never any calls made (network or otherwise) that return the file-descriptors that match the bad ones in lsof.

My question is, where are these "sock" type sockets being created? Does anyone else know where they come from and how to get rid of them?

Thanks!

Hello.

Are you running out of socket file descriptors? Check function names getrlimit and setrlimit, their prototypes can be found in file /usr/include/sys/resource.h. This should help.

Karlo

Yes, as I stated, I am running out of fd resources. I can see that using lsof and from /proc/$PID/fd. I believe my program sets itself a limit of 1024 file-descriptors, which it should never use more than 50 of, even when very busy. In fact, I can account for all socket opens and closes in the app and do not otherwise leak socket handles.

These sock-type sockets appear as if from no where...I have not been able to track their origin using strace, meaning no system call I can see returns values equal to the socket handle. If anyone has seen these types show up in lsof and knows why they may be created I would appreciate your knowledge.

FYI, One user of handles I could not account for is openlog and syslog since they do not return their handle IDs. I tried commenting them out of the code and it had no affect.

I faced with the same problem. The fd limit per process is 1024 by default. And the lsof output contained the same thing.
The solution to not run out of fds there was to change that system resource max number of open files (fds) per current process limit to, e. g. 65535. Try it.

Like (program is written right here, may contain syntactic errors):

---[cut here]---
#include <stdio.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/resource.h>

int main(int argc, char **argv) {
struct rlimit limit;

if (getrlimit(RLIMIT_NOFILE, &limit) == -1)
printf("getrlimit(): %s (errno=%d)\n", strerror(errno), errno);
else {
printf("Old limits for RLIMIT_NOFILE: rlim_cur=%d rlim_max=%d\n",
limit.rlim_cur, limit.rlim_max);

limit.rlim_cur = limit.rlim_max = 65535;

if (setrlimit(RLIMIT_NOFILE, &limit) == -1)
printf("setrlimit(): %s (errno=%d)\n", strerror(errno), errno);
else {
if (getrlimit(RLIMIT_NOFILE, &limit) == 0)
printf("New limits for RLIMIT_NOFILE: rlim_cur=%d rlim_max=%d\n",
limit.rlim_cur, limit.rlim_max);
else
printf("getrlimit(): %s (errno=%d)\n", strerror(errno), errno);
}
}

/* Process with your socket fd stuff here...
*/

return 0;
}
---[cut here]---

Wbr,
Karl Tiller

Oh OK, Thanks for that work-around idea!

The one reservation I have is that this is not a safe thing to do for an app expected to run on a production system. Ultimately, I would rather fix the leak than just give it a bigger bucket to spill into. I'll give your idea a try and then test to see what happens when I reach RLIM_INFINITY.