Anyone got any good resources on what this file really shows? I've got a server that runs out of file handles but whats reported by file-nr doesn't make sense - ObserveSo according to the info I've found so far... 2713 = allocated file handles, 1166 = available file handles, 235928 = max file handles (as set in /proc/sys/fs/file-max). I know that some documentation (including the kernel docs!) say that the 1166 = handles used but this isn't the case as every file I open a new file this number goes down.

First question - Why doesn't available_files = max_files - allocated_files ??

And why is the output from so very very different to allocated files?

Any info greatly appreciated!

Jamie...

Well from what I can see is that the second number is the amount of allocated handles which are available. Not the total amount of available handles.

If I test it with an application that keeps allocating filehandles. I can see that the second number decreases till it gets to something really low like 10, the first number stays constant while this happens. After it reaches a low point then the first number starts increasing and the second number stays low. Ofcourse the first number won't go any higher then the max.

I don't really know why lsof would show so many more lines. Unless you are running a lot of multithreading applications and it's showing multiple entries for the same file descriptors.

Well that's as much information as I could think of without going through the kernel code and find out what is really going on, which I don't really feel like doing.

Excuse my density on the subject... Whats the distinction between available allocated handles and the total available handles??
So as you open files you use the allocated files handles (middle number gets lower), when this number moves towards some threshold more file handles are made available for processes to use (the first number gets larger) yeah? If this is the case why doesn't the available allocated handles suddenly increase? Or have I misunderstood something (very likely).
Exactly how I feel about the problem. Not being a C coder it would be a challenge to say the least!

Thanks for the info so far its a great help.

Jamie...

Well I'm not an expert on the topic but I was just trying to show what conclusions I came up with. I'll try to explain my conclusions using example to make it more clear. This system I was trying things out on only has the max set to double the default:

#cat /proc/sys/fs/file-nr
2788 1270 8192

Now according to what I see 8192 is the maximum. 2788 is the amount allocated and 1270 is the amount of unused. Meaning that there would be 2788 - 1270 being used. Instead of 8192 -1270 being used. That's what I meant with the difference between 'available allocated handles' and 'total available handles'.

If I start an example application like bash. After starting bash once only the middle number will have decreased by 20. 'lsof | wc -l' increases by 26. The amount changed stays the same each time I start a new bash process. This stays the same till the middle number gets really low. Then it starts allocating new filehandles and the first number starts increasing by 20 instead.

I guess lsof reports more then just the open files for each process. Since it contains 6 lines more each time.

This is just what I see happening. So your conclusion out of that is as good as mine.

Now I get it!

cheers mate

Jamie...

I know I'm doing serious necro posting, but as a Google search for file-nr shows this second on the results, I thought it would be worth posting the link to this useful description I found:

http://www.netadmintools.com/part295.html