LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 04-21-2020, 03:36 AM   #1
realbro
LQ Newbie
 
Registered: Apr 2020
Posts: 3

Rep: Reputation: Disabled
file descriotor performance question


During real-time file monitoring in progress, many file discriptors are created. Would there be any impacts on performance if a large numbers of file discriptors generated?
(In case of approaching the maximum count.
Massive amount of file discriptors are opened from fanotify event.)
 
Old 04-22-2020, 12:21 AM   #2
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275
You need to check the implementation (if you want to be sure), I think the answer is no.
 
1 members found this post helpful.
Old 04-22-2020, 01:06 AM   #3
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Chewing up kernel resources unnecessarily is never a good thing - why do you have so many fd's ?.
 
1 members found this post helpful.
Old 04-22-2020, 01:29 AM   #4
realbro
LQ Newbie
 
Registered: Apr 2020
Posts: 3

Original Poster
Rep: Reputation: Disabled
Quote:
Originally Posted by syg00 View Post
Chewing up kernel resources unnecessarily is never a good thing - why do you have so many fd's ?.

Analyze the file based on the open fd. (yara)
 
Old 04-22-2020, 01:34 AM   #5
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
That is a symptom, not the reason - perhaps you should look at your design.
 
1 members found this post helpful.
Old 04-22-2020, 01:51 AM   #6
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275
yes, actually I was thinking about the following question[s]: What are you talking about, what kind or performance do you mean? How much is that "many file descriptors" at all?
 
Old 04-22-2020, 02:01 AM   #7
EdGr
Member
 
Registered: Dec 2010
Location: California, USA
Distribution: I run my own OS
Posts: 983

Rep: Reputation: 466Reputation: 466Reputation: 466Reputation: 466Reputation: 466
This sounds like the old joke in which the patient says "Doctor, it hurts when I do this." The doctor replies "Don't do that!"

Seriously, try not to write or run piggy software.
Ed
 
1 members found this post helpful.
Old 04-22-2020, 02:39 AM   #8
realbro
LQ Newbie
 
Registered: Apr 2020
Posts: 3

Original Poster
Rep: Reputation: Disabled
I am making anti-virus real-time monitoring system (using fanotify).
Fd is opened when a fanotify event (like open, access, close) occurs.
If the file scan takes a long time, the fd is stored in the queue with open status.
In fact, I'm a Windows platform developer and I didn't understand Linux enough, so I asked.

Thanks for all comments.

Currently, when an event occurs, the file path will be collected, fd is immediately closed, stored in the queue, and then opened again for use.
 
Old 04-22-2020, 02:49 AM   #9
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 21,692

Rep: Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275Reputation: 7275
I guess this is how it is planned to work and it is ok. But if you find any performance related problem we can discuss it (for example here)
 
1 members found this post helpful.
Old 04-22-2020, 08:12 PM   #10
syg00
LQ Veteran
 
Registered: Aug 2003
Location: Australia
Distribution: Lots ...
Posts: 21,103

Rep: Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117Reputation: 4117
Maybe it would be better to do the fanotify_init on the root, and only have the one fd (permanently) open ?. You still get the notifications without the continual setup/teardown cost.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
iSCSI write performance very poor while read performance is excellent dinominant Linux - Server 1 10-10-2012 10:51 AM
V-server Performance, IO Scheduler and Allocator resource usage and performance. debwalker Linux - Server 0 07-14-2012 04:21 PM
Software Raid 6 - poor read performance / fast write performance Kvothe Linux - Server 0 02-28-2011 03:11 PM
Lighttpd performance problem + RAID performance problem in a high load site phaz0r Linux - Server 0 11-16-2008 08:52 AM
LXer: Performance Technologies Announces Availability of AMC121 High-Performance Comp LXer Syndicated Linux News 0 09-18-2007 10:30 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 09:56 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration