LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices

Reply
 
LinkBack Search this Thread
Old 03-08-2007, 11:07 AM   #1
ToBe
Member
 
Registered: Aug 2003
Distribution: RHEL 4&5 Ubuntu LTS 6.06 & 8.04, FreeBSD
Posts: 74

Rep: Reputation: 20
Server-friendly way to delete MANY tiny files?


I've got a production server that is used 24/7.

My problem is I have an backup directory I no longer need. It is about 9 GB with thousands and thousands of tiny little files.

When I do a rm -Rf backup/ it makes the server slow down to a complete crawl, then our phones start ringing with complaining users.

I want an alternative way to remove all these files without bringing the web/email service to its knees.

Any suggestions?
 
Old 03-08-2007, 11:41 AM   #2
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,500

Rep: Reputation: 285Reputation: 285Reputation: 285
Well, you can script the file deletion with a loop and small delay to prevent hitting the server hard. For example, if you have 50,000 files, you can use a .07 second sleep between files to spread the deletion over an hour (plus the actual deallocation time).

Update: I also wanted to mention that later releases (FC6 for example) make this much easier with the ionice functionality. You can set your process to low cpu and idle I/O priority, so that impact is minimal. For example:

/usr/bin/nice -n19 /usr/bin/ionice -c3 /bin/rm -rf /backup

Last edited by macemoneta; 03-08-2007 at 11:46 AM.
 
Old 03-08-2007, 12:39 PM   #3
ToBe
Member
 
Registered: Aug 2003
Distribution: RHEL 4&5 Ubuntu LTS 6.06 & 8.04, FreeBSD
Posts: 74

Original Poster
Rep: Reputation: 20
Quote:
Originally Posted by macemoneta
Well, you can script the file deletion with a loop and small delay to prevent hitting the server hard. For example, if you have 50,000 files, you can use a .07 second sleep between files to spread the deletion over an hour (plus the actual deallocation time).

Update: I also wanted to mention that later releases (FC6 for example) make this much easier with the ionice functionality. You can set your process to low cpu and idle I/O priority, so that impact is minimal. For example:

/usr/bin/nice -n19 /usr/bin/ionice -c3 /bin/rm -rf /backup

Oh yea I think this is exactly what I was looking for... I'll try that. Thank you!
 
Old 03-08-2007, 12:59 PM   #4
macemoneta
Senior Member
 
Registered: Jan 2005
Location: Manalapan, NJ
Distribution: Fedora x86 and x86_64, Debian PPC and ARM, Android
Posts: 4,500

Rep: Reputation: 285Reputation: 285Reputation: 285
The ionice command is very handy. I've used it in the reverse situation as well. For example, on a machine that's being pounded by a long running I/O intensive operation, I can still burn a DVD without an underrun:

/usr/bin/nice -n -15 /usr/bin/ionice -c1 /usr/bin/growisofs ...

The growisofs is not a heavy user of I/O, but it is time sensitive. This insures that its requests get ordered to the top of the queue.

It was my sole driver for upgrading to FC6.
 
  


Reply

Tags
files


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Tiny Web Server? jantman Linux - Server 4 01-24-2007 12:41 PM
delete files in server that is older than 30 days using cronjob latheesan *BSD 2 06-14-2005 12:37 PM
Files delete from SAMBA server klmn1 Linux - General 2 04-17-2004 02:34 AM
How to delete the destination files while the source files deleted in cp -u ? myunicom Linux - General 4 09-26-2003 01:13 PM
tiny fuge install using sh and make files? moetjojo Linux - Newbie 1 05-17-2002 02:21 PM


All times are GMT -5. The time now is 02:39 AM.

Main Menu
 
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: @linuxquestions
Open Source Consulting | Domain Registration