LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-03-2008, 12:01 PM   #16
halo14
Senior Member
 
Registered: Apr 2004
Location: Surprise, AZ
Distribution: Debian | CentOS | Arch
Posts: 1,103

Rep: Reputation: 45

As has already been stated, simply set up a cron job to delete the files the every night at 2am or some time when the server sees less usage. You could really do it any time but to be as nice as possible to the users...
 
Old 01-07-2010, 08:00 PM   #17
doobiest
LQ Newbie
 
Registered: Jul 2003
Posts: 4

Rep: Reputation: 0
I have a similar issue at work and this is what I do. Keep in mind that my issue is a have thousands of subfolders for a particular application's cache, but this could be applied to thousands of files.

If your sever is decently fast just use this to fork off multiple rms to work in parallel

Code:
for list in `ls -d /path/to/cache`; do rm -Rf ${list} & done
I find that much faster than a straight up rm, theres thousands of sub dirs and even more files within. This will do 200 gigs in less than a half hour. Without doing parallel it takes hours.

Notice the 'ls -d' meaning list directories, from what I gather in your case you'll want to just do 'ls' , or rather probably 'ls -1' to get a simple list of files.

Other options:

create and dedicate a partition to the cache, mount it to the cache dir. And then cron a script to unmount, rebuild filesystem, remount. I think killing the partition table and making one from scratch would be quicker than going through the filesystem and marking the inodes deleted. Many tiny files suck performance-wise with something like that.

Also you could use find with the exec:

Code:
find -name /path/to/cache/* find . -exec rm -Rf{} \;
Personally I think the for loop I provided is better than using find solely due to the parallel behavior. Find would still just iterate through a sequential list, I don't see the benefit doing that vs. rm -Rf /path/to/cache.

In the end I think destroying and recreating the partition would be the fastest, however the issue is there that you cannot do it live. Meaning how will the cache get written to in the time since the partition is offline and being rebuilt?

Similarly with my for loop this doesn't account for files that were created at the same time as this loop being run. That's no good, not really safe. What you would likely want to do is write a script that will do an ls or ls -d looking for any file modify dates older than a day, an hour, whatever. Then fork off a pool of rm's to remove only those files. This is the safest.
 
Old 01-07-2010, 08:54 PM   #18
r3sistance
Senior Member
 
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375

Rep: Reputation: 217Reputation: 217Reputation: 217
Quote:
Originally Posted by H_TeXMeX_H View Post
What filesystem are you using ? If it's ext3 then I can see why it takes hours, I bet the same would only take minutes with JFS (fastest delete speed) or XFS.
Why delete at all? that's a pointless venture in most generic cases. I dunno how linux FSs work but I know that Windows generally does not delete files, it simply renames the file in a fasion that shows they are no longer in use and that the data won't truly be removed until it is overwritten... I would have assumed most other FS systems would have worked this way.

anyways I believe the reason why someone might say this would take hours is all the prompts and holding 'y' since their is no force option in the OPs given command what would be the answer here.
 
Old 01-08-2010, 07:15 AM   #19
MTK358
LQ 5k Club
 
Registered: Sep 2009
Posts: 6,443
Blog Entries: 3

Rep: Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721Reputation: 721
Quote:
Originally Posted by r3sistance View Post
Why delete at all? that's a pointless venture in most generic cases. I dunno how linux FSs work but I know that Windows generally does not delete files, it simply renames the file in a fasion that shows they are no longer in use and that the data won't truly be removed until it is overwritten... I would have assumed most other FS systems would have worked this way.

anyways I believe the reason why someone might say this would take hours is all the prompts and holding 'y' since their is no force option in the OPs given command what would be the answer here.
Is I understand that directory is not part of the filesystem, but just a folder that Windows creates.

The rf command actually DELETES FILES FOREVER.

I know that most GUI file managers do use the concept of a trash bin, but I don't know where the directory is.
 
Old 01-08-2010, 07:32 AM   #20
ilikejam
Senior Member
 
Registered: Aug 2003
Location: Glasgow
Distribution: Fedora / Solaris
Posts: 3,109

Rep: Reputation: 96
Quote:
I would have assumed most other FS systems would have worked this way
They do, but during an 'rm' the system has to decrement link counts, clear directory inode lists, update access times and various other stuff. This can take time if you're dealing with hundreds of thousands of files/directories. Not sure how NTFS handles directory deletions, though.

Dave
 
Old 01-08-2010, 07:37 AM   #21
H_TeXMeX_H
LQ Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287Reputation: 1287
I think what you mean is that with modern filesystems what happen is not that the file is overwritten (this never happens in any filesystem that I know of anyway), but rather the inode for the file is unallocated, and the file is thus "forgotten", but not "deleted". In this state a file can be overwritten by another file that is newly created, but if no new file is created, you can use a data carving program to retrieve an intact file without any problems.
 
Old 01-08-2010, 08:37 AM   #22
r3sistance
Senior Member
 
Registered: Mar 2004
Location: UK
Distribution: CentOS 6/7
Posts: 1,375

Rep: Reputation: 217Reputation: 217Reputation: 217
Somewhat closer to my point. Being that until a file is unallocated completely (all links to the inode being destroyed) a file is not deleted and when there are no links, the filesystem does not over-write the file with null-data or 0-fill it's place (what would be true deletion), the data is their until something else needs that place and the data is overwriten by a later process or application.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Can't delete folders from Windows Tenover Linux - Networking 8 12-23-2005 11:55 PM
Can't delete Folders. duffmckagan SUSE / openSUSE 1 09-19-2005 11:09 AM
HELP: can't delete files/folders :( kevingpo General 1 12-07-2004 07:51 PM
Can you delete folders after ./configure ? iLLuSionZ Linux - Newbie 2 11-19-2003 07:39 PM
Am I able to delete folders after install? SuSE_Surfer Linux - General 6 10-17-2003 11:41 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 07:52 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration