LinuxQuestions.org
Visit Jeremy's Blog.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 09-11-2008, 08:19 AM   #1
samengr
Member
 
Registered: Jan 2008
Posts: 59

Rep: Reputation: 15
help - script to delete the data


Hi All,



I have a huge folder of around 50G in size and have different subdirectories. I want to create a script that can find the files older than different dates for some of the subdirectories and delete the files.



I assume to exclude the subdirectory-1 and subdirectory-2 from the following command and delete all files older then 15 days for all other subdirectories.



find /../rootdirectory/ -type f -daystart -mtime +15 -not -path '/rootdirectory/subdirectory-1/' -not -path '/rootdirectory/subdirectory-1/' -not -path -exec rm {} \;



find /../rootdirectory/subdirectory-1/ -type f -daystart -mtime +30 -exec rm {} \;

find /../rootdirectory/subdirectory-2/ -type f -daystart -mtime +60 -exec rm {} \;




Well this is not working for me :-(



If someone can suggest some otherway to do this I would really appreciate.



Many thanks.
 
Old 09-11-2008, 08:49 AM   #2
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
Maybe you could explain what's 'not working' in detail.
ie what errors are occurring.
In any case this path ' /../' etc doesn't make sense . You can't go 'up' from the root dir ('/').
 
Old 09-12-2008, 11:11 AM   #3
samengr
Member
 
Registered: Jan 2008
Posts: 59

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by chrism01 View Post
Maybe you could explain what's 'not working' in detail.
ie what errors are occurring.
In any case this path ' /../' etc doesn't make sense . You can't go 'up' from the root dir ('/').
----------

well Simple is that I need a command/script that can delete files older than 15 days but leaving few sub-directories. I mean if i want to delete all files in all sub-direcotires of a current directory excluding two three subdirectories.

/../ that means the path to the main directory.

like /opt/shared-directory is the main directory that contains 50 subdirectories of individual students. Now i want to delete all files older than 15 days in all student directories except first three position holders.(i.e. want to exclude them)

To delete all the files from a specific directory ( and subdirectories) i can use this command

find /opt/shared-directory/ -type f -daystart -mtime +15 -exec rm {} \;

but how can i exclude two-three subdirectories from the /opt/shared-directory in the above command?

I hope now you can undertand what exactly i need.
Thanks.
 
Old 09-12-2008, 03:34 PM   #5
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
Code:
find ~/downloads/  -mtime +15 -not -wholename '*/helix*'
This will eliminate the directory (and it's contents)~/download/helix/ from the results, including the directory. Using -prune will still result in the directory being printed if you don't have a -type f.

Here I am grouping the two path tests and using -type f.
Code:
find ~/downloads/  -type f -mtime +15 \( -not -path '*/downloads/helix/*' -a -not -path "*/geda/*" \) -execdir ls -l '{}' \;
Would you mind explaining "/../" for me. Where would it have an advantage over simply "/"?
It starts at the root directory (the first '/') goes up one directory level (which seems odd since you are starting at root) and then selects the root directory again.

Last edited by jschiwal; 09-12-2008 at 03:36 PM.
 
Old 09-12-2008, 05:57 PM   #6
samengr
Member
 
Registered: Jan 2008
Posts: 59

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jschiwal View Post
Code:
find ~/downloads/  -mtime +15 -not -wholename '*/helix*'
This will eliminate the directory (and it's contents)~/download/helix/ from the results, including the directory. Using -prune will still result in the directory being printed if you don't have a -type f.

Here I am grouping the two path tests and using -type f.
Code:
find ~/downloads/  -type f -mtime +15 \( -not -path '*/downloads/helix/*' -a -not -path "*/geda/*" \) -execdir ls -l '{}' \;
Would you mind explaining "/../" for me. Where would it have an advantage over simply "/"?
It starts at the root directory (the first '/') goes up one directory level (which seems odd since you are starting at root) and then selects the root directory again.
------------

Thanks to all of you who are helping here on this forum.

Can some body look into this script "studentsCleaner.sh" whats wrong with this? It is working ok for all other students except shani and neetu.

[root@Lib ~]# cat /etc/cron.daily/studentsCleaner.sh
# Clean student files that is older than 30 days
# This ignores the following students as they have special lengths of time to live.
# Simon - keep for a year!
# Elaine - never delete
# Ali - keep for 60 days
# Armand - 90 days
# Max - 15 days
# Shani - 75 days
# Neetu - 100 days

find /opt/tomcat/students/ -type f -daystart -mtime +30 -not -path '/opt/tomcat/students/simon*' -not -path '/opt/tomcat/students/elaine*' -not -path '/opt/tomcat/students/ali*' -not -path '/opt/tomcat/students/armand*' -not -path '/opt/tomcat/students/max*' -not -
path '/opt/tomcat/students/neetu*' -not -
path '/opt/tomcat/students/shani*' -exec rm {} \;
# All above is one command

find /opt/tomcat/students/simon/ -type f -daystart -mtime +365 -exec rm {} \;

find /opt/tomcat/output/ali/ -type f -daystart -mtime +60 -exec rm {} \;

find /opt/tomcat/output/armand/ -type f -daystart -mtime +90 -exec rm {} \;

find /opt/tomcat/output/max/ -type f -daystart -mtime +15 -exec rm {} \;

find /opt/tomcat/output/shani/ -type f -daystart -mtime +75 -exec rm {} \;

find /opt/tomcat/output/neetu/ -type f -daystart -mtime +100 -exec rm {} \;


Infact it was working fine for all students, I inserted some new enteries for shani and neetu. Script is still running fine for other students but it is deleting files of shani and neetu older than 30 days.

I have checked the paths and the names are correct but unable to understand why the 30 days older files of shani and neetu are being deleted when i have mentioned to exclude the shani and neetu files in the first part and then clearly stated the length of these two students below.


I would really appreciate if someone can fix this issue.

Many thanks.
 
Old 09-13-2008, 01:37 AM   #7
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 62
In your first find you use '/opt/tomcat/students/neetu*', but in your later find, you use "/opt/tomcat/output/neetu/". Same with shani.

Do yourself a favor and SIMPLIFY your script by using variables:

Code:
u_neetu="/opt/tomcat/output/neetu/"
u_shani="/opt/tomcat/students/shani"

...

find ...-not -path ${u_neetu}'*' ... -not -path ${u_shani}'*' ...
Bonus points if you can use a variable for each user that indicates their file age. This way, you can use a loop with just a single parameterized find. Triple the points by creating a way to simply create a variable and have the script automatically exclude the special users from the first find, and create the second finds.

Also, DON'T exec for each and every file! Remove the -exec and instead use xargs:
Code:
find ... -print0 | xargs -0 rm -f
Guess what... not A and not B and not C is the same as not (A or B or C). This makes your find logic simpler.

jschiwal took the time to respond, yet you didn't answer the question or show that you paid much attention to the response. You won't get much help this way.

Last edited by Mr. C.; 09-13-2008 at 05:00 AM.
 
Old 09-13-2008, 04:56 AM   #8
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677Reputation: 677
In the future, please post code in [ code ] ... [ /code ] blocks. You can also split a long line by adding a "\" at the end of a line
to keep it from being a mile long.

There is a form of using find that you might want to consider:
Code:
find ./ -maxdepth 1 -type f \( -name "*.txt" -exec ls -l '{}' \; \) -o \
                            \( -name "*.mp4" -exec ls    '{}' \; \)
This allows you to apply different tests resulting in different actions. I also like to format code, if possible, so that it resembles a table. Imagine a case where you have an external drive or a public share filled with different types of files by different owners. You could run a script with a command like this to move files into each users respective home directory in places such as Documents/pictures/ and Documents/text/. Since each user in on another line, adding or removing a user is easy. However, using a variable in the find command and having the users listed at the top or included (sourced) from a separate file would be a better idea, if the action where the same.

Last edited by jschiwal; 09-13-2008 at 05:04 AM.
 
Old 09-13-2008, 05:07 AM   #9
samengr
Member
 
Registered: Jan 2008
Posts: 59

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Mr. C. View Post
In your first find you use '/opt/tomcat/students/neetu*', but in your later find, you use "/opt/tomcat/output/neetu/". Same with shani.

Do yourself a favor and SIMPLIFY your script by using variables:

Code:
u_neetu="/opt/tomcat/output/neetu/"
u_shani="/opt/tomcat/students/shani"

...

find ...-not -path ${u_neetu}'*' ... -not -path ${u_shani}'*' ...
Bonus points if you can use a variable for each user that indicates their file age. This way, you can use a loop with just a single parameterized find. Triple the points by creating a way to simply create a variable and have the script automatically exclude the special users from the first find, and create the second finds.

Also, DON'T exec for each and every file! Remove the -exec and instead use xargs:
Code:
find ... -print0 | xargs rm -f
Guess what... not A and not B and not C is the same as not (A or B or C). This makes your find logic simpler.

jschiwal took the time to respond, yet you didn't answer the question or show that you paid much attention to the response. You won't get much help this way.
================

Thanks for reply. You are right i should define the variables to simplify the script and definitely i will do that. but my question was why shani and neetu's 30 days older files are being deleted while i have specified to exclude them in the first find command. why the first find command doesnt exclude the shani and neetu directories? I am unable to understand why its behaviour is different for different students? see it is excluding simon, elaine, ali files.


"In your first find you use '/opt/tomcat/students/neetu*', but in your later find, you use "/opt/tomcat/output/neetu/". Same with shani."

Sorry, consider students {/opt/tomcat/students/} instead of output.

find /opt/tomcat/students/ -type f -daystart -mtime +30 -not -path '/opt/tomcat/students/simon*' -not -path '/opt/tomcat/students/elaine*' -not -path '/opt/tomcat/students/ali*' -not -path '/opt/tomcat/students/armand*' -not -path '/opt/tomcat/students/max*' -not -
path '/opt/tomcat/students/neetu*' -not -
path '/opt/tomcat/students/shani*' -exec rm {} \;
# All above is one command

find /opt/tomcat/students/simon/ -type f -daystart -mtime +365 -exec rm {} \;

find /opt/tomcat/students/ali/ -type f -daystart -mtime +60 -exec rm {} \;

find /opt/tomcat/students/armand/ -type f -daystart -mtime +90 -exec rm {} \;

find /opt/tomcat/students/max/ -type f -daystart -mtime +15 -exec rm {} \;

find /opt/tomcat/students/shani/ -type f -daystart -mtime +75 -exec rm {} \;

find /opt/tomcat/students/neetu/ -type f -daystart -mtime +100 -exec rm {} \;



Waiting for suggestions why the first find is not excluding the neetu and shani folders.

Many thanks.
 
Old 09-13-2008, 05:10 AM   #10
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 62
At this point, it might be best to show actual evidence, by way of showing the exact command and exact output. Show the files that are incorrectly deleted and their timestamps.
 
Old 09-13-2008, 05:41 PM   #11
samengr
Member
 
Registered: Jan 2008
Posts: 59

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Mr. C. View Post
At this point, it might be best to show actual evidence, by way of showing the exact command and exact output. Show the files that are incorrectly deleted and their timestamps.
=========

I really appreciate that you are helping me to fix this issue.

Sorry I didnt get you the command/script i have already pasted and I am pasting it again, I just coppied it from console

find /opt/tomcat/students/ -type f -daystart -mtime +30 -not - path '/opt/tomcat/students/simon*' -not -path '/opt/tomcat/students/elaine*' -not -path '/opt/tomcat/students/ali*' -not -path '/opt/tomcat/students/armand*' -not -path '/opt/tomcat/students/max*' -not -
path '/opt/tomcat/students/neetu*' -not -
path '/opt/tomcat/students/shani*' -exec rm {} \;
# All above is one command

find /opt/tomcat/students/simon/ -type f -daystart -mtime +365 -exec rm {} \;

find /opt/tomcat/students/ali/ -type f -daystart -mtime +60 -exec rm {} \;

find /opt/tomcat/students/armand/ -type f -daystart -mtime +90 -exec rm {} \;

find /opt/tomcat/students/max/ -type f -daystart -mtime +15 -exec rm {} \;

find /opt/tomcat/students/shani/ -type f -daystart -mtime +75 -exec rm {} \;

find /opt/tomcat/students/neetu/ -type f -daystart -mtime +100 -exec rm {} \;

and how can i show the deleted data ...? :-(
I put this script in /etc/cron.daily/ (daily executeable)


Any help?

Last edited by samengr; 09-13-2008 at 05:42 PM.
 
Old 09-13-2008, 06:27 PM   #12
Mr. C.
Senior Member
 
Registered: Jun 2008
Posts: 2,529

Rep: Reputation: 62
Create test data that you can test your script (or slightly modified version). This is where those variables I mentioned above comes in handy.

Use touch with the -t option to create files with old dates; some < 30 days, some > 30 but less than 75 or 100 days, and some over 75 or 100 days (depending upon the user shani or neetu). You can run ls -l of those files, show the output, run your script, and show ls -l after the script completes. The absence of files > 30 days would prove the case. Alternatively, so that you don't have to keep re-creating test days, place echo in front of your rm command so that your find just prints out what it would do.

You've been given hints and help about how to modify your script to aid YOU in debugging, and yet you're not following any advice. I can't tell where lines break because of pasting, or because you have explicitly broken them in your script. I see nothing that indicates the script fails in the way you say it does. Help yourself by following the advice of people who have years and years of experience.
 
Old 09-14-2008, 09:16 PM   #13
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Centos 6.9, Centos 7.3
Posts: 17,417

Rep: Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397Reputation: 2397
I dislike really long cmd lines, they get messy & tricky to debug, as mentioned.
I'd loop over all students with a std find cmd and then just 'continue' if current 'student' is an exception, then deal with those later as you have.
http://www.tldp.org/LDP/abs/html/loopcontrol.html
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
How to delete HDD data? sunram Solaris / OpenSolaris 8 10-13-2007 11:35 AM
How to delete HDD data? sunram Solaris / OpenSolaris 3 10-12-2007 03:57 AM
Windows going to Delete Data sanju_jetking Red Hat 6 03-20-2007 08:34 AM
dump is delete my data Linux For Ever Linux - General 1 09-21-2006 07:13 AM
data recovery: delayed delete rdrs Linux - Security 1 11-30-2005 06:46 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 01:41 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration