help

samengr · 09-11-2008, 07:19 AM

Hi All,

I have a huge folder of around 50G in size and have different subdirectories. I want to create a script that can find the files older than different dates for some of the subdirectories and delete the files.

I assume to exclude the subdirectory-1 and subdirectory-2 from the following command and delete all files older then 15 days for all other subdirectories.

find /../rootdirectory/ -type f -daystart -mtime +15 -not -path '/rootdirectory/subdirectory-1/' -not -path '/rootdirectory/subdirectory-1/' -not -path -exec rm {} \;

find /../rootdirectory/subdirectory-1/ -type f -daystart -mtime +30 -exec rm {} \;

find /../rootdirectory/subdirectory-2/ -type f -daystart -mtime +60 -exec rm {} \;

Well this is not working for me :-(

If someone can suggest some otherway to do this I would really appreciate.

Many thanks.

chrism01 · 09-11-2008, 07:49 AM

Maybe you could explain what's 'not working' in detail.
ie what errors are occurring.
In any case this path ' /../' etc doesn't make sense . You can't go 'up' from the root dir ('/').

samengr · 09-12-2008, 10:11 AM

Quote:

Originally Posted by chrism01

Maybe you could explain what's 'not working' in detail.
ie what errors are occurring.
In any case this path ' /../' etc doesn't make sense . You can't go 'up' from the root dir ('/').

----------

well Simple is that I need a command/script that can delete files older than 15 days but leaving few sub-directories. I mean if i want to delete all files in all sub-direcotires of a current directory excluding two three subdirectories.

/../ that means the path to the main directory.

like /opt/shared-directory is the main directory that contains 50 subdirectories of individual students. Now i want to delete all files older than 15 days in all student directories except first three position holders.(i.e. want to exclude them)

To delete all the files from a specific directory ( and subdirectories) i can use this command

find /opt/shared-directory/ -type f -daystart -mtime +15 -exec rm {} \;

but how can i exclude two-three subdirectories from the /opt/shared-directory in the above command?

I hope now you can undertand what exactly i need.
Thanks.

ehawk · 09-12-2008, 02:02 PM

http://dsl.org/cookbook/cookbook_10.html#SEC143

http://www.linuxquestions.org/questi...n-date-334460/

http://www.orafaq.com/forum/t/71912/0/

http://www.cyberciti.biz/faq/find-co...-ignore-files/

http://www.unix.com/unix-advanced-ex...den-files.html

http://www.unix.com/unix-dummies-que...les-today.html

jschiwal · 09-12-2008, 02:34 PM

Code:

find ~/downloads/  -mtime +15 -not -wholename '*/helix*'

This will eliminate the directory (and it's contents)~/download/helix/ from the results, including the directory. Using -prune will still result in the directory being printed if you don't have a -type f.

Here I am grouping the two path tests and using -type f.

Code:

find ~/downloads/  -type f -mtime +15 \( -not -path '*/downloads/helix/*' -a -not -path "*/geda/*" \) -execdir ls -l '{}' \;

Would you mind explaining "/../" for me. Where would it have an advantage over simply "/"?
It starts at the root directory (the first '/') goes up one directory level (which seems odd since you are starting at root) and then selects the root directory again.

samengr · 09-12-2008, 04:57 PM

Quote:

Originally Posted by jschiwal

Code:

find ~/downloads/  -mtime +15 -not -wholename '*/helix*'

This will eliminate the directory (and it's contents)~/download/helix/ from the results, including the directory. Using -prune will still result in the directory being printed if you don't have a -type f.

Here I am grouping the two path tests and using -type f.

Code:

find ~/downloads/  -type f -mtime +15 \( -not -path '*/downloads/helix/*' -a -not -path "*/geda/*" \) -execdir ls -l '{}' \;

Would you mind explaining "/../" for me. Where would it have an advantage over simply "/"?
It starts at the root directory (the first '/') goes up one directory level (which seems odd since you are starting at root) and then selects the root directory again.

------------

Thanks to all of you who are helping here on this forum.

Can some body look into this script "studentsCleaner.sh" whats wrong with this? It is working ok for all other students except shani and neetu.

[root@Lib ~]# cat /etc/cron.daily/studentsCleaner.sh
# Clean student files that is older than 30 days
# This ignores the following students as they have special lengths of time to live.
# Simon - keep for a year!
# Elaine - never delete
# Ali - keep for 60 days
# Armand - 90 days
# Max - 15 days
# Shani - 75 days
# Neetu - 100 days

find /opt/tomcat/students/ -type f -daystart -mtime +30 -not -path '/opt/tomcat/students/simon*' -not -path '/opt/tomcat/students/elaine*' -not -path '/opt/tomcat/students/ali*' -not -path '/opt/tomcat/students/armand*' -not -path '/opt/tomcat/students/max*' -not -
path '/opt/tomcat/students/neetu*' -not -
path '/opt/tomcat/students/shani*' -exec rm {} \;
# All above is one command

find /opt/tomcat/students/simon/ -type f -daystart -mtime +365 -exec rm {} \;

find /opt/tomcat/output/ali/ -type f -daystart -mtime +60 -exec rm {} \;

find /opt/tomcat/output/armand/ -type f -daystart -mtime +90 -exec rm {} \;

find /opt/tomcat/output/max/ -type f -daystart -mtime +15 -exec rm {} \;

find /opt/tomcat/output/shani/ -type f -daystart -mtime +75 -exec rm {} \;

find /opt/tomcat/output/neetu/ -type f -daystart -mtime +100 -exec rm {} \;

Infact it was working fine for all students, I inserted some new enteries for shani and neetu. Script is still running fine for other students but it is deleting files of shani and neetu older than 30 days.

I have checked the paths and the names are correct but unable to understand why the 30 days older files of shani and neetu are being deleted when i have mentioned to exclude the shani and neetu files in the first part and then clearly stated the length of these two students below.

I would really appreciate if someone can fix this issue.

Many thanks.

Mr. C. · 09-13-2008, 12:37 AM

In your first find you use '/opt/tomcat/students/neetu*', but in your later find, you use "/opt/tomcat/output/neetu/". Same with shani.

Do yourself a favor and SIMPLIFY your script by using variables:

Code:

u_neetu="/opt/tomcat/output/neetu/"
u_shani="/opt/tomcat/students/shani"

...

find ...-not -path ${u_neetu}'*' ... -not -path ${u_shani}'*' ...

Bonus points if you can use a variable for each user that indicates their file age. This way, you can use a loop with just a single parameterized find. Triple the points by creating a way to simply create a variable and have the script automatically exclude the special users from the first find, and create the second finds.

Also, DON'T exec for each and every file! Remove the -exec and instead use xargs:

Code:

find ... -print0 | xargs -0 rm -f

Guess what... not A and not B and not C is the same as not (A or B or C). This makes your find logic simpler.

jschiwal took the time to respond, yet you didn't answer the question or show that you paid much attention to the response. You won't get much help this way.

jschiwal · 09-13-2008, 03:56 AM

In the future, please post code in [ code ] ... [ /code ] blocks. You can also split a long line by adding a "\" at the end of a line
to keep it from being a mile long.

There is a form of using find that you might want to consider:

Code:

find ./ -maxdepth 1 -type f \( -name "*.txt" -exec ls -l '{}' \; \) -o \
                            \( -name "*.mp4" -exec ls    '{}' \; \)

This allows you to apply different tests resulting in different actions. I also like to format code, if possible, so that it resembles a table. Imagine a case where you have an external drive or a public share filled with different types of files by different owners. You could run a script with a command like this to move files into each users respective home directory in places such as Documents/pictures/ and Documents/text/. Since each user in on another line, adding or removing a user is easy. However, using a variable in the find command and having the users listed at the top or included (sourced) from a separate file would be a better idea, if the action where the same.

samengr · 09-13-2008, 04:07 AM

Quote:

Originally Posted by Mr. C.

In your first find you use '/opt/tomcat/students/neetu*', but in your later find, you use "/opt/tomcat/output/neetu/". Same with shani.

Do yourself a favor and SIMPLIFY your script by using variables:

Code:

u_neetu="/opt/tomcat/output/neetu/"
u_shani="/opt/tomcat/students/shani"

...

find ...-not -path ${u_neetu}'*' ... -not -path ${u_shani}'*' ...

Bonus points if you can use a variable for each user that indicates their file age. This way, you can use a loop with just a single parameterized find. Triple the points by creating a way to simply create a variable and have the script automatically exclude the special users from the first find, and create the second finds.

Also, DON'T exec for each and every file! Remove the -exec and instead use xargs:

Code:

find ... -print0 | xargs rm -f

Guess what... not A and not B and not C is the same as not (A or B or C). This makes your find logic simpler.

jschiwal took the time to respond, yet you didn't answer the question or show that you paid much attention to the response. You won't get much help this way.

================

Thanks for reply. You are right i should define the variables to simplify the script and definitely i will do that. but my question was why shani and neetu's 30 days older files are being deleted while i have specified to exclude them in the first find command. why the first find command doesnt exclude the shani and neetu directories? I am unable to understand why its behaviour is different for different students? see it is excluding simon, elaine, ali files.

"In your first find you use '/opt/tomcat/students/neetu*', but in your later find, you use "/opt/tomcat/output/neetu/". Same with shani."

Sorry, consider students {/opt/tomcat/students/} instead of output.

find /opt/tomcat/students/ -type f -daystart -mtime +30 -not -path '/opt/tomcat/students/simon*' -not -path '/opt/tomcat/students/elaine*' -not -path '/opt/tomcat/students/ali*' -not -path '/opt/tomcat/students/armand*' -not -path '/opt/tomcat/students/max*' -not -
path '/opt/tomcat/students/neetu*' -not -
path '/opt/tomcat/students/shani*' -exec rm {} \;
# All above is one command

find /opt/tomcat/students/simon/ -type f -daystart -mtime +365 -exec rm {} \;

find /opt/tomcat/students/ali/ -type f -daystart -mtime +60 -exec rm {} \;

find /opt/tomcat/students/armand/ -type f -daystart -mtime +90 -exec rm {} \;

find /opt/tomcat/students/max/ -type f -daystart -mtime +15 -exec rm {} \;

find /opt/tomcat/students/shani/ -type f -daystart -mtime +75 -exec rm {} \;

find /opt/tomcat/students/neetu/ -type f -daystart -mtime +100 -exec rm {} \;

Waiting for suggestions why the first find is not excluding the neetu and shani folders.

Many thanks.

Mr. C. · 09-13-2008, 04:10 AM

At this point, it might be best to show actual evidence, by way of showing the exact command and exact output. Show the files that are incorrectly deleted and their timestamps.

samengr · 09-13-2008, 04:41 PM

Quote:

Originally Posted by Mr. C.

At this point, it might be best to show actual evidence, by way of showing the exact command and exact output. Show the files that are incorrectly deleted and their timestamps.

=========

I really appreciate that you are helping me to fix this issue.

Sorry I didnt get you the command/script i have already pasted and I am pasting it again, I just coppied it from console

find /opt/tomcat/students/ -type f -daystart -mtime +30 -not - path '/opt/tomcat/students/simon*' -not -path '/opt/tomcat/students/elaine*' -not -path '/opt/tomcat/students/ali*' -not -path '/opt/tomcat/students/armand*' -not -path '/opt/tomcat/students/max*' -not -
path '/opt/tomcat/students/neetu*' -not -
path '/opt/tomcat/students/shani*' -exec rm {} \;
# All above is one command

find /opt/tomcat/students/simon/ -type f -daystart -mtime +365 -exec rm {} \;

find /opt/tomcat/students/ali/ -type f -daystart -mtime +60 -exec rm {} \;

find /opt/tomcat/students/armand/ -type f -daystart -mtime +90 -exec rm {} \;

find /opt/tomcat/students/max/ -type f -daystart -mtime +15 -exec rm {} \;

find /opt/tomcat/students/shani/ -type f -daystart -mtime +75 -exec rm {} \;

find /opt/tomcat/students/neetu/ -type f -daystart -mtime +100 -exec rm {} \;

and how can i show the deleted data ...? :-(
I put this script in /etc/cron.daily/ (daily executeable)

Any help?

Mr. C. · 09-13-2008, 05:27 PM

Create test data that you can test your script (or slightly modified version). This is where those variables I mentioned above comes in handy.

Use touch with the -t option to create files with old dates; some < 30 days, some > 30 but less than 75 or 100 days, and some over 75 or 100 days (depending upon the user shani or neetu). You can run ls -l of those files, show the output, run your script, and show ls -l after the script completes. The absence of files > 30 days would prove the case. Alternatively, so that you don't have to keep re-creating test days, place echo in front of your rm command so that your find just prints out what it would do.

You've been given hints and help about how to modify your script to aid YOU in debugging, and yet you're not following any advice. I can't tell where lines break because of pasting, or because you have explicitly broken them in your script. I see nothing that indicates the script fails in the way you say it does. Help yourself by following the advice of people who have years and years of experience.

chrism01 · 09-14-2008, 08:16 PM

I dislike really long cmd lines, they get messy & tricky to debug, as mentioned.
I'd loop over all students with a std find cmd and then just 'continue' if current 'student' is an exception, then deal with those later as you have.
http://www.tldp.org/LDP/abs/html/loopcontrol.html