LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices

Reply
 
Search this Thread
Old 02-03-2012, 02:34 AM   #31
gunnarflax
LQ Newbie
 
Registered: Nov 2010
Posts: 20

Original Poster
Rep: Reputation: 4

Quote:
Originally Posted by Reuti View Post
Looks like it would be nice if find has an option to specify besides -depth something to sort the entries by time, either in each directory or overall.

NB: stat -c %Y "$FILE" (uppercase y) outputs the seconds directly.
I don't use find in the script :S

Thanks for the tip about stat!
 
Old 03-10-2013, 02:45 PM   #32
caco3
LQ Newbie
 
Registered: Jun 2011
Posts: 4

Rep: Reputation: Disabled
Thank you for this great script!
I improved it a bit, so one can also define how many weeks it should keep the backups.
Also I added a bit of documentation and a dry run option:


Code:
#!/bin/bash

#----------------------------------------------#
#    Copyright (C) Niklas Rosenqvist, 2012
#    Improved by George Ruinelli, 2013
#    For updates see http://www.linuxquestions.org/questions/programming-9/how-can-i-purge-files-logarithmically-by-modification-date-bash-923914/page3.html 
#----------------------------------------------#
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.

#----------------------------#
# Smart rm old files
#----------------------------#

# This script will remove all files except the ones that match the following:
# - All files younger than -a days
# - For all files older than -a days, it keeps one per week, but only for -w weeks
# 

# -t: = path to directory to clean
# -a: = amount of days to save all files (default = 2, max 7)
# -w: = amount of weeks to save one file per week (default = 8)
# -d = delete directories as well
# -D = Dry run, do not actually do anything

show_error ()
{
        echo "${PROG_NAME}: ${1:-"Unknown error"}"

        echo "Valid parameters:"
        echo "-t: = path to directory to clean"
        echo "-a: = amount of days to save all files (default = 2, max 7)"
        echo "-w: = amount of weeks to save one file per week (default = 8)"
        echo "-d = delete directories as well"
        echo "-D = Dry run, not actually deleting any file"
}

PROG_NAME=$(basename $0)
TRGT_DIR=""
DAY_RM_THRESHOLD=2
WEEK_RM_THRESHOLD=8
DAY_SPAN=1
DAYS_TO_SAVE=$(date -d "2 days ago" +%s)
DIRECTORIES=1
FOUND_ONE=1
DRY_RUN=0

while getopts ":t:a:w:d:D" opt; do
        case $opt in
                t)
                        TRGT_DIR=$OPTARG
                ;;
                a)
                        #Check if age is an integer
                        if ! [[ "$OPTARG" =~ [0-97]+$ ]]; then
                                show_error "[ERROR]: Age (-a) must be an integer with the value 1-7"
                                exit 1
                        fi

                        if [ $OPTARG -lt 7 ]; then
                                DAY_RM_THRESHOLD=$OPTARG
                                DAYS_TO_SAVE=$(date -d "$(($OPTARG)) days ago" +%s)
                        else
                                DAY_RM_THRESHOLD=7
#                                 show_error "[WARNING]: Limiting age (-a) to 7"
                                DAYS_TO_SAVE=$(date -d "7 days ago" +%s)
                                DAY_SPAN=7
                        fi
                ;;
                w)
                        #Set weeks
                        if ! [[ "$OPTARG" =~ [0-97]+$ ]]; then
                                show_error "[ERROR]: Week (-w) must be an integer"
                                exit 1
                        fi

                        WEEK_RM_THRESHOLD=$OPTARG
                ;;
                d)
                        DIRECTORIES=0
                ;;
                D)
                        DRY_RUN=1
                ;;
        esac
done

#Reset $OPTIND
OPTIND=1

#Target must be a directory
if [ ! -d "$TRGT_DIR" ]; then
        show_error "[ERROR]: The target must exist and be a directory." 
        exit 1
fi

echo "[INFO]: Starting the logarithmic backup cleaning."

if [ $DRY_RUN -eq 1 ]; then
    echo "[INFO]: We are in dry run mode, not actually deleting any file!"
fi

echo "DAY_RM_THRESHOLD=$DAY_RM_THRESHOLD"
echo "WEEK_RM_THRESHOLD=$WEEK_RM_THRESHOLD"

#Find files to remove and put them in "files_to_remove.tmp"
ls -1 -t "$TRGT_DIR/" -I "*~" | while read FILE
do
        #Get the file's modification date
        FILE="$TRGT_DIR/$FILE"
        MTIME=$(date -d "$(stat -c %y "$FILE")" +%s)

        #If it's within the range to save all files we skip this one
        if [ $MTIME -ge $DAYS_TO_SAVE ]; then
                echo "Keep   $FILE"
                continue
        fi

        #Check if we should skip directories
        if [ $DIRECTORIES -eq 1 ] && [ -d "$FILE" ]; then
                continue
        fi

        #Increase the day span accordingly
        if [ $DAY_RM_THRESHOLD -lt 7 ]; then
                DAY_SPAN=1
        elif [ $DAY_RM_THRESHOLD -ge 7 ] && [ $DAY_RM_THRESHOLD -lt $((WEEK_RM_THRESHOLD*7)) ]; then
                DAY_SPAN=7
        elif [ $DAY_RM_THRESHOLD -ge 28 ] && [ $DAY_RM_THRESHOLD -lt $((28+30*11)) ]; then
                DAY_SPAN=30
        else
                DAY_SPAN=365
        fi

        #If the file's modification time is earlier than our date range we push it back one $DAY_SPAN
        if [ $MTIME -lt $(date -d "$(($DAY_RM_THRESHOLD+$DAY_SPAN)) days ago" +%s) ]; then
                DAY_RM_THRESHOLD=$(($DAY_RM_THRESHOLD+$DAY_SPAN))
                FOUND_ONE=1
        fi

        #Get date range
        DATE_END=$(date -d "$DAY_RM_THRESHOLD days ago" +%s)
        DATE_START=$(date -d "$(($DAY_RM_THRESHOLD+$DAY_SPAN)) days ago" +%s)

        #Have we found one to keep for this day?
        if [ $FOUND_ONE -eq 1 ] && [ $MTIME -ge $DATE_START ] && [ $MTIME -lt $DATE_END ]; then
                FOUND_ONE=0
                echo "Keep   $FILE"
        else
                echo "Remove $FILE"

                if [ $DRY_RUN -eq 0 ]; then
                      rm -R "$FILE"
                fi
        fi
done
#done | xargs -d '\n' rm -R

echo "[INFO]: Cleaning of old files complete!"

exit 0
 
  


Reply

Tags
bash, purge, smart, ubuntu


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Get file modification date/time in Bash script cmfarley19 Programming 12 01-19-2013 09:37 AM
Bash Script to Copy Modification Date from a file to his folder pjgm Programming 12 07-31-2011 08:33 AM
[SOLVED] copying files according to modification date and extension SriniKlr Linux - Newbie 5 01-03-2011 03:45 AM
[SOLVED] merge files by creation/modification date? andre.fm Linux - Newbie 5 10-04-2010 06:41 PM
copy folder/files according to modification date bkcreddy17 Programming 14 10-15-2008 07:24 PM


All times are GMT -5. The time now is 08:46 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration