Find files created on weekend,monthend,quarteley etc

sujithdeva · 03-14-2008, 02:20 AM

Hi ALL

I have 365 files created in a folder on daily basis.. its a daily cron created for the backup.

Now i would like to group the files created in end of week ,end of month etc in to a seperate folder .

can any one help me to to find the files using 'find' command to extract those files

Thanks

Sujith

konsolebox · 03-14-2008, 02:46 AM

For what I know you can only find your files based on the modification time and not on the creation time. At least not with linux e*fs. I'm not sure with other filesystems though. Still there are other possible ways to find the files. Some of these are by:

(a) depending on the average time a backup process goes - if it's just quick, you can base it from the modification time
(b) the filename of the file - are there dates added to the file?
(c) or the contents of the files - If dates are included inside the files, maybe the first lines can help

matthewg42 · 03-14-2008, 02:55 AM

Is there a naming convention for the file names which includes the date somehow?

sujithdeva · 03-14-2008, 03:15 AM

Yaa each days backup ended with time stamp like "20080313.tar.gz"

i need to get the find or xargs command syntax for listing files created on 7 th day of weeks last day of month etc..

Quote:

Originally Posted by matthewg42

Is there a naming convention for the file names which includes the date somehow?

konsolebox · 03-14-2008, 03:55 AM

for each 7th day of the month do you mean something like this:

Code:

year=2008
month=01
ls ${year}${month}{07,14,21,28}.tgz

If you mean every weekend and also for the last day, you'll need a function or script that understands the calendar system. You'll use that script to .. for example.. know the number of days in the month in question and the position of the first day of the month .. for example sunday.
You can use time(), date(), mktime() and strtotime() in php but i don't know how to do that in perl. You can use the external command date but only for only the current time or else you'll have to go alternating your clock and that won't be good.

matthewg42 · 03-14-2008, 04:04 AM

Well, xargs is not suitable here, but you can read the output of find into a while loop, and perform some operations on each file to determine which ones you want to handle.

Please provide examples of full file names.

matthewg42 · 03-14-2008, 04:37 AM

Here's an example shell script which will identify files from the end of week and end of month. Note that a date can be BOTH at the end of the week, AND at the end of the month. When you modify this script to do something with the files, make sure you take that into account.

Code:

#!/bin/bash

EOW_DAY=Sun

# prevent locale from affecting date output
export LANG=C
unset LC_TIME
unset LC_ALL

main () {
        for f in "$@"; do
                # chop the .tar.gz off the end
                d=${f%.tar.gz}

                # take the last 8 characters, assumed to be yyyymmdd date
                d=${d:$((${#d}-8)):${#d}}

                # split year, month and day
                y=${d:0:4}
                m=${d:4:2}
                d=${d:6:2}

                # test to see if we have an end of week and end of month date
                if is_end_of_week  $y $m $d ; then
                        echo "$f is end of week"
                fi

                if is_end_of_month $y $m $d ; then
                        echo "$f is end of month"
                fi
        done
}

is_end_of_week () {
        # should get three parameters - year, month, day.
        dow=$(date +%a -d $1-$2-$3)
        if [ "$dow" = "$EOW_DAY" ]; then
                return 0
        else
                return 1
        fi
}

is_end_of_month () {
        # should get three parameters - year, month, day.
        case "$2" in
        01|03|05|07|08|10|20)
                lastday=31
                ;;
        02)
                if [ $(( $1 % 4 )) -eq 0 ]; then
                        # leap year
                        lastday=29
                else
                        lastday=28
                fi
                ;;
        *)
                lastday=30
                ;;
        esac

        if [ "$lastday" = "$3" ]; then
                return 0
        else
                return 1
        fi
}

main "$@"

Save this into a file called myscript.sh, chmod 755 that file, and run it using the full path, and pass the names of files to process as parameters, e.g. if you create the file in your HOME directory, called myscript.sh:

Code:

chmod 755 ~/myscript
cd /where/your/logs/are
~/myscript.sh *.tar.gz

jschiwal · 03-14-2008, 04:47 AM

Code:

is_end_of_month () {
        # should get three parameters - year, month, day.
        case "$2" in
        01|03|05|07|08|10|20)
                lastday=31
                ;;
        02)
                if [ $(( $1 % 4 )) -eq 0 ]; then
                        # leap year
                        lastday=29
                else
                        lastday=28
                fi
                ;;
        *)
                lastday=30
                ;;
        esac

The rule for a leap year isn't that simple. Don't tell Microsoft however:
http://www.theregister.co.uk/2008/02...ttack_ballmer/

matthewg42 · 03-14-2008, 04:56 AM

Quote:

Originally Posted by jschiwal

The rule for a leap year isn't that simple. Don't tell Microsoft however:
http://www.theregister.co.uk/2008/02...ttack_ballmer/

You know the daft thing is that I actually listened to this story being talked about on a podcast and still just wrote this bug. Hmm. How can I vindicate myself? Urr, aha! yes, this script fails to get all leap years correctly as a feature, to maintain compatibility with Microsoft Excel! Refuse to fix!

sujithdeva · 03-14-2008, 05:03 AM

Hi

The full file name is like

mysqldump_20080309.tar.gz
mysqldump_20080310.tar.gz
mysqldump_20080311.tar.gz
mysqldump_20080311.tar.gz
mysqldump_20080313.tar.gz

I need to move the above files into another folder ie created on each weekends (sunday),end day of month etc.. ie for segrating weekend backups,monthly backup etc

Thanks

Sujith

Quote:

Originally Posted by matthewg42

Well, xargs is not suitable here, but you can read the output of find into a while loop, and perform some operations on each file to determine which ones you want to handle.

Please provide examples of full file names.

matthewg42 · 03-14-2008, 05:06 AM

OK, I can't help myself.

Change the functions like this:

Code:

is_end_of_month () {
        # should get three parameters - year, month, day.
        case "$2" in
        01|03|05|07|08|10|20)
                lastday=31
                ;;
        02)

                if is_leap_year $1; then
                        lastday=29
                else
                        lastday=28
                fi
                ;;
        *)
                lastday=30
                ;;
        esac

        if [ "$lastday" = "$3" ]; then
                return 0
        else
                return 1
        fi
}

is_leap_year () {
        # take a year as a parameter
        if [ $(($1 % 4)) -eq 0 ]; then
                if [ $(($1 % 100)) -eq 0 ]; then
                        if [ $(($1 % 400)) -eq 0 ]; then
                                return 0
                        else
                                return 1
                        fi
                else
                        return 0
                fi
        else
                return 1
        fi

}

jschiwal · 03-14-2008, 05:11 AM

Quote:

Originally Posted by matthewg42

You know the daft thing is that I actually listened to this story being talked about on a podcast and still just wrote this bug. Hmm. How can I vindicate myself? Urr, aha! yes, this script fails to get all leap years correctly as a feature, to maintain compatibility with Microsoft Excel! Refuse to fix!

So you listen to Linux Outlaws I'm guessing.

I'm constantly amazed how some things that seem they should be easy in fact cause great difficulty. Different countries had adopted the Gregorian calendar during different years. Different states in the US on different dates as well. If you have a database that records historic events, that needs to be taken into account.

Have you ever heard of leap-seconds? Due to tidal forces, the rotation of the earth slows down over time. Every few years a second is added before the New Year. However, there are two dates in the year when might be added. And the US wants to not do this for the standard atomic clock. I think the argument is to count the seconds from the base time and let the clients do the adjusting.

Funny thing is since 2000, the earth's has started rotating faster! No one knows why. ( At least according to the Wikipedia article on Leap Seconds )

I applied the updates last month and one of them was for a 30 minute adjustment for the time in Venezuela. Their dictator is obviously the center of the Universe! It reminded me of the movie "Banana's" where it was declared that underwear should be worn on the outside of the pants.

As I understand it, there is only one timezone in all of China! I wonder if it is called "CPCT"?

matthewg42 · 03-14-2008, 05:21 AM

Quote:

Originally Posted by jschiwal

So you listen to Linux Outlaws I'm guessing.

And all the others. I'm a Linux-podcast-o-holic. Linux Outlaws is a nice one.

hro · 03-14-2008, 07:08 AM

I see this as a classroom assignment for use of 'date' command. For details see 'info date'. Putting everything inside 'find' is just classroom sadism.

Code:

#!/bin/bash
# hro 2008-03-14
for f in /var/log/*.bz2
    do (
    da=$(date -r $f +%Y%m%d)
    weekday=$(date '--date='$da +%w)
    day=${da:6:2}
    month=${da:4:2}
    nextday=$(date '--date=tomorrow '$da +%d)
    echo -n 'm='$month 'd='$day' ' 
    case $nextday in
        01) echo -n "last day ";;
        *) echo -n "........ ";;
    esac
    case $weekday in
        0 | 6) echo "weekend" $f ;;
        *) echo "workday" $f ;;
    esac
    ) done

jschiwal · 03-14-2008, 03:13 PM

Quote:

Originally Posted by matthewg42

And all the others. I'm a Linux-podcast-o-holic. Linux Outlaws is a nice one.

Don't let Jeremy find out you listen to "The Linux Link Tech Show". Dan sounds like a stalker!