LinuxQuestions.org

LinuxQuestions.org (/questions/)
-   Linux - Newbie (https://www.linuxquestions.org/questions/linux-newbie-8/)
-   -   Find files created on weekend,monthend,quarteley etc (https://www.linuxquestions.org/questions/linux-newbie-8/find-files-created-on-weekend-monthend-quarteley-etc-627967/)

sujithdeva 03-14-2008 02:20 AM

Find files created on weekend,monthend,quarteley etc
 
Hi ALL



I have 365 files created in a folder on daily basis.. its a daily cron created for the backup.

Now i would like to group the files created in end of week ,end of month etc in to a seperate folder .

can any one help me to to find the files using 'find' command to extract those files

Thanks

Sujith

konsolebox 03-14-2008 02:46 AM

For what I know you can only find your files based on the modification time and not on the creation time. At least not with linux e*fs. I'm not sure with other filesystems though. Still there are other possible ways to find the files. Some of these are by:

(a) depending on the average time a backup process goes - if it's just quick, you can base it from the modification time
(b) the filename of the file - are there dates added to the file?
(c) or the contents of the files - If dates are included inside the files, maybe the first lines can help

matthewg42 03-14-2008 02:55 AM

Is there a naming convention for the file names which includes the date somehow?

sujithdeva 03-14-2008 03:15 AM

Yaa each days backup ended with time stamp like "20080313.tar.gz"

i need to get the find or xargs command syntax for listing files created on 7 th day of weeks last day of month etc..

Quote:

Originally Posted by matthewg42 (Post 3088301)
Is there a naming convention for the file names which includes the date somehow?


konsolebox 03-14-2008 03:55 AM

for each 7th day of the month do you mean something like this:
Code:

year=2008
month=01
ls ${year}${month}{07,14,21,28}.tgz

If you mean every weekend and also for the last day, you'll need a function or script that understands the calendar system. You'll use that script to .. for example.. know the number of days in the month in question and the position of the first day of the month .. for example sunday.
You can use time(), date(), mktime() and strtotime() in php but i don't know how to do that in perl. You can use the external command date but only for only the current time or else you'll have to go alternating your clock and that won't be good.

matthewg42 03-14-2008 04:04 AM

Well, xargs is not suitable here, but you can read the output of find into a while loop, and perform some operations on each file to determine which ones you want to handle.

Please provide examples of full file names.

matthewg42 03-14-2008 04:37 AM

Here's an example shell script which will identify files from the end of week and end of month. Note that a date can be BOTH at the end of the week, AND at the end of the month. When you modify this script to do something with the files, make sure you take that into account.

Code:

#!/bin/bash

EOW_DAY=Sun

# prevent locale from affecting date output
export LANG=C
unset LC_TIME
unset LC_ALL

main () {
        for f in "$@"; do
                # chop the .tar.gz off the end
                d=${f%.tar.gz}

                # take the last 8 characters, assumed to be yyyymmdd date
                d=${d:$((${#d}-8)):${#d}}

                # split year, month and day
                y=${d:0:4}
                m=${d:4:2}
                d=${d:6:2}

                # test to see if we have an end of week and end of month date
                if is_end_of_week  $y $m $d ; then
                        echo "$f is end of week"
                fi

                if is_end_of_month $y $m $d ; then
                        echo "$f is end of month"
                fi
        done
}

is_end_of_week () {
        # should get three parameters - year, month, day.
        dow=$(date +%a -d $1-$2-$3)
        if [ "$dow" = "$EOW_DAY" ]; then
                return 0
        else
                return 1
        fi
}

is_end_of_month () {
        # should get three parameters - year, month, day.
        case "$2" in
        01|03|05|07|08|10|20)
                lastday=31
                ;;
        02)
                if [ $(( $1 % 4 )) -eq 0 ]; then
                        # leap year
                        lastday=29
                else
                        lastday=28
                fi
                ;;
        *)
                lastday=30
                ;;
        esac

        if [ "$lastday" = "$3" ]; then
                return 0
        else
                return 1
        fi
}

main "$@"


Save this into a file called myscript.sh, chmod 755 that file, and run it using the full path, and pass the names of files to process as parameters, e.g. if you create the file in your HOME directory, called myscript.sh:
Code:

chmod 755 ~/myscript
cd /where/your/logs/are
~/myscript.sh *.tar.gz


jschiwal 03-14-2008 04:47 AM

Code:

is_end_of_month () {
        # should get three parameters - year, month, day.
        case "$2" in
        01|03|05|07|08|10|20)
                lastday=31
                ;;
        02)
                if [ $(( $1 % 4 )) -eq 0 ]; then
                        # leap year
                        lastday=29
                else
                        lastday=28
                fi
                ;;
        *)
                lastday=30
                ;;
        esac

The rule for a leap year isn't that simple. Don't tell Microsoft however:
http://www.theregister.co.uk/2008/02...ttack_ballmer/

matthewg42 03-14-2008 04:56 AM

Quote:

Originally Posted by jschiwal (Post 3088366)
The rule for a leap year isn't that simple. Don't tell Microsoft however:
http://www.theregister.co.uk/2008/02...ttack_ballmer/

You know the daft thing is that I actually listened to this story being talked about on a podcast and still just wrote this bug. Hmm. How can I vindicate myself? Urr, aha! yes, this script fails to get all leap years correctly as a feature, to maintain compatibility with Microsoft Excel! Refuse to fix! :p

sujithdeva 03-14-2008 05:03 AM

Hi

The full file name is like

mysqldump_20080309.tar.gz
mysqldump_20080310.tar.gz
mysqldump_20080311.tar.gz
mysqldump_20080311.tar.gz
mysqldump_20080313.tar.gz


I need to move the above files into another folder ie created on each weekends (sunday),end day of month etc.. ie for segrating weekend backups,monthly backup etc

Thanks

Sujith

Quote:

Originally Posted by matthewg42 (Post 3088335)
Well, xargs is not suitable here, but you can read the output of find into a while loop, and perform some operations on each file to determine which ones you want to handle.

Please provide examples of full file names.


matthewg42 03-14-2008 05:06 AM

OK, I can't help myself.

Change the functions like this:
Code:

is_end_of_month () {
        # should get three parameters - year, month, day.
        case "$2" in
        01|03|05|07|08|10|20)
                lastday=31
                ;;
        02)

                if is_leap_year $1; then
                        lastday=29
                else
                        lastday=28
                fi
                ;;
        *)
                lastday=30
                ;;
        esac

        if [ "$lastday" = "$3" ]; then
                return 0
        else
                return 1
        fi
}

is_leap_year () {
        # take a year as a parameter
        if [ $(($1 % 4)) -eq 0 ]; then
                if [ $(($1 % 100)) -eq 0 ]; then
                        if [ $(($1 % 400)) -eq 0 ]; then
                                return 0
                        else
                                return 1
                        fi
                else
                        return 0
                fi
        else
                return 1
        fi

}


jschiwal 03-14-2008 05:11 AM

Quote:

Originally Posted by matthewg42 (Post 3088373)
You know the daft thing is that I actually listened to this story being talked about on a podcast and still just wrote this bug. Hmm. How can I vindicate myself? Urr, aha! yes, this script fails to get all leap years correctly as a feature, to maintain compatibility with Microsoft Excel! Refuse to fix! :p

So you listen to Linux Outlaws I'm guessing.

I'm constantly amazed how some things that seem they should be easy in fact cause great difficulty. Different countries had adopted the Gregorian calendar during different years. Different states in the US on different dates as well. If you have a database that records historic events, that needs to be taken into account.

Have you ever heard of leap-seconds? Due to tidal forces, the rotation of the earth slows down over time. Every few years a second is added before the New Year. However, there are two dates in the year when might be added. And the US wants to not do this for the standard atomic clock. I think the argument is to count the seconds from the base time and let the clients do the adjusting.

Funny thing is since 2000, the earth's has started rotating faster! No one knows why. ( At least according to the Wikipedia article on Leap Seconds )

I applied the updates last month and one of them was for a 30 minute adjustment for the time in Venezuela. Their dictator is obviously the center of the Universe! It reminded me of the movie "Banana's" where it was declared that underwear should be worn on the outside of the pants.

As I understand it, there is only one timezone in all of China! I wonder if it is called "CPCT"?

matthewg42 03-14-2008 05:21 AM

Quote:

Originally Posted by jschiwal (Post 3088386)
So you listen to Linux Outlaws I'm guessing.

:D And all the others. I'm a Linux-podcast-o-holic. Linux Outlaws is a nice one.

hro 03-14-2008 07:08 AM

I see this as a classroom assignment for use of 'date' command. For details see 'info date'. Putting everything inside 'find' is just classroom sadism.

Code:

#!/bin/bash
# hro 2008-03-14
for f in /var/log/*.bz2
    do (
    da=$(date -r $f +%Y%m%d)
    weekday=$(date '--date='$da +%w)
    day=${da:6:2}
    month=${da:4:2}
    nextday=$(date '--date=tomorrow '$da +%d)
    echo -n 'm='$month 'd='$day' '
    case $nextday in
        01) echo -n "last day ";;
        *) echo -n "........ ";;
    esac
    case $weekday in
        0 | 6) echo "weekend" $f ;;
        *) echo "workday" $f ;;
    esac
    ) done


jschiwal 03-14-2008 03:13 PM

Quote:

Originally Posted by matthewg42 (Post 3088393)
:D And all the others. I'm a Linux-podcast-o-holic. Linux Outlaws is a nice one.

Don't let Jeremy find out you listen to "The Linux Link Tech Show". Dan sounds like a stalker! :eek:

konsolebox 03-14-2008 11:18 PM

this command doesn't reset the time right?:
Code:

date +%a -d $1-$2-$3
. i haven't seen that :p

jschiwal 03-15-2008 12:27 AM

If you use:
date +%a -d "$day/$month/$year"
This will display the day of the week on that date. I changed the variable names to indicate what they represent.
Code:

day=10 month=12 year=2007
> date +%a -d "$day/$month/$year"
Fri

The computers date & time settings won't be changed.

Your form uses the year for the first variable:

Code:

day=10 month=12 year=2007
jschiwal@hpamd64:~> date +%a -d "$year-$month-$day"
Mon

With this form of date, dates are in alphabetical order.

konsolebox 03-15-2008 12:43 AM

Thanks I know what you mean. In order for the script to work, you'll also need to know what weekday name of a specific date. So I thought of the external date command and looked for it's syntax and I found none and that is why i referred to other functions in php or perl. I should have looked a little bit further in date's options. my mistake again :p.

matthewg42 03-15-2008 01:22 AM

Quote:

Originally Posted by konsolebox (Post 3089138)
this command doesn't reset the time right?:
Code:

date +%a -d $1-$2-$3
. i haven't seen that :p

No. The -d option just means "use a date/time defined in the argument to this parameter", and the +%a just specifies that the output format is using the %a format string (week day). $1 $2 and $3 are the values of the first three arguments to the function - called the "positional parameters" in shell talk. I wrote the function to expect the year, month and day, so putting them together with hyphens between makes a date which the date program can understand, e.g. if the positional parameters are 2007, 02, and 12, the string passed to the date command would be "2007-02-12".

The date command can be used to set the system date, but only if you use the -s (or --set) option, and have sufficient privileges to do it (i.e. you are root).

You should read the date manual page (enter terminal, type "man date"). Manual pages look a little confusing at first, but they all follow more or less the same format, and once you have a little experience reading them, you will be able to answer this sort of question for ourself with ease, and a lot more quickly than by asking others.

To my mind, the really nice thing about manual pages is that they are arranged like a tabloid news story. The first line tries to encapsulate the general gist of the page (i.e. a one line summary of that the subject of the page is), then there's a short overview description and so o, getting more and more detailed towards the end of the page.

Most manual pages have the same main sections, which are printed in CAPTIALS at the start of a line. If your manual browser is set up as most are, using less as a pager, you can search for a section, like OPTIONS easily, just by hitting / and then typing
Code:

^OPT
.
The ^ means "start of line" in less searches.

jschiwal 03-15-2008 01:58 AM

There is also the info manual which does include examples. It really helps to see an example that has a format argument.

konsolebox 03-15-2008 03:02 AM

i did read the manual page of date .. i always do that since i'd prefer and find it quicker to do 'man date' than 'date --help' .. i just have not looked good enough though. btw we can also have our own way to look for the days of the week without depending on 'date'.. i know a way to do this and it's not really big.. if i'll have the time.. i'll build the script.. just for fun :)

Quote:

Originally Posted by jschiwal (Post 3089247)
There is also the info manual which does include examples. It really helps to see an example that has a format argument.

what is not good in info pages is that they are just similar to manual pages like sed.. so sometimes i don't look at them anymore

Edit:
to make things clearer i just looked at these:
Code:

      date [OPTION]... [+FORMAT]
      date [-u|--utc|--universal] [MMDDhhmm[[CC]YY][.ss]]

and not this:
Code:

      -d, --date=STRING
              display time described by STRING, not `now'

obviously in the first ones you won't see an immediate info about being able to add a custom date.. adding a custom was what i was looking for.

hro 03-15-2008 08:45 AM

'info date' info is much more complete than 'man date'


All times are GMT -5. The time now is 06:02 AM.