Bash Scripting with Dates

MrBriz · 11-21-2008, 07:48 AM

Hi Everyone,

I have a folder structure on my Mac as follows...

Images
2007-11-21
work
home
2007-11-22
work
home
2007-11-23
work
home
2007-11-24
work
home
~~~~~~~~~~~~~~~~~
2008-11-23
work
home
2008-11-24
work
home
Etc..

What I want to do is write a little bash script that takes two dates and if I want the work or home folders then loops through the entire directory between the given dates and gets all the images from either the work or home folders, then copies those images into a temporary dir giving them a sequential number.

Any ideas? So far I have (and it's not much!)...

--------------------------
#!/bin/bash
# Input Date 1
echo -n "Input date from (YYYY-MM-DD) [ENTER]: "
read FROM_DATE

# Input Date 2
echo -n "Input date to (YYYY-MM-DD) [ENTER]: "
read TO_DATE

# Work or Home
echo -n "Work or Home (w or h) [ENTER]: "
read W_OR_H

# Loop through folders between date ranges

# Loop through images in folder

# Copy image to temp dir with incrementing name
--------------------------

Tinkster · 11-21-2008, 11:19 AM

Hi, and welcome to LQ!

If it wasn't for the hyphens in the directories names
it would be as easy as

Code:

for i in {20081103..20081106}; do
  do something with ${i}
done

No maths required.

Cheers,
Tink

colucix · 11-21-2008, 11:43 AM

Once you've got the dates you can increment FROM_DATE by one in a while loop, until it reaches TO_DATE + 1. For example:

Code:

while [ $FROM_DATE \< $(date -d "$TO_DATE 1 day" +%Y-%m-%d) ]
do
  echo processing $FROM_DATE
  % your commands here
  FROM_DATE=$(date -d "$FROM_DATE 1 day" +%Y-%m-%d)
done

Note that the ASCII comparison is valid in this case, since you have the date in yyyymmdd format (with hyphen or any other characters in it). Also you can skip the echo command to prompt the user, by means of the -p option of read.

Tinkster · 11-21-2008, 11:46 AM

And I just realised that I can't use a variable in {..};
seq however, works.

Code:

for i in $( seq $from_date $to_date ); do
  do something with ${i}
done

colucix · 11-21-2008, 11:52 AM

Tinkster, your approach is good, but only for dates belonging to the same month and year. If the start date is 2008-10-23 and the end date is 2008-11-02, it fails. I think some processing using the date command is always necessary.

Tinkster · 11-21-2008, 11:54 AM

True that ... guess I'm not quite up too scratch; too early in the morning ;D

teknik · 11-21-2008, 12:16 PM

look up the "find" command. http://content.hccfl.edu/pollock/Unix/FindCmd.htm

At a quick glance, you could probably use something like the -mmin option.

Something like:

Code:

# should find all files in current dir (and its sub dirs) that
# are last modified between MIN_AGO_LOWER and MIN_AGO_UPPER *minutes* ago
find . -mmin +${MIN_AGO_LOWER} -mmin -${MIN_AGO_UPPER} -name '*.

Poking around, I found this thread:
http://www.unix.com/tips-tutorials/3...tion-bash.html

Then I put the following script together (UNTESTED, no idea if it actually works, but its a start)

Code:

#!/bin/bash


#procs

date2stamp () {
    date --utc --date "$1" +%s
}

stamp2date (){
    date --utc --date "1970-01-01 $1 sec" "+%Y-%m-%d %T"
}

dateDiff (){
    case $1 in
        -s)   sec=1;      shift;;
        -m)   sec=60;     shift;;
        -h)   sec=3600;   shift;;
        -d)   sec=86400;  shift;;
        *)    sec=86400;;
    esac
    dte1=$(date2stamp $1)
    dte2=$(date2stamp $2)
    diffSec=$((dte2-dte1))
    if ((diffSec < 0)); then abs=-1; else abs=1; fi
    echo $((diffSec/sec*abs))
}

# vars

LOWER_DATE=$1 # furthest away date
UPPER_DATE=$2 # closer date
TARGET_DIR=$3 # path to your home or work folder NO TRAILING SLASH, but PREFIX WITH ./ if under current dir

LOWER=$(dateDiff -m "${LOWER_DATE}" "now")
UPPER=$(dateDiff -m "${UPPER_DATE}" "now")

# you can change the first ./ to match whatever is the root directory of where you'll be looking
find ./ -mmin +${LOWER} -mmin -${UPPER} -iregex '.*\(\.jpg\|\.gif\|\.png\)' -not -iregex '${TGT_DIR}/\(.*\)' -exec cp {} ${TGT_DIR}/ \;

Call it like ./whatever_you_called_it.sh 2008-11-15 2008-11-20 ./BACKUPS

Again, no idea if it works (if it does, theres a bonus

)....but it should give you some ideas.

jlinkels · 11-21-2008, 04:57 PM

Colocix' solution seems to be the simplest one.

However, when performing date calculations, it is almost ALWAYS better to convert the time to an integer, being the Unix time stamp.

Code:

mytimestamp=$(date +%s)

Converting back is easy:

Code:

date -d "1970-01-01 $mytimestamp sec"

For this particular problem, adding one day offset is fine, it works even on February 28.

For all other problems, not using entire days, converting to a timestamp is almost mandatory. The date function never misses!

jlinkels

MrBriz · 11-22-2008, 06:42 AM

Wow, I woke up to a whole bunch of responses! I was a little despondent last night as I hadn't yet seen any!

Thanks everyone, I will tinker and let everyone know the outcome (probably take me a week or so as I'm a lazy sod and never find time to program at home as the xbox steals it all!)

Thanks again,

James AKA MrBriz

MrBriz · 11-22-2008, 06:45 PM

Done....

#!/bin/bash
# Vars

FROM_DATE=$1
TO_DATE=$2
W_OR_H=$3

TEMP_DIR=temp
BASE_FOLDER=/Users/jandc/Library/isightcapture

TEMP_PATH=$BASE_FOLDER/$TEMP_DIR

LOG_FILE=$BASE_FOLDER/log

EXPECTED_ARGS=3
E_BADARGS=65

if [ $# -ne $EXPECTED_ARGS ]
then
echo "Usage: `basename $0` [FROM YYYY-MM-DD] [TO YY-MM-DD] [(W)ork or (H)ome]"
exit $E_BADARGS
fi

case $W_OR_H in
W)
W_OR_H=work
;;
H)
W_OR_H=home
;;
esac

# Increase the TO_DATE as I can't seem to figure out how to do <= !!!!???

TO_YEAR=$(echo $TO_DATE | cut -d "-" -f1)
TO_MON=$(echo $TO_DATE | cut -d "-" -f2)
TO_DAY=$(echo $TO_DATE | cut -d "-" -f3)

TO_DATE=$(date -v${TO_YEAR}y -v${TO_MON}m -v${TO_DAY}d -v+1d +%Y-%m-%d)

# Empty the log file
touch $LOG_FILE
echo "" > $LOG_FILE

# Empty the temp dir
rm -Rf $TEMP_PATH
mkdir $TEMP_PATH

# Set count

COUNT=1

while [ $FROM_DATE \< $TO_DATE ]
do
echo processing $FROM_DATE/$W_OR_H

# Check the directory exists, if not skip to next

# Loop through all the images in the directory

COUNT=$(
ls $BASE_FOLDER/$FROM_DATE/$W_OR_H/*.jpg | {
while read FILE ; do
# Copy the current to the image to the temp directory with count as the name
#echo " Copy $FILE to $TEMP_PATH/$COUNT.jpg"
echo Copy $FILE to $TEMP_PATH/$COUNT.jpg >> $LOG_FILE
# Increment count
cp $FILE $TEMP_PATH/$COUNT.jpg
COUNT=$(($COUNT+1))
done
echo $COUNT
}
)

# Increment date

FROM_YEAR=$(echo $FROM_DATE | cut -d "-" -f1)
FROM_MON=$(echo $FROM_DATE | cut -d "-" -f2)
FROM_DAY=$(echo $FROM_DATE | cut -d "-" -f3)

FROM_DATE=$(date -v${FROM_YEAR}y -v${FROM_MON}m -v${FROM_DAY}d -v+1d +%Y-%m-%d)

done

colucix · 11-23-2008, 05:36 AM

Just out of curiosity... does it work??? I'm asking because I've never seen the -v option to the date command. Indeed it is not valid on my system:

Code:

$ TO_DATE=2008-11-08
$ TO_YEAR=$(echo $TO_DATE | cut -d "-" -f1)
$ TO_MON=$(echo $TO_DATE | cut -d "-" -f2)
$ TO_DAY=$(echo $TO_DATE | cut -d "-" -f3)
$ TO_DATE=$(date -v${TO_YEAR}y -v${TO_MON}m -v${TO_DAY}d -v+1d +%Y-%m-%d)
date: invalid option -- v
Try `date --help' for more information.

MrBriz · 11-23-2008, 08:07 AM

Quote:

Originally Posted by colucix

Just out of curiosity... does it work??? I'm asking because I've never seen the -v option to the date command. Indeed it is not valid on my system:

Code:

$ TO_DATE=2008-11-08
$ TO_YEAR=$(echo $TO_DATE | cut -d "-" -f1)
$ TO_MON=$(echo $TO_DATE | cut -d "-" -f2)
$ TO_DAY=$(echo $TO_DATE | cut -d "-" -f3)
$ TO_DATE=$(date -v${TO_YEAR}y -v${TO_MON}m -v${TO_DAY}d -v+1d +%Y-%m-%d)
date: invalid option -- v
Try `date --help' for more information.

Yes, it does

I'm running it on a Mac with OS X 10.5

I tried doing it other ways but that was the only way I could get it working. Plus the stupid variable scope issues with the counter inside the pipped ls while loop got me stumped for a while!

Only thing I'm trying to do no is while looping through the images adding a text string to the bottom of the image so that when I create the timelapse photo I can notice the days change as I'm so good at positioning the camera in the same place if I don't change the clothes I'm wearing I can't see the change!

Anyway thanks for all the help it's been real handy.... all I need to do now is get imagemagick to work!

MrBriz · 11-23-2008, 02:40 PM

Quote:

Originally Posted by colucix

Just out of curiosity... does it work??? I'm asking because I've never seen the -v option to the date command. Indeed it is not valid on my system:

Code:

$ TO_DATE=2008-11-08
$ TO_YEAR=$(echo $TO_DATE | cut -d "-" -f1)
$ TO_MON=$(echo $TO_DATE | cut -d "-" -f2)
$ TO_DAY=$(echo $TO_DATE | cut -d "-" -f3)
$ TO_DATE=$(date -v${TO_YEAR}y -v${TO_MON}m -v${TO_DAY}d -v+1d +%Y-%m-%d)
date: invalid option -- v
Try `date --help' for more information.

Just so you know it ends up being...

date -v2008y -v03m -v01d -v+1d +%Y-%m-%d

Would echo out

2008-03-02

And

date -v2008y -v03m -v01d -v+1d +%Y-%m-%d

would echo out

2008-02-29

colucix · 11-23-2008, 03:04 PM

Quote:

Originally Posted by MrBriz

Yes, it does

I'm running it on a Mac with OS X 10.5

I should have imagined. Mac OS X is something unknown to me. Thanks for your clarification.