Archive Date Path

hattori.hanzo · 11-15-2010, 10:59 PM

I have a script which archives the processed files to an 'archive' directory.

Code:

DATE=$(date "+%Y%m%d%H%M")
cp $LOGFOLDER\data/${HOST[$i]}.csv $LOGFOLDER\archive/${HOST[$i]}_$DATE.csv
gzip $LOGFOLDER\archive/${HOST[$i]}_$DATE.csv
md5sum $LOGFOLDER\archive/${HOST[$i]}_$DATE.csv.gz > $LOGFOLDER\archive/${HOST[$i]}_$DATE.csv.gz.md5

I would like to organize the archives by directory 'days' for example like this:

Code:

/Archives/2010/11/16/
/Archives/2010/11/17/
/Archives/2010/11/18/

Is it as simple as appending the year, month & day to the file path?

Thanks & Regards,

grail · 11-15-2010, 11:48 PM

Well I am not real sure I understand the correlation between your current information and the desired setup.

Quote:

Is it as simple as appending the year, month & day to the file path?

yes it is that simple to use mkdir create the directory structure you require

Is there a relationship between archive and Archives?

How are you making the directory structure currently?

hattori.hanzo · 11-16-2010, 12:16 AM

Thanks for the response.

So I would do something like this? Sorry, I dont have access to a shell prompt at this moment in time.

Code:

DATE=$(date "+%Y%m%d%H%M")
YEAR=$(date "+%Y")
MONTH=$(date "+%m")
DAY=$(date "+%d")

mkdir $LOGFOLDER\data\$YEAR\$MONTH\$DAY
cp $LOGFOLDER\data/${HOST[$i]}.csv $LOGFOLDER\archive\$YEAR\$MONTH\DAY/${HOST[$i]}_$DATE.csv
gzip $LOGFOLDER\archive\$YEAR\$MONTH\DAY/${HOST[$i]}_$DATE.csv
md5sum $LOGFOLDER\archive\$YEAR\$MONTH\DAY/${HOST[$i]}_$DATE.csv.gz > $LOGFOLDER\archive\$YEAR\$MONTH\DAY/${HOST[$i]}_$DATE.csv.gz.md5

catkin · 11-16-2010, 12:23 AM

Quote:

Originally Posted by hattori.hanzo

So I would do something like this?

Yes but it would be shorter and easier to understand (and would avoid the DAY for $DAY typo) as

Code:

DATE=$(date "+%Y%m%d%H%M")
YEAR=$(date "+%Y")
MONTH=$(date "+%m")
DAY=$(date "+%d")
DATA_DIR=$LOGFOLDER\data\$YEAR\$MONTH\$DAY
ARCHIVE_DIR=$LOGFOLDER\archive\$YEAR\$MONTH\$DAY

mkdir $DATA_DIR $ARCHIVE_DIR
cp $LOGFOLDER\data/${HOST[$i]}.csv $DATA_DIR/${HOST[$i]}_$DATE.csv
gzip $ARCHIVE_DIR/${HOST[$i]}_$DATE.csv
md5sum $ARCHIVE_DIR/${HOST[$i]}_$DATE.csv.gz > $ARCHIVE_DIR/${HOST[$i]}_$DATE.csv.gz.md5

EDIT: I just noticed that the original scrippet makes the data directory but does not use it, using the archive directory instead. Dropping the data directory and adding BASENAME it becomes:

Code:

DATE=$(date "+%Y%m%d%H%M")
YEAR=$(date "+%Y")
MONTH=$(date "+%m")
DAY=$(date "+%d")
DIR=$LOGFOLDER\archive\$YEAR\$MONTH\$DAY
BASENAME=${HOST[$i]}.csv

mkdir $DIR
cp $LOGFOLDER\data/$BASENAME $DIR
gzip $DIR/$BASENAME
md5sum $DIR/$BASENAME.gz > $DIR/$BASENAME.gz.md5

grail · 11-16-2010, 12:23 AM

Well I am not sure why some of your slashes are in different directions, ie / or \, but as the structure may not already exist
you should use the -p option to be sure to create any parents required:

Code:

mkdir -p $LOGFOLDER\data\$YEAR\$MONTH\$DAY

catkin · 11-16-2010, 12:39 AM

Quote:

Originally Posted by grail

Well I am not sure why some of your slashes are in different directions, ie / or \

Hi grail

Oops! I missed that! It will cause breakage. Here's what happens testing at the command line:

Code:

c@CW8:~$ $ARCHIVE_DIR/$BASENAME^C
c@CW8:~$ DATE=$(date "+%Y%m%d%H%M")
c@CW8:~$ YEAR=$(date "+%Y")
c@CW8:~$ MONTH=$(date "+%m")
c@CW8:~$ DAY=$(date "+%d")
c@CW8:~$ DIR=$LOGFOLDER\data\$YEAR\$MONTH\$DAY
c@CW8:~$ echo $DIR
data$YEAR$MONTH$DAY

Must be DIR=$LOGFOLDER/data/$YEAR/$MONTH/$DAY