Once a week a backup

alaios · 05-15-2015, 04:23 AM

Dear all,
I have started writing my thesis for my master studies. I am taking backups regularly on an external hard disk (copy paste with my mouse

)

I wanted to ask if I can with my linux have some type of cron job that on Sundays compress a specific folder and send that new zip file over email.

Can I do all these actions with a cron script? Pack files, Compress File, USe unique name and then send an email?

I will write some script later tonight and I would share with code and pseudo code where I find problems in this reasioning I am using.

Regards
Alex

jayakrishnan · 05-15-2015, 04:46 AM

run a cp command using cron

chrism01 · 05-15-2015, 08:39 AM

Absolutely do-able.
Check any num of bash tutorials on the web.
Hint: for debugging, try adding 'set -xv' as the next line after the bash invocation.

alaios · 06-08-2015, 07:22 AM

Dear all,
you will find attached my current version

I have tried to explain in the code what I am doing.
I am also trying to capture error messages and notify me accordingly

Code:

#!/bin/bash

# Description. This script is for compressing my thesis folder and then sending it over email. Once the email is succesfully sent remove the compressed file. 

# What type of checks should I do

SCRIPTNAME=$(basename ${0})
MAILADDRESS="HideForNow"
THESISFOLDER="HideForNow"
TEMPSAVEFOLDER="HideForNow"

doEmail() { case "$1" in
        3)      REASON="Backup Script Finished. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Completed"
                ;;
        4)      REASON="Failed To compress File. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed"
                ;;
        5)      REASON="Failed to send email $(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;
 *) echo "Please don't do that, Dave..." >/dev/stderr;;
esac; echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS; }





# 4. Missing a date format without spaces 
tar -jcvf $(TEMPSAVEFOLDER)myname_thesis$(date).tar.bz2 "$THESISFOLDER$"

# Questions. How to chech the above is correctly completed? How to mail it? and how to remove it afterward from the hard disk. 


# 5.
doEmail 5; exit 0; # Exit 0 when normally returns

Major ones are.
a.Make the compression of the file, sending it over email and removing the compressed file afterwards
b. Use a date time stamp for the file name that has no spaces
c. Track if error happened and report the error. I tried to include the function doEmail that sends the eror message based on the current status.

That would be all for now
Regards
Alex

Beryllos · 06-08-2015, 08:58 AM

I'll start with the easy question: How to list the date without spaces.

You can specify the date format with codes given in the manual (man date).

Examples:

Code:

$ date +F    # This is sufficient for daily backups
2015-06-08
$ date +%F_%T    # This appends the time, but ':' separators may be a problem
2015-06-08_09:53:43
$ date +%F_%H%M%S    # This expresses the time without ':' separators
2015-06-08_095350

It is worth noting that files (or directories) named according to these formats will appear in chronological order when listed in alphabetical order (as with the ls command). Sometimes this is useful for managing multiple backup sets.

Beryllos · 06-08-2015, 10:01 AM

To detect errors and take appropriate actions, you need to test the exit codes. Each command in your script returns an integer value, known as exit code, which can be examined via the variable $? immediately after the command.

Exit code 0 (zero) generally means there was no functional error. There may still be logical errors, as when you specify the wrong files to archive.

If your script requires more specific information about errors, you will have to look up the possible exit codes for each command of interest. Sometimes they are listed in the man pages. Other times you might have to search online (e.g., google tar exit codes).

Example at the command line:

Code:

$ tar -cjf my_sound_files.tar.bz2 sounds
$ echo $?
0

Example in a script:

Code:

#!/bin/bash
# Example of exit code tests

tar -cjf my_sound_files.tar.bz2 sounds

if [[ $? == 0 ]]
then
    echo "no error"
else
    echo "error"
fi

alaios · 06-09-2015, 08:06 AM

Dear all,
I have been using your feedback to improve my script as following
-Using date to create a unique file name that contains no spaces. Since I would be taking these backups one per week the selected date format would be fine.
-Using date at once at the beginning to make a unique file that is used next in the code.
My code should do the following:
-Compress a folder to a file
-Send the file as attachment over email
-Remove the more recent file.

1. Currently I am getting a syntax error but I can not really find it
2. I am also missing the part where I want to remove files (so there are not files accumulated on my hard disk). What would be the safe way to do it.. so acceidentaly one day my script removes files I still need?

I would like to thank you in advance for your further support.
The code follows:

Code:

#!/bin/bash

# Description. This script is for compressing my thesis folder and then sending it over email. Once the email is succesfully sent remove the compressed file. 

# What type of checks should I do

SCRIPTNAME=$(basename ${0})
MAILADDRESS="HideForNow"
THESISFOLDER="HideForNow"
TEMPSAVEFOLDER="HideForNow"

TIMESTAMP=$(data +F)
COMPRESSEDFILENAME=$(TEMPSAVEFOLDER)myname_thesis$(TIMESTAMP).tar.bz2

doEmail() { case "$1" in
        4)      REASON="Failed To compress File. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed"
                ;;
        5)      REASON="Failed to send email $(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;
        6)      REASON="Failed to remove file$(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;

        7)      REASON="Backup Script Finished. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Completed"
                ;;

 *) echo "Please don't do that, Dave..." >/dev/stderr;;
esac; echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS; }





# 4. Compress Email 
tar -jcvf COMPRESSEDFILENAME "$THESISFOLDER"


if [[ $? -ne 0 ]]
    do Email 4; exit 1; # Compressing the file failed
fi

#5. Send the File by email.
mail -r $MAILADDRESS -s "Thesis Backup $TIMESTAMP" -a $COMPRESSEDFILENAME $MAILADDRESS;


if [[ $? -ne 0 ]]
    do Email 5; exit 1; # Sending email failed 
fi


# 6. Remove the file that it was sent over email. Would not be dangerous to call rm and delete also other files? 


# 7. Everything went as planned. Bye By 
doEmail 7; exit 0; # Exit 0 when normally returns

schneidz · 06-09-2015, 08:19 AM

^ i see this:

Quote:

Originally Posted by alaios

...
TIMESTAMP=$(data +F)...

should be

Code:

TIMESTAMP=$(date +%F)

Quote:

Originally Posted by alaios

...
b. Use a date time stamp for the file name that has no spaces

Code:

schneidz@xbmc:~$ date +Y-j-H-%M-%S
2015-160-09-18-09

if you copy-paste the error someone will be able to diagonose the problem easier.

Beryllos · 06-09-2015, 07:39 PM

Quote:

Originally Posted by Beryllos

Code:

$ date +F    # This is sufficient for daily backups
2015-06-08

Sorry, I must have inadvertently deleted the % character after I pasted the command and its output. The command as written above would output only the letter F.

The correct command is:

Code:

$ date +%F
2015-06-09

Beryllos · 06-09-2015, 10:04 PM

Quote:

Originally Posted by alaios

1. Currently I am getting a syntax error but I can not really find it

These are simple mistakes that you should first try to figure out on your own. LinuxQuestions should not be your first (or even second) round of debugging. Run your script, or run fragments of it on the command line, and analyze the error messages. For example, look at this:

Code:

$ TIMESTAMP=$(data +F)
-bash: data: command not found

That tells you that "data" is not a valid command. Let's try "date" instead:

Code:

$ TIMESTAMP=$(date +F)
$ echo $TIMESTAMP
F

Oops! forgot the % character:

Code:

$ TIMESTAMP=$(date +%F)
$ echo $TIMESTAMP
2015-06-09

That's better.

The following error message look similar:

Code:

$ COMPRESSEDFILENAME=$(TEMPSAVEFOLDER)myname_thesis$(TIMESTAMP).tar.bz2
-bash: TEMPSAVEFOLDER: command not found
-bash: TIMESTAMP: command not found

The names "TEMPSAVEFOLDER" and "TIMESTAMP" were used in a place where bash requires a command, so it didn't work. The command may be written in this way:

Code:

COMPRESSEDFILENAME="$TEMPSAVEFOLDER"myname_thesis"$TIMESTAMP".tar.bz2

The error message of an incorrectly formed "if" command is a little tricky:

Code:

if [[ $? == 0 ]]
    echo "It worked."
fi
-bash: syntax error near unexpected token `fi'

but with experience you will come to recognize the meaning of that. It means that something between "if" and "fi" is missing, though it is not always easy to figure out what. In this case, the "then" keyword is missing. This is the correct form:

Code:

if [[ $? == 0 ]]
then
    echo "It worked."
fi

The following line contains an error that will not give an error message:

Code:

tar -jcvf COMPRESSEDFILENAME "$THESISFOLDER"

The only problem is that the tar file will be named "COMPRESSEDFILENAME" literally.

You need the $ character to read out the value of the variable COMPRESSEDFILENAME:

Code:

tar -jcvf "$COMPRESSEDFILENAME" "$THESISFOLDER"

The quotation marks are not necessary in the above line, but some programmers throw them in anyway because they are needed in other situations.

alaios · 06-11-2015, 05:31 AM

Dear all,
Thanks for your help to improve my script.
Now I am able to compress the file correctly .. getting the right timestamp and having the folders with their correct names. Two issues
1 The if statements give me a syntax error
2 I am not sure which would be the safest way to remove a file... What is the less prone mode to write the rm command?

The code with the fixes. ( I turns until and the tar command and fails on the if below)

Code:

#!/bin/bash

# Description. This script is for compressing my thesis folder and then sending it over email. Once the email is succesfully sent remove the compressed file. 

# What type of checks should I do

SCRIPTNAME=$(basename ${0})
MAILADDRESS="HideForNow"
THESISFOLDER="HideForNow"
TEMPSAVEFOLDER="HideForNow"
TIMESTAMP=$(date +%F)
COMPRESSEDFILENAME="$TEMPSAVEFOLDER"myname_thesis"$TIMESTAMP".tar.bz2






doEmail() { case "$1" in
        4)      REASON="Failed To compress File. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed"
                ;;
        5)      REASON="Failed to send email $(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;
        6)      REASON="Failed to remove file$(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;

        7)      REASON="Backup Script Finished. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Completed"
                ;;

 *) echo "Please don't do that, Dave..." >/dev/stderr;;
esac; echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS; }





# 4. Compress Email 
tar -jcvf "$COMPRESSEDFILENAME" "$THESISFOLDER"


if [[ $? == 0 ]]
then
    do Email 4; exit 1; # Compressing the file failed
fi

#5. Send the File by email.
mail -r $MAILADDRESS -s "Thesis Backup $TIMESTAMP" -a $COMPRESSEDFILENAME $MAILADDRESS;


if [[ $? == 0 ]]
then
    do Email 5; exit 1; # Sending email failed 
fi


# 6. Remove the file that it was sent over email. Would not be dangerous to call rm and delete also other files? 


# 7. Everything went as planned. Bye By 
doEmail 7; exit 0; # Exit 0 when normally returns

Beryllos · 06-11-2015, 11:40 AM

Here are a few more things to fix:

Your function is defined as "doEmail", but in two places you invoke it as "do Email". Remove the extra space.
Both of your if statements should test for not-equal, like this:
Code:
```
if [[ $? != 0 ]]
```
To delete the tar file, you could simply use:
Code:
```
rm -f $COMPRESSEDFILENAME
```
I added the -f option so that it will ignore (that is, will not give error message) if the file does not exist.
I recommend putting this rm command in every exit path, to delete the tar file even if there are errors.
You asked:

Quote:

Would not be dangerous to call rm and delete also other files?

I cannot think of any way it could go wrong and delete other files if you use the above command. If $COMPRESSEDFILENAME names an existing file, it will be deleted. If $COMPRESSEDFILENAME names a file that does not exist, the rm command will do nothing. If $COMPRESSEDFILENAME is undefined, the rm command will do nothing.

alaios · 06-16-2015, 02:55 PM

Thajnks I will give it a try when I am back at my office. (I was sick theprevious days)

chrism01 · 06-17-2015, 05:56 AM

1. The comparator (operator) to use for numeric comparisons is '-eq' not '==' even though it may work in this circumstance
http://www.tldp.org/LDP/abs/html/comparison-ops.html

2. post #10: embedded vars.
To embed (interpolate) a var value into a longer string, use curly brackets {} thus

Code:

var1=fred
var3=22
x=string${var1}more${var3}

# then we get 
x=stringfredmore22

HTH

alaios · 06-19-2015, 01:23 AM

Thanks everyone for the great comments.

Now the code seems to work.. up to the point of compressing my thesis directory (I can see the tar.bz2 file in my directory).. although I get an email saying that it failed to compress the file....

I guess my if statements do not check correctly if thinks have progressed.

I also still have commented the following

# 6. Remove the file that it was sent over email. Would not be dangerous to call rm and delete also other files?
#rm -f $COMPRESSEDFILENAME

I am scared that the $COMPRESSEDFILENAME might get some "funny" value and destroy my file system. I am running these scripts as root user and I still do not feel safe.

Regards
Alex

Code:

#!/bin/bash

# Description. This script is for compressing my thesis folder and then sending it over email. Once the email is succesfully sent remove the compressed file. 

# What type of checks should I do

SCRIPTNAME=$(basename ${0})
MAILADDRESS="HideForNow"
THESISFOLDER="HideForNow"
TEMPSAVEFOLDER="HideForNow"
TIMESTAMP=$(date +%F)
COMPRESSEDFILENAME="$TEMPSAVEFOLDER"myname_thesis"$TIMESTAMP".tar.bz2




  doEmail() { case "$1" in
        4)      REASON="Failed To compress File. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed"
                ;;
        5)      REASON="Failed to send email $(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;
        6)      REASON="Failed to remove file$(date). $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Script failed "
                ;;

        7)      REASON="Backup Script Finished. $(echo $SCRIPTNAME)"
                SUBJECT="Thesis Backup Completed"
                ;;

 *) echo "Please don't do that, Dave..." >/dev/stderr;;
esac; echo "${REASON}" | mail -r $MAILADDRESS -s "${SUBJECT}" $MAILADDRESS; }





# 4. Compress Email 
tar -jcvf "$COMPRESSEDFILENAME" "$THESISFOLDER"


if [[ $! -eq 0 ]]
then
    doEmail 4; exit 1; # Compressing the file failed
fi

#5. Send the File by email.
mail -r $MAILADDRESS -s "Thesis Backup $TIMESTAMP" -a $COMPRESSEDFILENAME $MAILADDRESS;


if [[ $! -eq 0 ]]
then
    doEmail 5; exit 1; # Sending email failed 
fi


# 6. Remove the file that it was sent over email. Would not be dangerous to call rm and delete also other files? 
#rm -f $COMPRESSEDFILENAME

# 7. Everything went as planned. Bye By 
doEmail 7; exit 0; # Exit 0 when normally returns