LinuxQuestions.org
Welcome to the most active Linux Forum on the web.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-01-2011, 11:39 AM   #16
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59

Quote:
Originally Posted by chandkur View Post
hi I got this error while executing your script. I was attempting to try on files with .log1 extension.

[ddmsuser@WTXTT02 download-data]$ ./backItUp.sh /home/ddmsuser/DDMSPurge/*.log1
/home/ddmsuser/DDMSPurge/*.log1 does not exist.
That error is correct. That means there is no such file in the directory you have specified that ends in .log1.

You do this:

Code:
ls -l /home/ddmsuser/DDMSPurge/*.log1
See, if you get any output or any errors. Post the output here.

See this test also:

Code:
-bash-2.05b# touch /home/demo/logfile.log1
-bash-2.05b# ls -l /home/demo/logfile.log1
-rw-r--r--  1 root root 0 Mar  1 18:15 /home/demo/logfile.log1
-bash-2.05b# ./backItUp.sh /home/demo/logfile.log1
Destination Backup File Name: backup
/home/demo/logfile.log1
Archiving completed successfully.

/home/demo/logfile.log1 successfully deleted.

backup-01-03-2011.tar:   98.9% -- replaced with backup-01-03-2011.tar.gz
Compressing completed successfully.

backup-01-03-2011.tar.gz:        OK
gzip testing completed successfully.

-rw-r--r--  1 root root 150 Mar  1 18:15 backup-01-03-2011.tar.gz
#######DONE#######.
-bash-2.05b#
Now check this one which has error:

Code:
-bash-2.05b# touch /home/demo/logfile.log2
-bash-2.05b# ls -l /home/demo/logfile*
-rw-r--r--  1 root root 0 Mar  1 18:16 /home/demo/logfile.log2
-bash-2.05b# ./backItUp.sh /home/demo/logfile.log1
/home/demo/logfile.log1 does not exist.

-bash-2.05b#
In the script this block of codes checks for the existance of each file / directory you specify:

Code:
for File in $*; do
        if [ ! -e $File ]; then
                echo -e "$File does not exist.\n"
                exit 1
        fi
done
You see, if the file test fails then you get that error and the script execution is halted.

Last edited by devUnix; 03-01-2011 at 11:42 AM.
 
Old 03-01-2011, 11:42 AM   #17
chandkur
LQ Newbie
 
Registered: Feb 2011
Posts: 16

Original Poster
Rep: Reputation: 0
I guess your name is mahadeva from the script. Sorry for the previous post, i tried again for .txt files and it seems to work fine..
Now I am trying to tweak your code, so it does two things :
1. Accept a second parameter i.e destination directory
2. copy this gzip file to the destination directory.


Thanks for all the help.
 
Old 03-01-2011, 11:44 AM   #18
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Quote:
Originally Posted by chandkur View Post
Hi yaar, thanks for all the help.. maine aapki hindi mein comments nahin padatha.. sach mein accha laga aapke comments pad kar..
array yaar.. mein ne test kiya aapki nayi script se.. par wo error aya hai..
mein buffalo NY mein kaam karta hoon yaar.. aap kaha se ho ?
Tech M mein hoon yaar. Recently join kiya hai yaha pe. AT&T ke liye kaam kar raha hoon Noida mein.
 
Old 03-01-2011, 11:50 AM   #19
chandkur
LQ Newbie
 
Registered: Feb 2011
Posts: 16

Original Poster
Rep: Reputation: 0
Oh great, isn't Tech M now owns Satyam too right ?
Mein idhar Fedex mein kaam karta hoon.

mujhe thodisi aur help kardena yaar..
1. as I might configure this as cron job/task, I dont want someone to enter the archive name, can we take care of it inside the script itself.

2. and as i mentioned in the previous post, want this script to accept a second parameter that will be the destination directory , where the archived file will be copied.

Thanks
Chandra
 
Old 03-01-2011, 11:51 AM   #20
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
It does these things:

It checks for the existance of a source file or directory name or file globbing such as *.log.

If that is fine then it asks you to enter a name for the target file name that will be used for the archive (using tar) and it will be compressed (using gzip).

We are also appending dd-mm-yyyy to the target name. (If you want, you can modify that line and include hh-mm-ss also and the rest of the script will not require any achanges because thereafter we are using the variable to substitute its value wherever required. I had not done that in the first script. So, you see, this is how variables are useful.

Once everything is fine then we go ahead with creating the archive and then check it if it does the work as expected otherwise throw an error. Then gziping the archive and finally checking if we get the target file or not. That is all to it!
 
Old 03-01-2011, 11:54 AM   #21
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Quote:
Originally Posted by chandkur View Post
mujhe thodisi aur help kardena yaar..
1. as I might configure this as cron job/task, I dont want someone to enter the archive name, can we take care of it inside the script itself.

2. and as i mentioned in the previous post, want this script to accept a second parameter that will be the destination directory , where the archived file will be copied.
Okay, I had guessed so that you would be using it with cron. Yes, that that is where the script becomes more useful. I will get back to you in some minutes. I may take some half an hour as I am looking around some other stuffs also. Okay?
 
Old 03-01-2011, 12:05 PM   #22
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Quote:
Originally Posted by chandkur View Post
1. as I might configure this as cron job/task, I dont want someone to enter the archive name, can we take care of it inside the script itself.

2. and as i mentioned in the previous post, want this script to accept a second parameter that will be the destination directory , where the archived file will be copied.
Without modifying the script, we can still achieve them:

echo "/myBackupDir/tmp.bk" | backItUp.sh /tmp


To see how it is working, just add the line in bold:

BackupFile=${REPLY}-$(date +%d-%m-%Y).tar
echo -e "The Target Backup File Name is: $BackupFile\n"

Actually, what you are echoing will become the value to the $REPLY variable. Trick!

Last edited by devUnix; 03-01-2011 at 12:06 PM.
 
Old 03-01-2011, 12:06 PM   #23
chandkur
LQ Newbie
 
Registered: Feb 2011
Posts: 16

Original Poster
Rep: Reputation: 0
Quote:
I will get back to you in some minutes. I may take some half an hour as I am looking around some other stuffs also. Okay?
Sure no problem, let me know when you are ready.
 
Old 03-01-2011, 12:33 PM   #24
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Chanrda, we do not need to change the script as I showed above.

See this cronjob output send in email:

Code:
Date: Tue, 1 Mar 2011 19:04:00 GMT
From: root@host-6-153.linuxzoo.net (Cron Daemon)
To: root@host-6-153.linuxzoo.net
Subject: Cron <root@host-6-153> echo "/tmp/backup-logs" | /root/bin/backItUp.sh /tmp/logs/*.log
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>

Destination Backup File Name: The Target Backup File Name is: /tmp/backup-logs-01-03-2011.tar

/tmp/logs/1.log
/tmp/logs/2.log
Archiving completed successfully.

/tmp/logs/1.log successfully deleted.

/tmp/logs/2.log successfully deleted.

/tmp/backup-logs-01-03-2011.tar:         98.9% -- replaced with /tmp/backup-logs-01-03-2011.tar.gz
Compressing completed successfully.

/tmp/backup-logs-01-03-2011.tar.gz:      OK
gzip testing completed successfully.

-rw-r--r--  1 root root 156 Mar  1 19:04 /tmp/backup-logs-01-03-2011.tar.gz
#######DONE#######.
Now check this one (as the log files are already gone! (See above)):
Code:
Date: Tue, 1 Mar 2011 19:06:01 GMT
From: root@host-6-153.linuxzoo.net (Cron Daemon)
To: root@host-6-153.linuxzoo.net
Subject: Cron <root@host-6-153> echo "/tmp/backup-logs" | /root/bin/backItUp.sh /tmp/logs/*.log
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>

/tmp/logs/*.log does not exist.
Here is the cron job I have scheduled to be run every minute (you choose your own date and time and minute, etc.):

Code:
*/1     *       *       *       *       echo "/tmp/backup-logs" | /root/bin/backItUp.sh /tmp/logs/*.log
Here are some sample files:
Code:
-bash-2.05b# mkdir /tmp/logs/
-bash-2.05b# cd /tmp/logs/
-bash-2.05b# touch 1.log 1.txt 1.tmp 2.log 3.log1
Code:
-bash-2.05b# ls -l /tmp
total 12
-rw-r--r--  1 root root  156 Mar  1 19:04 backup-logs-01-03-2011.tar.gz
drwxr-xr-x  2 root root 4096 Mar  1 19:04 logs
-bash-2.05b#
You specify whatever directory you want to use as a storehouse for your archives. I have used /tmp in the example.

Yes, yaar it works fine. That is how Unix / Linux commands are designed for use. The power comes from the use of pipe- |.


We can however still do one thing that target file name be created automatically. We can use only timestamp as the name of the archive. So that if you create two archives for the same date then they are not overwritten.

Just edit this line:

Code:
BackupFile=${REPLY}/$(date +%d-%m-%Y).tar
OR

Code:
BackupFile=${REPLY}/$(date +%d-%m-%Y-%T).tar
We have only replaced the "-" with the "/" so that the name that you supply becomes the directory name under which something like this:

01-03-2011-19:15:43.tar will be created and then it will finally become 01-03-2011-19:15:43.tar.gz and you provide the directory name in the echo statement.

Last edited by devUnix; 03-01-2011 at 12:42 PM.
 
Old 03-01-2011, 12:43 PM   #25
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Does that solve your problem? Give it some tries befor you node in agreement!
 
Old 03-01-2011, 12:50 PM   #26
chandkur
LQ Newbie
 
Registered: Feb 2011
Posts: 16

Original Poster
Rep: Reputation: 0
Two things,

First in order to remove the user interaction for entering the backup/archive name, i would remove these two lines from the script :
Code:
echo -n "Destination Backup File Name: "
read
if [ `echo $REPLY | awk '{print length}'` -eq 0 ]; then
        echo -e "Destination Backup File Name is Required.\n"
        exit 1
fi
and instead will assign REPLY = ChandraBackups.

Please tell me if this is right.

Next as the archive is getting created in the same directory from where the script is executed. I would like script to accept a parameter for destination path( some other location) where the archive file has to be created.
Please help me with this functionality.
 
Old 03-01-2011, 01:03 PM   #27
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Quote:
Originally Posted by chandkur View Post
Two things,

First in order to remove the user interaction for entering the backup/archive name, i would remove these two lines from the script :
Code:
echo -n "Destination Backup File Name: "
read
if [ `echo $REPLY | awk '{print length}'` -eq 0 ]; then
        echo -e "Destination Backup File Name is Required.\n"
        exit 1
fi
and instead will assign REPLY = ChandraBackups.

Please tell me if this is right.

Next as the archive is getting created in the same directory from where the script is executed. I would like script to accept a parameter for destination path( some other location) where the archive file has to be created.
Please help me with this functionality.

Edit this line as it is here (I by mistake used %T for time which will not work):

Code:
BackupFile=${REPLY}/$(date +%d-%m-%Y-%H-%M-%S).tar
And try in a crontab:

Crontab:

Code:
*/1     *       *       *       *       echo "/tmp/backup-logs" | /root/bin/backItUp.sh /tmp/logs/*.log
Output:

Code:
Date: Tue, 1 Mar 2011 19:31:01 GMT
From: root@host-6-153.linuxzoo.net (Cron Daemon)
To: root@host-6-153.linuxzoo.net
Subject: Cron <root@host-6-153> echo "/tmp/backup-logs" | /root/bin/backItUp.sh /tmp/logs/*.log
X-Cron-Env: <SHELL=/bin/sh>
X-Cron-Env: <HOME=/root>
X-Cron-Env: <PATH=/usr/bin:/bin>
X-Cron-Env: <LOGNAME=root>

Destination Backup File Name: The Target Backup File Name is: /tmp/backup-logs/01-03-2011-19-31-01.tar

/tmp/logs/1.log
/tmp/logs/2.log
Archiving completed successfully.

/tmp/logs/1.log successfully deleted.

/tmp/logs/2.log successfully deleted.

/tmp/backup-logs/01-03-2011-19-31-01.tar:        98.9% -- replaced with
+/tmp/backup-logs/01-03-2011-19-31-01.tar.gz
Compressing completed successfully.

/tmp/backup-logs/01-03-2011-19-31-01.tar.gz:     OK
gzip testing completed successfully.

-rw-r--r--  1 root root 154 Mar  1 19:31 /tmp/backup-logs/01-03-2011-19-31-01.tar.gz
#######DONE#######.
This will not require any user-interaction! We should keep those lines so that we can use the script without editing it. It is not a good idea to give a static directory name for the archives. But that is possible because of the "echo" statement:

echo "/tmp/backup-logs" | /root/bin/backItUp.sh /tmp/logs/*.log


Now, your archives will be stored in "/tmp/backup-logs" or "/path/to/ChandraBackups" instead of in the current directory (possibly your home directory).


I can still give you another version to remove the interactivity, but that is not required at all.


See, how archives are being placed in:

Code:
bash-2.05b# ls -ltr /tmp/backup-logs
total 8
-rw-r--r--  1 root root 154 Mar  1 19:31 01-03-2011-19-31-01.tar.gz
-rw-r--r--  1 root root 171 Mar  1 19:40 01-03-2011-19-40-01.tar.gz
bash-2.05b#

It is just what you need, yaar!
 
Old 03-01-2011, 02:04 PM   #28
chandkur
LQ Newbie
 
Registered: Feb 2011
Posts: 16

Original Poster
Rep: Reputation: 0
hello dost,

bahut bahut dhanyavaad, aapki script chaka chak chal raha hai..

Still doing some more tests..

Thank you very much for all your help.
 
Old 03-01-2011, 02:06 PM   #29
devUnix
Member
 
Registered: Oct 2010
Location: Bengaluru, India
Distribution: RHEL 5.1 on My PC, & SunOS / Sun Solaris, RHEL, SuSe, Debian, FreeBSD and other Linux flavors @ Work
Posts: 584

Rep: Reputation: 59
Chandra, since we are using file globbing which is being passed to the script (command-line argument as it is called) we are in effect passing several arguments. So, to access the second argument (the target directory/file name) we need to use $2 inside the script but it won't give the expected output because of file globbing *.log being passed to the script. $2's value will be overwritten with the second file's name.

Well, it does not mean we are limited to do things. I will see it and let you know when it is done. But, you can continue using the script as it is and use the format of the command in crontab I have shown above.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
Log file and command output archiving software harry edwards Linux - Server 2 11-01-2008 06:28 PM
archiving files in c++ mkrems Programming 2 04-03-2008 09:18 PM
archiving rotated log files tjainsworth Linux - Security 4 07-12-2006 01:00 PM
Webserver Log Rotation, Stats and Archiving. graq Linux - Enterprise 1 02-24-2006 06:57 PM
Archiving Email Files ? dolphans1 Mandriva 6 06-15-2005 10:12 AM

LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie

All times are GMT -5. The time now is 11:13 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration