LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-12-2004, 01:55 PM   #1
Johnny Mac
LQ Newbie
 
Registered: Nov 2004
Location: USA
Posts: 7

Rep: Reputation: 0
Issue tarring data


I have written a short script and created a daily cron to run it. The script runs well except for the 3rd line. Here is the script...
-------------------------
#!/bin/sh
tar -uf /backups/mysql/mysql.`date +%Y-%m-%d`.tar /var/lib/mysql
tar -uf /backups/mysql/mysql.appended.tar /backups/mysql/*.tar
gzip -c /backups/mysql/mysql.appended.tar > /backups/mysql/mysql.appended.tar.gz
cp -f /backups/mysql/mysql.appended.tar.gz /mnt/floppy/
--------------------------

As you can see, it takes all the data in my MySQL databases and makes a tar file using the date in the file name. Then it is supposed to take all the dated files and append them to a different tar file, only adding the dates files that don't already exist in this "mysql.appended.tar" file. It then takes the appeneded file, compresses it, then copies it to the floppy, overwriting the previous copy that existed on the floppy from the day before.

The problem I am having is that when creating the appended file from the dated files, instead of adding only the newest file to the archive, it is adding them all to the file and needlessly increasing the file. Am I using the incorrect switch in tar or should I be adding an additional switch? The man page is leading me to believe that my switches should be correct but I am obviously doing something incorrectly. Any thoughts or ideas would be greatly appreciated.
 
Old 11-12-2004, 02:24 PM   #2
LasseW
Member
 
Registered: Oct 2004
Distribution: Fedora 7, OpenSuse 10.2
Posts: 108

Rep: Reputation: 15
I think there's a slight flaw in your logic here: the first tar command will create a new file. All files in the mysql directory will be new to the dated file, so the u flag isn't selective at all, it's the same as using the c flag. I'm also wondering what the second tar command does: note that the target file mysql.appended.tar will match the source pattern *.tar.
 
Old 11-12-2004, 03:33 PM   #3
bigrigdriver
LQ Addict
 
Registered: Jul 2002
Location: East Centra Illinois, USA
Distribution: Debian Jessie 8.4
Posts: 5,873

Rep: Reputation: 348Reputation: 348Reputation: 348Reputation: 348
As far as the third line is concerned, if you want to compress the tarball, it's only necessary to enter the command gzip /backups/mysql/mysql.appended.tar. The result will be /backups/mysql/mysql.appended.tar.gz.
Or, you could add the -z option to the tar command to compress the tarball.
 
Old 11-12-2004, 03:41 PM   #4
Johnny Mac
LQ Newbie
 
Registered: Nov 2004
Location: USA
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally posted by LasseW
I think there's a slight flaw in your logic here: the first tar command will create a new file. All files in the mysql directory will be new to the dated file, so the u flag isn't selective at all, it's the same as using the c flag. I'm also wondering what the second tar command does
I wish to create a daily backup of the mysql datases:
#tar -uf /backups/mysql/mysql.`date +%Y-%m-%d`.tar /var/lib/mysql

The first day it runs it gives me:
mysql.2004-11-08.tar

Then the dated archive is tarred into a master archive named "mysql.appended.tar":
#tar -uf /backups/mysql/mysql.appended.tar /backups/mysql/*.tar
Resulting in "mysql.appended.tar.gz"

The next day the script runs again and creates "mysql.2004-11-09.tar".
Now I have two dated tar files in my backup directory:

mysql.2004-11-08.tar
mysql.2004-11-09.tar

When the second step of the script runs, it takes the existing tar files (*.tar) then and appends them to the master tar file (mysql.appended.tar). Because of the "u" switch, it should ignore the file from the 8th since it already exists in the master archive and add only the tar from the 9th.

The same thing happens on the 10th, creating "mysql.2004-11-10.tar". when the files are appended to the master archive the ones from the 8th and 9th should be ignored and only the one from the 10th should be added. And so on...


What I have instead is a master tar file that doubles in size each time the script runs. For instance, on the second day I found that the script was appending all the tar files to the master archive instead of only the newest one that was not already in the archive. When I extract the existing archive I found two copies of each dated archive file. I want to know why the "u" switch is not working as I thought it should.

[root@xxxxxxx floppy]# tar zxvf mysql.appended.tar.gz
backups/mysql/mysql.2004-11-08.tar
backups/mysql/mysql.2004-11-09.tar
backups/mysql/mysql.2004-11-08.tar
backups/mysql/mysql.2004-11-09.tar


Since my databases are small enough to fit a weeks worth of zipped backups onto a floppy this provides me a small level of security should my hdd fail. In case a floppy goes bad I also want to leave a copy of the dated archives on the hdd as well in case I need to restore the databases. I suppose I can simply move the dated archive file, once added to the master archive, to a different directory so it will not be added again to the master archive the next day but I don't understand why it is duplicating the dated files in the first place.

Quote:
Originally posted by LasseW
note that the target file mysql.appended.tar will match the source pattern *.tar.
I thought of that but if it did, why does it not appear in the extracted file list when I extract the master archive? Instead of...

[root@xxxxxxx floppy]# tar zxvf mysql.appended.tar.gz
backups/mysql/mysql.2004-11-08.tar
backups/mysql/mysql.2004-11-09.tar
backups/mysql/mysql.2004-11-08.tar
backups/mysql/mysql.2004-11-09.tar

Why do I not see...

[root@xxxxxxx floppy]# tar zxvf mysql.appended.tar.gz
backups/mysql/mysql.2004-11-08.tar
backups/mysql/mysql.2004-11-09.tar
backups/mysql/mysql.appended.tar
backups/mysql/mysql.2004-11-08.tar
backups/mysql/mysql.2004-11-09.tar
backups/mysql/mysql.appended.tar

Just curious as to why it is doing what it is doing. TIA!
 
Old 11-12-2004, 03:53 PM   #5
Johnny Mac
LQ Newbie
 
Registered: Nov 2004
Location: USA
Posts: 7

Original Poster
Rep: Reputation: 0
Quote:
Originally posted by bigrigdriver
As far as the third line is concerned, if you want to compress the tarball, it's only necessary to enter the command gzip /backups/mysql/mysql.appended.tar. The result will be /backups/mysql/mysql.appended.tar.gz.
Or, you could add the -z option to the tar command to compress the tarball.
I use the "-c" because I want to keep the original file as well as the zip. If I just gzip it will simply convert the original file, no? Or am I even more confused than I thought?
 
Old 11-12-2004, 03:58 PM   #6
Johnny Mac
LQ Newbie
 
Registered: Nov 2004
Location: USA
Posts: 7

Original Poster
Rep: Reputation: 0
I modified the script so that only the dated tar files are appended to the master archive

----------------------------------------------
#!/bin/sh
tar -uf /backups/mysql/mysql.`date +%Y-%m-%d`.tar /var/lib/mysql
tar -uf /backups/mysql/mysql.appended.tar /backups/mysql/mysql.200*.tar
gzip -c /backups/mysql/mysql.appended.tar > /backups/mysql/mysql.appended.tar.gz
cp -f /backups/mysql/mysql.appended.tar.gz /mnt/floppy/
-----------------------------------------------

That should resolve the issue of copying the master tar file into itself each day. For all I know this alone may have resolved my whole issue. What do you think? I suppose tomorrow I will find out after the script runs.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
tarring a dd stream john8675309 Linux - Software 1 04-22-2005 04:04 PM
WPA-PSK TKIP Ndiswrapper - No data issue sti2envy Linux - Wireless Networking 0 03-10-2005 12:53 AM
set password when bzipping/tarring/gzipping alice95089 Linux - Software 14 01-03-2005 04:09 PM
Issue copying data files from CD fryak Fedora 3 05-28-2004 12:49 PM
un-tarring multiple files phil1076 Linux - General 19 12-18-2003 03:03 PM


All times are GMT -5. The time now is 10:02 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration