LinuxQuestions.org
Register a domain and help support LQ
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 03-25-2011, 02:50 AM   #1
alaios
Senior Member
 
Registered: Jan 2003
Location: Aachen
Distribution: Opensuse 11.2 (nice and steady)
Posts: 2,185

Rep: Reputation: 45
Tar in Depth


Dear all ,
I would like to ask for your help concerning creating archives with tar.

Yesterday I used this command
Code:
tar -zcvf /media/disk/ap-hostname`date '+%d-%B-%Y'`.tar.gz /etc /root /home
to create one tar.gz file that contains my /home, /etc, /root directory.

a) The process ended with a 88GB file size (which is ok) but with the following message
Code:
tar: Exiting with failure status due to previous errors
. I have searched a little but I could not find what went wrong.

b) What are the limitations of tar and gz for backups. Of course I fully understand that they can not be used for differential backups (if it is called like that)

c) Let's say that my backup will be a file of 100GB and I want to see the contents of the .tar.gz. In kde there is a program called ark. Can ark handle so big files? Does it use my hard disk (eg. /tmp) to uncompress the file so to show me its contents? It might be the case that might be the compressed file is much bigger than the left space on the hard disk?

d) How can I do an integrity check when my tar.gz file is created?


I would like to thank you in advance for your help.

Best Regards
Alex

Last edited by alaios; 03-25-2011 at 03:01 AM.
 
Old 03-25-2011, 04:17 AM   #2
corp769
LQ Guru
 
Registered: Apr 2005
Posts: 5,817

Rep: Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002
Before I go anywhere with this, can you post the output of df -h?
 
Old 03-25-2011, 04:24 AM   #3
kakaka
Member
 
Registered: Sep 2003
Posts: 382

Rep: Reputation: 87
The v option requests verbose descriptions of what tar is doing. So you should be able to capture that output into a sort of log file. That way, if anything goes wrong, it's usually fairly easy to see what happened.

I generally use:

tar v... > tar.log 2>&1

from a Bourne-like shell ( for example bash ). I'm using ellipses ... to represent whatever other options might be needed. That captures both the standard output which lists each file that is included in the tar file, as well as the standard error output which contains the error messages.

As to limitations, there are a variety of limitations, and in a sense they change depending on what options you choose. So it's a bit of a broad topic to address generally.

I'm not entirely clear from what perspective you are asking the questions about disk space. But I can say this, with some versions of tar, gzip is run dynamically to create the tar file. You can verify this for your version in at least three ways. One, the command:

Code:
ps --forest gaxl
run from the command prompt, while tar is running, should show you that the gzip command is also running, as a child process of the tar command.

Two, the command:

Code:
file tarfile.tar.gz
where tarfile.tar.gz should be replaced with the actual name of your tar file, should produce output showing, not that it is a tar file, but instead, that it is a compressed file, and that command is also to be run while the tar command is still running.

At one point in the past, some forms of compression used with tar were almost no different than running the tar command, and then running the compression command separately, except that the tar command ran the compression command for you, if you supplied the correct option to tar. In fact, with some forms of compression, that may still be the case. But at least in the Linux distro I use, gzip works right along with tar.

Three: you can monitor the space available on /tmp with the command

df -k /tmp

while the tar command is still running. In my case I see no substantial use of space on /tmp even while I'm creating a huge tar file with gzip compression.
 
Old 03-25-2011, 07:14 AM   #4
alaios
Senior Member
 
Registered: Jan 2003
Location: Aachen
Distribution: Opensuse 11.2 (nice and steady)
Posts: 2,185

Original Poster
Rep: Reputation: 45
These are the errors my 2>> captured

tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: /home/ap/.gvfs: Cannot stat: Permission denied
tar: /home/ap: file changed as we read it


I do not think are so crucial. Are they?

I will reply to the rest later on again
 
Old 03-25-2011, 07:18 AM   #5
corp769
LQ Guru
 
Registered: Apr 2005
Posts: 5,817

Rep: Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002Reputation: 1002
Seeing the permission denied is making me wonder... What user are you running it as?
 
1 members found this post helpful.
Old 03-25-2011, 07:25 AM   #6
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Hi,
Quote:
Originally Posted by alaios View Post
These are the errors my 2>> captured

tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: Removing leading `/' from member names
tar: /home/ap/.gvfs: Cannot stat: Permission denied
tar: /home/ap: file changed as we read it

I do not think are so crucial. Are they?
The first three can be ignored. This is tar telling you that the leading / is removed from /etc, /home and /root. This to make sure you can restore to any directory and not completely overwrite /etc, /home and/or /root when untarring.

The permission denied message: This depends on the importance of the .gvfs file. If you do want to back it up you need to investigate this. I suspect that it can be ignored (you might want to exclude this when tarring). BTW: Are you running the tar command as root user?

The file changed as we read it can be ignored in most cases. Tar tells you that the content changed while making the tar file. Take log files for example, these files are open and tend to grow, tar will tell you when this happens.

Hope this clears things up a bit.
 
1 members found this post helpful.
Old 03-29-2011, 04:52 AM   #7
alaios
Senior Member
 
Registered: Jan 2003
Location: Aachen
Distribution: Opensuse 11.2 (nice and steady)
Posts: 2,185

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by corp769 View Post
Seeing the permission denied is making me wonder... What user are you running it as?
root
 
Old 03-29-2011, 04:53 AM   #8
alaios
Senior Member
 
Registered: Jan 2003
Location: Aachen
Distribution: Opensuse 11.2 (nice and steady)
Posts: 2,185

Original Poster
Rep: Reputation: 45
Quote:
Originally Posted by druuna View Post
Hi,


The permission denied message: This depends on the importance of the .gvfs file. If you do want to back it up you need to investigate this. I suspect that it can be ignored (you might want to exclude this when tarring). BTW: Are you running the tar command as root user?
Thanks a lot for the answer. Yes I am running this as root! I do not even know what this .gvfs file might be. I think I will ignore it
 
Old 03-29-2011, 05:27 AM   #9
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by alaios View Post
Thanks a lot for the answer.
You're welcome

BTW: If this is solved can you put op the [SOLVED] tag (Fisrt post -> Thread Tools).
 
Old 03-29-2011, 07:03 AM   #10
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
tar: /home/ap/.gvfs: Cannot stat: Permission denied

http://en.wikipedia.org/wiki/GVFS

It is normal to not be able to read the ~/.gvfs directory. It shouldn't be backed up. You should exclude it from backup.

If you use KDE, and follow symbolic links, exclude the symbolic links in the hidden .kde4 directory that point to directories inside /tmp/.

There are some hidden directories that are simply caches, that you could exclude. E.G. ~/.cache and ~/.thumbnails.
There can sometimes be thousands of very small files there that would be a waste of time backing up.

You might want to create a file containing the patterns to exclude and use the --exlude-from option.
 
Old 03-29-2011, 07:27 AM   #11
druuna
LQ Veteran
 
Registered: Sep 2003
Posts: 10,532
Blog Entries: 7

Rep: Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387Reputation: 2387
Quote:
Originally Posted by profi81 View Post
i would need help
Do not hi-jack a thread that is not yours and is not related to your problem. Please start your own thread. Reported.


EDIT: Moderator moved profi81's post to its own thread

Last edited by druuna; 04-22-2011 at 11:24 AM.
 
Old 04-04-2011, 02:02 AM   #12
alaios
Senior Member
 
Registered: Jan 2003
Location: Aachen
Distribution: Opensuse 11.2 (nice and steady)
Posts: 2,185

Original Poster
Rep: Reputation: 45
Dear all
I have my fist backup session run successfully.
Today after turning on my pc the kde couldnot launch. I found that all my space was occupied. Inside my /tmp I had a subfolder called *backup* of size of 80+Gb! .

Is it possible that this was created by tar or gz while they were writing these data to my extenal hard disk?
My backupscript was called backup.sh
Regards
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
how can i decompress this tar.tar file? hmmm sounds new.. tar.tar.. help ;) kublador Linux - Software 14 10-25-2016 03:48 AM
"Invalid tar magic" error msg. when I try to tar ldmud *.tar file in DSL pixxi451 Linux - Newbie 4 07-04-2010 09:32 AM
BackUp & Restore with TAR (.tar / .tar.gz / .tar.bz2 / tar.Z) asgarcymed Linux - General 5 12-31-2006 03:53 AM
tar | ssh (tar > .tar) syntax issues EarlMosier Linux - Software 6 12-21-2006 01:28 AM


All times are GMT -5. The time now is 10:30 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration