LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 11-04-2006, 06:37 PM   #1
opto
Member
 
Registered: Jun 2006
Location: Pennsylvania
Distribution: Slackware64 , OS X , OpenBSD
Posts: 56

Rep: Reputation: 15
Can I back up hidden files using tar?


I want to back up my use home directory including "hidden" files (file that start with "."). I made a tar.gz file successfully but when I untarred and decompressed the files to make sure everything was there (ls -la command) I did not see any of my hidden files. Here are the commands that I used:

<code>user@terminal:~$tar -czvf user_backup.tar.gz*
</code>

I wanted to check to make sure that all of the files were there so I copied the user_backup.tar.gz file to /user/newapps/test and untarred and decompressed user_backup.tar.gz. When I checked the files using the <code>ls -la</code> none of the hidden files were there (like .bash_profile).

Does anybody know which commands to give to include when you want hidden files in a tarball? If the files are in the tarball, is there a special command to show reveal them?
 
Old 11-04-2006, 07:08 PM   #2
opto
Member
 
Registered: Jun 2006
Location: Pennsylvania
Distribution: Slackware64 , OS X , OpenBSD
Posts: 56

Original Poster
Rep: Reputation: 15
I finally figured out that I should not use the *.
 
Old 11-04-2006, 07:55 PM   #3
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
The reason is because when you enter the command:

Code:
tar czf backup.tar.gz *
...the shell expands the pattern * before passing it to tar. * matches all the non-hidden files and directories in the current working directory, so if you had the following files and sub-directories:
Code:
Documents
Desktop
myfile.txt
.local
.bashrc
...then * matches only Documents, Desktop and myfile.txt. so tar will get the arguments:
Code:
czf backup.tar.gz Desktop Documents myfile.txt
You can explicitly tell tar to archive the current working directory by replacing the * with a . (. means "this directory"), and since tar recurses all sub-directories in the required source directory, you'll get everything, including the hidden files.
 
Old 11-04-2006, 08:04 PM   #4
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
It would probaby be better if you used: tar -C /home -czvf user_backup.tar.gz <your-home-directory>
that would backup the containing directory.

There is a problem in how the argument * .* would be expanded by the shell. For example, forn inside a directory testdir:
~/testdir> ls -pa
./ ../ .bashrc .sample .skel/ tempfile tempfile2 testtar.tar .xmms/

~/testdir> echo . .*
. .. .bashrc .sample .skel .xmms

The ".." part would cause tar to tar the entire home partition and not just your home directory.

This should work as well: tar cvzf ${USER}_$(date +%F) ${HOME}

If you don't have a large capacity tape drive, and you use KDE, then installing dar and kdar may be a better way to go. It makes it easier to backup large sized directories to DVD. If your home directory has more than 4.4 GB of compressed data, where do you put such a large file. Especially if your $HOME directory is over half the full. You could use split but joining 4.4GB segments from DVD backup would be problematic. The T in tar is for tape, remember.

If you have a large external USB or Firewire drive, you could create the archive there. However, if it is a FAT32 drive you need to pipe the output of tar through split to break it up into segments less than 2GB in size. A size of 250MB may be more reasonable to work with. Then you can use par2 to create parity files to protect the backup from corruption.

You can use cat to join the segments and pipe the output to tar. This even works for other tar operations like listing the archive.

For a demonstration, I'll backup my Documents directory:
jschiwal@hpmedia:~> cd ~/; tar cvf - ~/Documents/pdfdocs/ | split -b50m -d - /media/ACOM/backups/docs-$(date +%F).
I have a lot of Documents!
ls -lh /media/ACOM/backups/
ls -lh /media/ACOM/backups/
total 174M
-rwxr-xr-x 1 jschiwal root 50M 2006-11-04 13:50 docs-2006-11-04.00
-rwxr-xr-x 1 jschiwal root 50M 2006-11-04 13:50 docs-2006-11-04.01
-rwxr-xr-x 1 jschiwal root 50M 2006-11-04 13:50 docs-2006-11-04.02
-rwxr-xr-x 1 jschiwal root 24M 2006-11-04 13:50 docs-2006-11-04.03

cat docs-2006-11-04.* | tar --list
home/jschiwal/Documents/pdfdocs/
home/jschiwal/Documents/pdfdocs/DLL.pdf
home/jschiwal/Documents/pdfdocs/HLA_callingHLA.pdf
home/jschiwal/Documents/pdfdocs/oscar-by-laws.pdf
home/jschiwal/Documents/pdfdocs/pam_appl.pdf
...

Using kdar, you can perform incremental backups. Also, files you deleted in the past will be deleted during a full restore process which includes incremental backups.

I use K3b to backup particular items to DVD. If you save the selection, it produces a .k3b file. This is actually a zip file containing a maindata.xml file. I'll extract the naindata.xml and use it to delete what I've backed up,to free up space:

sed -e '/<url>/!d' maindata.xml -e 's/^<url>\(.*\)<\/url>/\1/' | tr '\n' '\000' | xargs -0 rm

If a filename contains an ampersand symbol or '<' or '>', then you'll need to add more sed commands, because XML converts these characters: '&' -> "&amp;", '<' -> "&lt;", '>' -> "&gt;"
 
Old 11-04-2006, 08:07 PM   #5
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671Reputation: 671
Thank you matthewg42. I spaced out using simply '.'.

I worked so long on my response, but yours was more accurate and timely.
 
Old 11-04-2006, 09:47 PM   #6
opto
Member
 
Registered: Jun 2006
Location: Pennsylvania
Distribution: Slackware64 , OS X , OpenBSD
Posts: 56

Original Poster
Rep: Reputation: 15
Thanks for the responses. I archived my home directory from another directory while in root. Next time I'll use . so I don't to archive under root.
 
Old 11-05-2006, 09:35 AM   #7
soggycornflake
Member
 
Registered: May 2006
Location: England
Distribution: Slackware 10.2, Slamd64
Posts: 249

Rep: Reputation: 31
Note though that * expansion is shell dependant. In zsh, * includes dot-files and explicity does NOT match . or ..
(it's just common sense).
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
LXer: Hands on: Back up files using the tar command and rsync LXer Syndicated Linux News 0 07-04-2006 03:54 AM
Using (s)tar for back-up to hard disk creating very large files jlinkels Linux - Software 3 10-25-2005 08:55 PM
move files in tar.gz back to into original folders GUIPenguin Linux - General 1 05-09-2005 12:11 PM
tar and hidden files jjthomas Linux - General 4 04-07-2005 08:37 PM
Making back up files hidden on Mandrake 10 BluePyre Mandriva 5 04-29-2004 11:03 AM


All times are GMT -5. The time now is 08:34 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration