I want to back up my use home directory including "hidden" files (file that start with "."). I made a tar.gz file successfully but when I untarred and decompressed the files to make sure everything was there (ls -la command) I did not see any of my hidden files. Here are the commands that I used:

<code>user@terminal:~$tar -czvf user_backup.tar.gz*
</code>

I wanted to check to make sure that all of the files were there so I copied the user_backup.tar.gz file to /user/newapps/test and untarred and decompressed user_backup.tar.gz. When I checked the files using the <code>ls -la</code> none of the hidden files were there (like .bash_profile).

Does anybody know which commands to give to include when you want hidden files in a tarball? If the files are in the tarball, is there a special command to show reveal them?

I finally figured out that I should not use the *.

The reason is because when you enter the command:

...the shell expands the pattern * before passing it to tar. * matches all the non-hidden files and directories in the current working directory, so if you had the following files and sub-directories:
...then * matches only Documents, Desktop and myfile.txt. so tar will get the arguments:
You can explicitly tell tar to archive the current working directory by replacing the * with a . (. means "this directory"), and since tar recurses all sub-directories in the required source directory, you'll get everything, including the hidden files.

It would probaby be better if you used: tar -C /home -czvf user_backup.tar.gz <your-home-directory>
that would backup the containing directory.

There is a problem in how the argument * .* would be expanded by the shell. For example, forn inside a directory testdir:
~/testdir> ls -pa
./ ../ .bashrc .sample .skel/ tempfile tempfile2 testtar.tar .xmms/

~/testdir> echo . .*
. .. .bashrc .sample .skel .xmms

The ".." part would cause tar to tar the entire home partition and not just your home directory.

This should work as well: tar cvzf ${USER}_$(date +%F) ${HOME}

If you don't have a large capacity tape drive, and you use KDE, then installing dar and kdar may be a better way to go. It makes it easier to backup large sized directories to DVD. If your home directory has more than 4.4 GB of compressed data, where do you put such a large file. Especially if your $HOME directory is over half the full. You could use split but joining 4.4GB segments from DVD backup would be problematic. The T in tar is for tape, remember.

If you have a large external USB or Firewire drive, you could create the archive there. However, if it is a FAT32 drive you need to pipe the output of tar through split to break it up into segments less than 2GB in size. A size of 250MB may be more reasonable to work with. Then you can use par2 to create parity files to protect the backup from corruption.

You can use cat to join the segments and pipe the output to tar. This even works for other tar operations like listing the archive.

For a demonstration, I'll backup my Documents directory:
jschiwal@hpmedia:~> cd ~/; tar cvf - ~/Documents/pdfdocs/ | split -b50m -d - /media/ACOM/backups/docs-$(date +%F).
I have a lot of Documents!
ls -lh /media/ACOM/backups/
ls -lh /media/ACOM/backups/
total 174M
-rwxr-xr-x 1 jschiwal root 50M 2006-11-04 13:50 docs-2006-11-04.00
-rwxr-xr-x 1 jschiwal root 50M 2006-11-04 13:50 docs-2006-11-04.01
-rwxr-xr-x 1 jschiwal root 50M 2006-11-04 13:50 docs-2006-11-04.02
-rwxr-xr-x 1 jschiwal root 24M 2006-11-04 13:50 docs-2006-11-04.03

cat docs-2006-11-04.* | tar --list
home/jschiwal/Documents/pdfdocs/
home/jschiwal/Documents/pdfdocs/DLL.pdf
home/jschiwal/Documents/pdfdocs/HLA_callingHLA.pdf
home/jschiwal/Documents/pdfdocs/oscar-by-laws.pdf
home/jschiwal/Documents/pdfdocs/pam_appl.pdf
...

Using kdar, you can perform incremental backups. Also, files you deleted in the past will be deleted during a full restore process which includes incremental backups.

I use K3b to backup particular items to DVD. If you save the selection, it produces a .k3b file. This is actually a zip file containing a maindata.xml file. I'll extract the naindata.xml and use it to delete what I've backed up,to free up space:

sed -e '/<url>/!d' maindata.xml -e 's/^<url>\(.*\)<\/url>/\1/' | tr '\n' '\000' | xargs -0 rm

If a filename contains an ampersand symbol or '<' or '>', then you'll need to add more sed commands, because XML converts these characters: '&' -> "&amp;", '<' -> "&lt;", '>' -> "&gt;"

Thank you matthewg42. I spaced out using simply '.'.

I worked so long on my response, but yours was more accurate and timely.

Thanks for the responses. I archived my home directory from another directory while in root. Next time I'll use . so I don't to archive under root.

Note though that * expansion is shell dependant. In zsh, * includes dot-files and explicity does NOT match . or ..
(it's just common sense).