Trying to tar 100GB worth of stuff. Halfway through tar always fails. Need help asap
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Introduction to Linux - A Hands on Guide
This guide was created as an overview of the Linux Operating System, geared toward new users as an exploration tour and getting started guide, with exercises at the end of each chapter.
For more advanced trainees it can be a desktop reference, and a collection of the base knowledge needed to proceed with system and network administration. This book contains many real life examples derived from the author's experience as a Linux system and network administrator, trainer and consultant. They hope these examples will help you to get a better understanding of the Linux system and that you feel encouraged to try out things on your own.
Click Here to receive this Complete Guide absolutely free.
I'm not sure this would cause the symptoms you are seeing, but by any chance is /home/stuff within the tree that is being archived? tar does try to reject attempts to include the archive within itself, but there are situations where it cannot detect that. It is best to put the archive elsewhere, or exclude it explicitly on the command line.
I am trying to tar 100gb worth of stuff, heres how my command looks. Mind you it is centos 6.4:
tar czvf /home/stuff . --numeric-owner
It always fails about halfway through for some reason. Its not an error it will just randomly freeze for a second on ssh and stop. it stops when the tar file size is around 6GB's
There are a couple of problems here; I believe the argument for --numeric-owner needs to be just after the czvf options. Your destination tarfile is "stuff" in the /home tree. You are archiving all files from the directory level at which you have run this command. If the /home tree is included in this, then your archive file is circularly included in your archive and that's the main reason why it's hanging.
If your intention is to archive your /home/stuff tree, then the command should look something like:
tar czvf --numeric-owner /home/other-location/out-tar-file.tar.bz2 /home/stuff
However no one yet knows what your true archive intentions were, so you should state that information clearly if you're still stuck trying to get this working.
Mind your syntax! That command will create a file named "--numeric-owner", since that string will be taken as the argument for the "f" option.
FWIW, the original location of the "--numeric-owner" option was fine.
Well corrected. FWIW I typically fall prey to stuff like that because I've never used the --numeric-owner argument; therefore I'd not have it in there; hence not used to playing around with fitting extra arguments like that in there. I do prefer to be more simple versus adding options like that, and just use the most common form of tar.
What in the world was that envisioned for anyways? Always use numbers for user/group names? So ... from system to system that does ... what? I.e. if a user or number doesn't exist on the other system, the result is the same, and typically when one de-archives it falls under the ownership of the userID extracting the archive, doesn't it? Worse, if the user doesn't exist on another system; however that user number is for a different user ... then? I never even thought to care, but you are correct to point out my syntax; I should've just omitted that argument totally.
I liked this.
"edited by muneer123; 05-23-14 at 10:36 AM. Reason: whoopsie daisy"
Ok, just for my reference and archival.
The ‘--numeric-owner’ option allows (ANSI) archives to be written without user/group name information or such information to be ignored when extracting. It effectively disables the generation and/or use of user/group name information. This option forces extraction using the numeric ids from the archive, ignoring the names.
This is useful in certain circumstances, when restoring a backup from an emergency floppy with different passwd/group files for example. It is otherwise impossible to extract files with the right ownerships if the password file in use during the extraction does not match the one belonging to the file system(s) being extracted. This occurs, for example, if you are restoring your files after a major crash and had booted from an emergency floppy with no password file or put your disk into another machine to do the restore.
The numeric ids are always saved into tar archives. The identifying names are added at create time when provided by the system, unless ‘--format=oldgnu’ is used. Numeric ids could be used when moving archives between a collection of machines using a centralized management for attribution of numeric ids to users and groups. This is often made through using the NIS capabilities.
When making a tar file for distribution to other sites, it is sometimes cleaner to use a single owner for all files in the distribution, and nicer to specify the write permission bits of the files as stored in the archive independently of their actual value on the file system. The way to prepare a clean distribution is usually to have some Makefile rule creating a directory, copying all needed files in that directory, then setting ownership and permissions as wanted (there are a lot of possible schemes), and only then making a tar archive out of this directory, before cleaning everything out. Of course, we could add a lot of options to GNU tar for fine tuning permissions and ownership. This is not the good way, I think. GNU tar is already crowded with options and moreover, the approach just explained gives you a great deal of control already. "