LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Distributions > Fedora
User Name
Password
Fedora This forum is for the discussion of the Fedora Project.

Notices

Reply
 
Search this Thread
Old 11-27-2007, 05:47 AM   #1
fedusr
LQ Newbie
 
Registered: Nov 2007
Distribution: fedora 8
Posts: 5

Rep: Reputation: 0
tar, split, burn to dvd result to tar file corruption


I upgraded from fedora 7 to fedora 8. I started backing up the entire »/home« directory using Red Hat Magazine's Tips and Tricks: 'Splitting tar archives on the fly', because I use 4.7GB DVD+R medias and my »/home« size is over 5.5 GB.

Following the tip above:

»tar -czf /dev/stdout /home | split -d -b 4000 m - backupPART«

It produced two files: »backupPART00« and »backupPART01«.

I burned each on a DVD+R using Gnome's 'Burn folder mechanism'.

Then I upgraded to fedora 8 and tried to unsplit (»cat«) and to extract the tar archive file.

»cat backupPART* >> /dev/stdout | tar -xzf /dev/stdin«


But, I get an error:

»gzip: stdin: not in gzip format
tar: Child died with signal 13

...«

The (first) splited »backupPART01« files seems to be corrupt.

I don't know, what's going wrong. Tar creating, splitting, burning to DVD+R?

Do you know any 'recovering' tools for corrupt TAR files? May be, I can recover at least a few Gigabytes of my 'the one and only' backup.

Last edited by fedusr; 11-27-2007 at 05:50 AM.
 
Old 11-27-2007, 07:26 AM   #2
pixellany
LQ Veteran
 
Registered: Nov 2005
Location: Annapolis, MD
Distribution: Arch/XFCE
Posts: 17,802

Rep: Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728Reputation: 728
Quote:
»cat backupPART* >> /dev/stdout | tar -xzf /dev/stdin«
This makes no sense to me. First, why would you combine the files, append the result to /dev/stdout, and then attempt to pipe to tar? I don't even know what you would be piping....

Also, in "cat backupPART*", how do you know which one goes first?

How about:
cat backupPART00 backupPART01 | tar -xzf
 
Old 11-27-2007, 05:22 PM   #3
eberta
LQ Newbie
 
Registered: Nov 2007
Posts: 3

Rep: Reputation: 0
My guess looking at your command line is the /dev/stdout stuff messed you up. It is very unsafe to do:
tar -czf /dev/stdout /home | split -d -b 40000 - backupPART

What if tar outputed an error message or warning? It would also go to stdout and mess up the binary bits.

You can try:
tar -vztf backupPART00
to see if it tells you anything.
Other than that all you can do is manually hack backupPART00.
Open it up in emacs binary mode or simply vi it to look for hints.
Since gzip looks at I think just a first few bytes to determine if the file is a gzip file, that means the problem is at the top of the file.
You might find a warning like:
"tar: Removing leading `/' from member names"
Just remove "tar: Removing leading `/' from member names" and you might get lucky assuming nothing else got written to stdout that messed up tar or the compression.
 
Old 11-28-2007, 06:55 AM   #4
fedusr
LQ Newbie
 
Registered: Nov 2007
Distribution: fedora 8
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by pixellany View Post
This makes no sense to me. First, why would you combine the files, append the result to /dev/stdout, and then attempt to pipe to tar? I don't even know what you would be piping....
I took that procedure from Red Hat Magazine' Tips & Tricks website:
http://www.redhatmagazine.com/2007/1...es-on-the-fly/

I was positive about this. As an non-admin I'm not familiar with the behind the scene of piping. But I thought, 'cat' determines files and steps it's i/o units as file not as byte or line.


Quote:
Also, in "cat backupPART*", how do you know which one goes first?
Bash orders alphabetically (0-Z), I assumed.

Quote:
How about:
cat backupPART00 backupPART01 | tar -xzf

cat backupPART00 backupPART01 | tar -xzf -
stops also by
gzip: stdin: not in gzip format
tar: Child returned status 1
 
Old 11-28-2007, 07:07 AM   #5
fedusr
LQ Newbie
 
Registered: Nov 2007
Distribution: fedora 8
Posts: 5

Original Poster
Rep: Reputation: 0
Quote:
Originally Posted by eberta View Post
My guess looking at your command line is the /dev/stdout stuff messed you up. It is very unsafe to do:
tar -czf /dev/stdout /home | split -d -b 40000 - backupPART

What if tar outputed an error message or warning? It would also go to stdout and mess up the binary bits.
Indeed, that is what I assumed first of all, because while running tar creation of a 6 GB '/home'-directory I opened up a new (tabbed) terminal windows for listing (ls). I don't know if the output went to the same 'stdout-process' resp. split's stdin. On the other hand, the diagnostic and error messages stream stderr should not directed to split's stdin and the three streams (stdout,-in & -err) are attached to user's tty (should not interact with another open Terminal window).

Quote:
You can try:
tar -vztf backupPART00
to see if it tells you anything.
Unfortunatly, showing up the tarball file entries (tar -t) results to:

gzip: stdin: not in gzip format
tar: Child returned status 1



Quote:
Other than that all you can do is manually hack backupPART00.
Yes, I'm goin to learn the tar file format http://www.gnu.org/software/tar/manu...13.html#SEC163
But what exactly does TarFixer do?

I ran gzrt -The gzip Recovery Toolkit- against the tarball and it crashed at written file size 1.6 GB by memory access error/failure (http://www.urbanophile.com/arenn/hacking/gzrt/gzrt.html)

Thank you for your help. I'll post it, if i have something new.

Last edited by fedusr; 11-28-2007 at 07:10 AM. Reason: mistyped
 
Old 11-28-2007, 07:45 AM   #6
eberta
LQ Newbie
 
Registered: Nov 2007
Posts: 3

Rep: Reputation: 0
tarfixer is not going to help since the problem seems to be with the compression.
If you have xxd try:
xxd backupPART00 | head -10
and past the results into a reply. That will at least provide a hint as to why it thinks this is not a gzip file.
If you do not have the xxd program, try:
head -1 backupPART00
This will not be that helpful and there is a chance that the binary might mess up the terminal, but if you do not have xxd it is the only simple thing I can think of to get a hint as to what the problem is.
 
Old 11-28-2007, 12:21 PM   #7
fedusr
LQ Newbie
 
Registered: Nov 2007
Distribution: fedora 8
Posts: 5

Original Poster
Rep: Reputation: 0
The first lines of backupPART00 looks like a normal tarball (as said above):

$ xxd backupPART00 | head -10
0000000: 2f68 6f6d 652f 617a 2f0a 2f68 6f6d 652f /home/az/./home/
0000010: 617a 2f2e 6173 7065 6c6c 2e65 6e2e 7072 az/.aspell.en.pr
0000020: 6570 6c0a 2f68 6f6d 652f 617a 2f2e 6763 epl./home/az/.gc
0000030: 6f6e 662f 0a2f 686f 6d65 2f61 7a2f 2e67 onf/./home/az/.g
0000040: 636f 6e66 2f61 7070 732f 0a2f 686f 6d65 conf/apps/./home
0000050: 2f61 7a2f 2e67 636f 6e66 2f61 7070 732f /az/.gconf/apps/
0000060: 6c69 6665 7265 612f 0a2f 686f 6d65 2f61 liferea/./home/a
0000070: 7a2f 2e67 636f 6e66 2f61 7070 732f 6c69 z/.gconf/apps/li
0000080: 6665 7265 612f 2567 636f 6e66 2e78 6d6c ferea/%gconf.xml
0000090: 0a2f 686f 6d65 2f61 7a2f 2e67 636f 6e66 ./home/az/.gconf
$ head -1 backupPART00


resp.:

$ head -1 backupPART00
/home/az/



---

But I thougth, the z option compresses only the data part of a tarball member not the entire tarball incl. member headers. (However, I'm not able to puzzle out the tar file format and tar's z-option vs. gzip-ping a tarball)



Editing:
The size of backupPART00 file is larger than the maximum Emacs buffer size, so Emacs cannot open the 4 GB file.



.

Last edited by fedusr; 11-28-2007 at 12:31 PM. Reason: mistyped
 
Old 11-28-2007, 02:11 PM   #8
eberta
LQ Newbie
 
Registered: Nov 2007
Posts: 3

Rep: Reputation: 0
tar passes everything through gzip.
Here is what a good tar.gz file looks like:
xxd testtmp.tar.gz | head -10
0000000: 1f8b 0800 afc5 4d47 0003 ecdd 6f8f 5347 ......MG....o.SG
0000010: 96c7 f13c 9e57 719f ed8c 04a6 aace 39f5 ...<.Wq.......9.
0000020: a7a5 ac34 2189 166d fe6d 80cc 3c43 a6db ...4!..m.m..<C..
0000030: 80b5 6ebb b7ed 6688 b42f 7eeb d405 3601 ..n...f../~...6.
0000040: d264 94b4 bf1a a5ad 2821 81f4 c7b7 ca3e .d......(!.....>
0000050: bee7 feaa ae0f e717 f73e b9e1 4708 2196 .........>..G.!.
0000060: 52fc 9ffe 78f7 9fe3 d731 9424 2225 15fd R...x....1.$"%..
0000070: 24c4 2025 7e32 d94d 3f31 7f5c ed0f cbcb $. %~2.M?1.\....
0000080: 69fa e472 b73b 5cf7 e73e f6fb ffa2 8f43 i..r.;\..>.....C
0000090: 9fff c57e 77fa dfab c3fe a65e 08bf 76fe ...~w......^..v.

The "1f8b 0800" is the gzip "magic" number. If it is not present, then it is not a gzip file.

Also a non-gziped tar file looks like:
xxd testtmp.tar | head -10
0000000: 746d 702f 0000 0000 0000 0000 0000 0000 tmp/............
0000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
0000060: 0000 0000 3030 3031 3737 3700 3030 3030 ....0001777.0000
0000070: 3030 3000 3030 3030 3030 3000 3030 3030 000.0000000.0000
0000080: 3030 3030 3030 3000 3130 3732 3333 3337 0000000.10723337
0000090: 3237 3400 3031 3033 3731 0020 3500 0000 274.010371. 5...

So it does look like the output of an "ls" or something like that is corrupting the start of your backupPART00.
Because backupPART00 is so big it is hard to edit it.
Anyway to confirm that you even have a good gzip tar, I would do stuff like:
xxd backupPART00 | head -1000 | less
and look for 1f8b 0800. I guess you can also try:
xxd backupPART00 | grep "1f8b 0800"
to see if you have hope of editing the file. If you find "1f8b 0800" then you have hope of fixing the problem. You will probably have to find some binary editor (maybe bvi) that can handle such a huge file, or further break backupPART00 into parts, do the fix, and reassemble. Or maybe use dd skip=?? to skip everything up till the byte where "1f8b 0800" starts.

Last edited by eberta; 11-28-2007 at 09:13 PM.
 
Old 11-30-2007, 04:39 PM   #9
fedusr
LQ Newbie
 
Registered: Nov 2007
Distribution: fedora 8
Posts: 5

Original Poster
Rep: Reputation: 0
Smile

Thank you, eberta.

I'm able to replicate that messy tarball! When I run the tar creation & split command above, I added the --verbose (-v) option. All the verbose output - the copied files in full path names - did join split's <stdin>. But I do not understand the bahavior of 'tar'. It should be <stderr>, or not?

However, as you mentioned, all I have to do is to get rid off tar's copied files listing ('ls'-stylish) from my messed up tarball.

I haven't found a unix-like hex editor managing large files. The Java JHeditor crashed under icedtea & Sun's latest JRE. It would be a hard job editing a 6 GB file by ('search' &) hand. I going to try some scripts reading byte by byte and deleting strings starting with '/home/' and ending to <Linefeed>. Due to being a absolute Linux Newbie, I'll need some time to find out...

I'll report on sucess:-)

.

Last edited by fedusr; 11-30-2007 at 04:41 PM. Reason: something forgotten
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
how can i decompress this tar.tar file? hmmm sounds new.. tar.tar.. help ;) kublador Linux - Software 10 02-23-2008 05:40 AM
BackUp & Restore with TAR (.tar / .tar.gz / .tar.bz2 / tar.Z) asgarcymed Linux - General 5 12-31-2006 02:53 AM
gave wrong syntax for tar as tar -cvzf file file.tgz how to recover the file gautham Linux - General 4 04-13-2005 03:15 AM
large tar backup file to be burn stefane321 Linux - General 2 07-01-2004 02:33 PM


All times are GMT -5. The time now is 05:21 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration