LinuxQuestions.org
Review your favorite Linux distribution.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices


Reply
  Search this Thread
Old 03-19-2008, 02:51 AM   #1
sridhar_dct3
Member
 
Registered: May 2007
Location: india,chennai
Distribution: debian,ubuntu linux
Posts: 30

Rep: Reputation: 15
Lightbulb gzip: stdin: invalid compressed data--format violated ftp ascii transfer


Dear friends,

I have very serious problem in my hand. If anyone already solved this issue, please help me.

Problem definition:
I transferred a tar+gzip file from a windows machine to a linux machine using ascii mode(ftp).

Unfortunately I am removed the original file from windows machine.

When tried 'tar -zxvf file.tgz' in linux it says the following error:
T/
T/xaa
tar: Skipping to next header

gzip: stdin: invalid compressed data--format violated
tar: Child returned status 1
tar: Error exit delayed from previous errors

I tried to use the following commands to recover the file:
gzrt-0.5
fixgz
gunzip < file.tgz | tar -xvf -

nothing helped me.

Please if anyone has solution, reply me.

Last edited by sridhar_dct3; 03-19-2008 at 03:09 AM.
 
Old 03-19-2008, 08:37 AM   #2
Guttorm
Senior Member
 
Registered: Dec 2003
Location: Trondheim, Norway
Distribution: Debian and Ubuntu
Posts: 1,453

Rep: Reputation: 447Reputation: 447Reputation: 447Reputation: 447Reputation: 447
Hi

When transfering the file with ASCII mode FTP, it is converted like if you used dos2unix on the file. So you could try the command "unix2dos file.tgz", to convert it back. I think it should work, but not 100% sure, so take a backup of the file in case it gets even more corrupted.

Edit:
If you dont have the commands unix2dos and dos2unix, install the tofrodos package.

Last edited by Guttorm; 03-19-2008 at 08:40 AM.
 
Old 03-19-2008, 10:31 AM   #3
lbukys
LQ Newbie
 
Registered: Mar 2008
Posts: 8

Rep: Reputation: 0
Lightbulb Recovering from FTP ASCII transfer of compressed files

The FTP ASCII mode transfer destroys data by transforming dissimilar original bytes to identical output values -- so there is no simple way to invert the destructive transform. Compressed files make every bit count, and mashing 3 bits/byte in 1/256 bytes of the file leads the decompressors astray rapidly.

This is probably the most common cause of corrupt compressed files. It's also the most difficult one to fix.

None of the commercial or free repair programs fixes this case. Mostly those repair programs leave the corrupted data in place and replace the checksums/CRCs with new values that don't cause complaints about the corruption. This can be OK if the problem is a file truncation or a bad block late in the file, and you want to extract what you can before the corrupted region.

So for 99% of the cases of FTP/ASCII corruption scattered throughout the binary file, the answer is "Transfer it again, in binary mode. If you don't have the uncorrupted original, get it from backups, or you're out of luck."

However, there is a technique that I have successfully applied to this problem. It can be done. It is computationally very expensive, and requires custom programming to make the recovery program know how to recognize corruption. Basically, the space of possible bit repairs can be searched, and the decompressed results evaluated on whether the repair was correct. It's intractable in the general case, but turns out to be feasible if anything about the structure of the uncompressed data is known. I was able to recover million-line web server logs with this technique. It can be used to fix short corrupted files without much effort, but for anything but tiny files, it requires custom coding of a heuristic function to guide the search for repairs -- which means that it's not a solution for you unless the data is extremely important to you, you have no backup, and you're willing to engage a consultant to customize the software -- then it's possible. (I described this successful file recovery in a little more detail at my web site.)
 
Old 03-20-2008, 12:36 AM   #4
sridhar_dct3
Member
 
Registered: May 2007
Location: india,chennai
Distribution: debian,ubuntu linux
Posts: 30

Original Poster
Rep: Reputation: 15
I have tried the following also :

dos2unix , unix2dos, transferring the corrupted file back in ascii mode to the windows machine.

The compressed file contains voice(wav) compressed by bzip2 program.

nothing helped me.

Last edited by sridhar_dct3; 03-20-2008 at 12:51 AM.
 
Old 03-20-2008, 01:22 AM   #5
chrism01
LQ Guru
 
Registered: Aug 2004
Location: Sydney
Distribution: Rocky 9.2
Posts: 18,359

Rep: Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751Reputation: 2751
If you used bzip2, you need the -j flag (bzip2 specific) to the tar cmd, not -z (gzip specific)
 
Old 03-20-2008, 02:14 AM   #6
sridhar_dct3
Member
 
Registered: May 2007
Location: india,chennai
Distribution: debian,ubuntu linux
Posts: 30

Original Poster
Rep: Reputation: 15
welcome, but the file is created like the following :

tar -zcvf file.tgz /tmp/a1.bz2 /tmp/a2.bz2 /tmp/a3.bz2
 
Old 08-08-2008, 02:15 AM   #7
sridhar_dct3
Member
 
Registered: May 2007
Location: india,chennai
Distribution: debian,ubuntu linux
Posts: 30

Original Poster
Rep: Reputation: 15
Finally I could not able to recover files from a corrupted file .

Anyway, on the other hand I have learnt how to recover files from ext3 file system using ext3grep tool.
 
Old 01-21-2011, 01:59 AM   #8
kudzai
LQ Newbie
 
Registered: Jan 2011
Posts: 1

Rep: Reputation: 0
solution to your problem

because ftp is so old, it does not automatically detect what type of file you are transferig. Therefore you need to specify whether it is a binary or ASCII file for it not to corrupt anything. This is how it worked in my situation, i first made an ftp connection from my linux machine to a wndows server. I then specified binary before i get the file. Then i transferred it across.
ie
linux1:~ # ftp 172.16.0.1
Connected to 172.16.0.1.
220 Microsoft FTP Service
Name (172.16.0.1:root): anonymous
331 Anonymous access allowed, send identity (e-mail name) as password.
Password:
230 User logged in.
Remote system type is Windows_NT.
ftp> ls
229 Entering Extended Passive Mode (|||49786|)
125 Data connection already open; Transfer starting.
12-01-09 03:55PM 81296677 psp-8.31.sles10.x86_64.en.tar.gz
01-20-11 04:13PM <DIR> v6.3.2b
01-17-11 02:05PM 910460830 v6.3.2b.zip
226 Transfer complete.
ftp> binary
200 Type set to I.
ftp> get psp-8.31.sles10.x86_64.en.tar.gz
local: psp-8.31.sles10.x86_64.en.tar.gz remote: psp-8.31.sles10.x86_64.en.tar.gz
229 Entering Extended Passive Mode (|||49799|)
125 Data connection already open; Transfer starting.
100% |****************************************************************| 79391 KB 106.37 MB/s 00:00 ETA
226 Transfer complete.
81296677 bytes received in 00:00 (106.33 MB/s)
ftp> bye
221 Goodbye.
 
Old 07-28-2012, 02:15 PM   #9
WoAnerges
Member
 
Registered: Apr 2012
Posts: 50

Rep: Reputation: Disabled
The answer

These errors are happening, because folder contents were changed after archiving process were started.
!!!stop all active processes before archiving folder contents!!!
 
Old 07-28-2012, 02:22 PM   #10
lbukys
LQ Newbie
 
Registered: Mar 2008
Posts: 8

Rep: Reputation: 0
WoAnerges is incorrect... Files changing during archiving will produce a corrupt archived member (some combination of old and new states), but the archive itself and its checksums will not be incorrect (the archiver knows what it wrote, and checksums that).
 
Old 07-28-2012, 02:25 PM   #11
WoAnerges
Member
 
Registered: Apr 2012
Posts: 50

Rep: Reputation: Disabled
Quote:
Originally Posted by lbukys View Post
WoAnerges is incorrect... Files changing during archiving will produce a corrupt archived member (some combination of old and new states), but the archive itself and its checksums will not be incorrect (the archiver knows what it wrote, and checksums that).
Checksumes will may or may not be correct, but archive will be broken forever and no one will be able to restore it.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
gzlib: stdin: invalid compressed data -crc error pashford Linux - General 4 04-29-2008 07:21 AM
FC 6: gunzip: stdin: invalid compressed data--format violated jnonaka Linux - Newbie 14 03-23-2007 02:24 PM
apt-get update error: "gzip: stdin: not in gzip format" haertig Debian 0 08-02-2006 05:21 PM
Invalid Compressed Format (err=1) OpenCircuit Slackware 1 12-11-2005 01:31 PM
Invalid compressed format..... ceh383 Slackware 3 05-24-2003 11:08 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:31 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration