gzip: stdin: invalid compressed data--format violated ftp ascii transfer
Linux - SoftwareThis forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
When transfering the file with ASCII mode FTP, it is converted like if you used dos2unix on the file. So you could try the command "unix2dos file.tgz", to convert it back. I think it should work, but not 100% sure, so take a backup of the file in case it gets even more corrupted.
Edit:
If you dont have the commands unix2dos and dos2unix, install the tofrodos package.
Recovering from FTP ASCII transfer of compressed files
The FTP ASCII mode transfer destroys data by transforming dissimilar original bytes to identical output values -- so there is no simple way to invert the destructive transform. Compressed files make every bit count, and mashing 3 bits/byte in 1/256 bytes of the file leads the decompressors astray rapidly.
This is probably the most common cause of corrupt compressed files. It's also the most difficult one to fix.
None of the commercial or free repair programs fixes this case. Mostly those repair programs leave the corrupted data in place and replace the checksums/CRCs with new values that don't cause complaints about the corruption. This can be OK if the problem is a file truncation or a bad block late in the file, and you want to extract what you can before the corrupted region.
So for 99% of the cases of FTP/ASCII corruption scattered throughout the binary file, the answer is "Transfer it again, in binary mode. If you don't have the uncorrupted original, get it from backups, or you're out of luck."
However, there is a technique that I have successfully applied to this problem. It can be done. It is computationally very expensive, and requires custom programming to make the recovery program know how to recognize corruption. Basically, the space of possible bit repairs can be searched, and the decompressed results evaluated on whether the repair was correct. It's intractable in the general case, but turns out to be feasible if anything about the structure of the uncompressed data is known. I was able to recover million-line web server logs with this technique. It can be used to fix short corrupted files without much effort, but for anything but tiny files, it requires custom coding of a heuristic function to guide the search for repairs -- which means that it's not a solution for you unless the data is extremely important to you, you have no backup, and you're willing to engage a consultant to customize the software -- then it's possible. (I described this successful file recovery in a little more detail at my web site.)
because ftp is so old, it does not automatically detect what type of file you are transferig. Therefore you need to specify whether it is a binary or ASCII file for it not to corrupt anything. This is how it worked in my situation, i first made an ftp connection from my linux machine to a wndows server. I then specified binary before i get the file. Then i transferred it across.
ie
linux1:~ # ftp 172.16.0.1
Connected to 172.16.0.1.
220 Microsoft FTP Service
Name (172.16.0.1:root): anonymous
331 Anonymous access allowed, send identity (e-mail name) as password.
Password:
230 User logged in.
Remote system type is Windows_NT.
ftp> ls
229 Entering Extended Passive Mode (|||49786|)
125 Data connection already open; Transfer starting.
12-01-09 03:55PM 81296677 psp-8.31.sles10.x86_64.en.tar.gz
01-20-11 04:13PM <DIR> v6.3.2b
01-17-11 02:05PM 910460830 v6.3.2b.zip
226 Transfer complete.
ftp> binary
200 Type set to I.
ftp> get psp-8.31.sles10.x86_64.en.tar.gz
local: psp-8.31.sles10.x86_64.en.tar.gz remote: psp-8.31.sles10.x86_64.en.tar.gz
229 Entering Extended Passive Mode (|||49799|)
125 Data connection already open; Transfer starting.
100% |****************************************************************| 79391 KB 106.37 MB/s 00:00 ETA
226 Transfer complete.
81296677 bytes received in 00:00 (106.33 MB/s)
ftp> bye
221 Goodbye.
These errors are happening, because folder contents were changed after archiving process were started. !!!stop all active processes before archiving folder contents!!!
WoAnerges is incorrect... Files changing during archiving will produce a corrupt archived member (some combination of old and new states), but the archive itself and its checksums will not be incorrect (the archiver knows what it wrote, and checksums that).
WoAnerges is incorrect... Files changing during archiving will produce a corrupt archived member (some combination of old and new states), but the archive itself and its checksums will not be incorrect (the archiver knows what it wrote, and checksums that).
Checksumes will may or may not be correct, but archive will be broken forever and no one will be able to restore it.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.