LinuxQuestions.org
Support LQ: Use code LQ3 and save $3 on Domain Registration
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices



Reply
 
Search this Thread
Old 10-01-2006, 09:39 PM   #1
saturndude
Member
 
Registered: Mar 2005
Location: across the river from Louisville KY
Distribution: Mandriva 2010.2 (64-bit)
Posts: 57

Rep: Reputation: 15
gunzip setup STINKS!


Hello,

I'm not exactly new to linux, but this is certainly not a success story or a question about certification, so I can't post it in "Linux - General".

I would like to make gunzip respect any suffix as valid without question. I think PKZIP 2.04G [MS-DOS, FAT] and PKZIP 2.50 CLI [95+, FAT32/NTFS] allowed any extension, I am used to that and want to continue it in linux.

I'm running Mandriva 2006 retail box. According to man pages, I have zip version 2.3.1. Unzip is 5.5.2 (although I saw "1.2.4" somewhere). I think gunzip actually calls the "engine" of gzip, but the "unzip" command might not use the "engine" of zip. Confusing.

AFAIK, zip's behavior can be altered by (1) an environment variable or (2) an alias. When I pipe output of the "set" command to a text file, I should see "ZIP" or "ZIPOPT" at the beginning of a line, right? All I see is:

_=unzip

I don't think this will change zip's behavior -- am I right?

As far as aliases changing a program's behavior, per-user aliases can be used here, though Mandriva's install did not implement any:

/home/users-name/.bashrc, or /root/.bashrc for admin/root.

Mandriva 2006 puts system-wide aliases in /etc/profile.d/alias.sh. In the default install, this line appears in that file:

alias gunzip= "gunzip -S """

According to man pages, this should do what I want. But when I type:

gunzip /path/to/zipfile-with-odd.ext

I get "gunzip: compressed data not read from a terminal. Use -f to force decompression. For help, type 'gunzip -h'"


I have tried changing some system-wide aliases. Each time I do so, I log out and back in again on one text console and use that one for testing. AFAIK, this is sufficient for the new aliases to take effect, but only on that one console (*nix doesn't need rebooting for nearly as many types of system changes as MS WIN). (after LILO and init finish, I have 10 text mode VCs, with NO users or VCs automatically logged in. All work done for this post was done in text mode except for actually posting).


With the factory alias above, if I use -f as was suggested (log out, then back in first to be sure), like this:

gunzip -f -S "" /path/to/file-with-exotic.ext

I see "gunzip: /path/to/file-with-exotic.ext: cannot decompress onto itself" echoed back to the screen.

Also, IIRC, patching won't change basic behavior (only squash rare, exotic bugs) because these utilities are mature. So patching up won't fix this (though I will patch).

Should I just change the system wide alias for gunzip to point to unzip and be done with it?

I'm so pissed off, I'd like to delete /bin/gunzip entirely, but I can't (I don't know what might depend on gunzip, like browsers displaying jpegs or movies, formats that are customarily stored as compressed).

Can gunzip be made to do what I want? How?

I'd really appreciate your help on this one.

Last edited by saturndude; 10-01-2006 at 09:41 PM.
 
Old 10-01-2006, 10:39 PM   #2
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 16

Rep: Reputation: 233Reputation: 233Reputation: 233
Gzip doesn't record the filename when compressing a file. The file it creates depends on the filename before the .gz extension. For example, gunzip void.tar.gz will create void.tar. Renaming void.tar.gz to new.tar.gz then uncompressing it will output new.tar.

To uncompress files with unknown extensions, use -c to output to stdout then redirect to a file. For example

# gunzip -S "" -c file.ext > file
 
Old 10-02-2006, 10:21 AM   #3
saturndude
Member
 
Registered: Mar 2005
Location: across the river from Louisville KY
Distribution: Mandriva 2010.2 (64-bit)
Posts: 57

Original Poster
Rep: Reputation: 15
My example, with results

Interesting. Instinct tells me (can't put it into words) that what you propose can't be put into an alias. I think it needs (a) typing in from the command line each time, or (b) using a variable, which means I'll need a script. Is this correct?

If it helps, I am using the hosts.zip file from www.hosts-file.net for this entire thread. It is simple and easy to work with.

> Gzip doesn't record the filename when compressing a file.

I am not sure how that relates. I am trying to unzip the hosts.oddext file, not compress anything. The odd extension on the zip file is not relevant. Whatever tool was originally used to compress, the full name and extension of each file in the archive (with the original timestamp of each) is known and stored in the zipfile, and is recreated upon unzipping.

I suspect gunzip was purposefully written to give an error whenever the "wrong" extension was encountered. (restricting extension goes against the easy-going DOS / Win95+ behavior -- was this considered "easier" or more sensible? By whom?) Trying to defeat the very purpose of it (with an alias) might seem silly, but that's what I'm trying to do. And I can't understand why the alias Mandriva installed does not work, when the man page says it should. Perhaps the quotes do not 'escape' it correctly? (I use bash).

When I try your suggestion, I type:

gunzip -S "" -c ./hosts.oddext > file

I get:

"-S No such file or directory
gunzip: ./hosts.oddext has more than one entry--rest ignored"

So instead, I type:

gunzip ./hosts.oddext -c -S "" ./hosts.oddext > ./hosts

I get:

gunzip: ./hosts.oddext: cannot decompress onto itself


If I try this:

gunzip -c ./hosts.oddext -S "" ./hosts.oddext > ./hosts

I get:

gunzip: ./hosts.oddext: cannot decompress onto itself
gunzip: ./hosts.oddext: cannot decompress onto itself

Yes, it echoes twice.

I thought I read in the man pages that one of these utilities cannot handle more than one member (when compressing). Does this carry over to decompressing? That would explain the first thing I tried in this post.

I have to go now, and check back in a day or two. I really do welcome any help anybody can give on this.

Last edited by saturndude; 10-02-2006 at 10:41 AM.
 
Old 10-02-2006, 04:57 PM   #4
konsolebox
Senior Member
 
Registered: Oct 2005
Distribution: Gentoo, Slackware, LFS
Posts: 2,245
Blog Entries: 16

Rep: Reputation: 233Reputation: 233Reputation: 233
These commands work fine with me. I'm not sure but perhaps you're uncompressing a file with different compression format. remember gzip (gz) is different from pkzip (zip).

# gunzip -c -S "" void.ext
# gunzip -S "" -c void.ext

If gunzip still doesn't work for you, perhaps upgrading to a new version or trying another command like zcat will do

# zcat ./hosts.oddext > ./hosts
 
Old 10-03-2006, 01:36 AM   #5
saturndude
Member
 
Registered: Mar 2005
Location: across the river from Louisville KY
Distribution: Mandriva 2010.2 (64-bit)
Posts: 57

Original Poster
Rep: Reputation: 15
Not making sense

> remember gzip (gz) is different from pkzip (zip).

Really? I'm getting some confusing stuff from the man pages.

As far as I can tell, zip 2.31 (which I have) is 100 percent, utterly and completely compatible with PKZIP 2.04g for MS-DOS (and probably PKZIP 2.50 as well). The man page for zip also says it is:

"analogous to a combination of the UNIX commands tar(1) and compress(1) and is compatible with PKZIP (Phil Katz' ZIP for MSDOS systems)."

The "compress" man page says compress uses "adaptive Lempel-Ziv coding algorithm".

The man page for gzip claims it uses the same algorithm as zip, but it also says:

"Compression is generally much better than that achieved by LZW (as used in compress)...."

Wait a minute.... I thought "compress" used LZ77, just like zip ("analogous to tar and compress"). So either compress uses LZ77 or LZW. The manpage for "compress" points out that Lempel-Ziv and Lempel-Ziv-Welch are different patents, so compress uses one or the other. One of these manpages is lying to me.

Can you enlighten me? I'm lost. All I can do is start cussing at this point.

Do I have to go to my local library and look up the two patent numbers? I could, but that wouldn't tell me which algorithm is used where, so that wouldn't help.



Now on to new business -- from the manpage for zip:

"Zip has one compression method (deflation) and can also store files without compression."

from the man page for gzip:

"Files created by zip can be uncompressed by gzip only if they have a single member compressed with the 'deflation' method."


I decided to compress a single file with zip (a *.pdf file). Zip said that deflation reduced the file size by 29%. The "ls" command quickly showed the deflated file had the .zip extension. When I used gunzip to inflate it (single member, deflation method) the operation failed. The alias Mandriva's install program put in does not work, and yet the man page says it will work.

When I type gunzip ./pdf-file.zip", the screen echoes "gunzip: compressed data not read from a terminal. Use -f to force decompression. For help, type: gunzip -h"

When I use -f to force, I get "gunzip: ./pdf-file.zip: unknown suffix -- ignored"


I always thought the "g" in gunzip stood for "gnu", or, "GNU's NOT UNIX". In the spirit of free software, I thought gunzip would be better than unzip. I'm rethinking that now. Gunzip is a loser.

Can anybody explain how to get gunzip to respect *_ANY_* suffix? What's going on here?
 
Old 10-03-2006, 10:55 AM   #6
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 719

Rep: Reputation: 72
Hi, saturndude.

You have the answers, but there is some glue missing. In *nix, there are separate utilities for packing a group of files together and for compression, namely tar for the former and gzip for the latter. (Other utilities for compression have existed, pack and compress, for example, but gzip seems to be the most often used now.)

In other worlds (where PKzip began life), both actions may be done by a single program, namely zip.

So gzip and zip really do different things and are not strictly comparable programs.

However, in the case of a single element in a zip archive, gunzip can handle it. You saw the error message from gunzip, which now may make more sense.

In *nix we would usually do a tar, then a gzip. If one were to gzip a group of files, one would end up with the same number of files, each separately compressed, e.g.

a b c -> gzip -> a.gz b.gz c.gz

so you'd more often see:

a b c -> tar -> archive.tar -> gzip -> archive.tar.gz

(Because this is done so often, there is an option in GNU tar to do the compression / de-compression from within tar, seemingly as a single step with gathering and separating.)

I was able to extract and inflate the files contained in the hosts.zip file with unzip, as installed on my Linux box.

If this differentiation of responsibilities doesn't make sense, I suggest you do a few of these operations as experiments, and I think you'll then understand the difference.

Best wishes ... cheers, makyo
 
Old 10-03-2006, 01:07 PM   #7
saturndude
Member
 
Registered: Mar 2005
Location: across the river from Louisville KY
Distribution: Mandriva 2010.2 (64-bit)
Posts: 57

Original Poster
Rep: Reputation: 15
Thank you. I'll read your post again in a day or two, it will make more sense then. Just wanted to post quickly.

I looked at the man page for tar, and I *_think_* it is the same as the old MS-DOS "concatenate". That is, it removes the space between the end of the file and the end of the cluster, cramming the files together in one big file.

Near the end of the Windows 3.1 era, when hard drives reached 500 megs or more, cluster sizes exploded. Tarring short files together would save a lot of slack space in this case.

And, depending on how your tape backup was logically oriented, there might be no provision for slack space. (I used Travan-4 briefly.)

I've never used tar, but I imagine that's what it does and how it is used. Have I described tar correctly?

> So gzip and zip really do different things...

Are you saying that gzip will not handle the slack space correctly when compressing or decompressing? If so, this could throw off the comparison with checksums, "magic numbers" or uncompressed file sizes stored inside archives made with PKZIP, correct?

Last edited by saturndude; 10-03-2006 at 01:08 PM.
 
Old 10-03-2006, 02:22 PM   #8
makyo
Member
 
Registered: Aug 2006
Location: Saint Paul, MN, USA
Distribution: {Free,Open}BSD, CentOS, Debian, Fedora, Solaris, SuSE
Posts: 719

Rep: Reputation: 72
Hi.

I don't know very much about DOS / Windows, so I cannot compare them in any detail.
Quote:
Originally Posted by saturndude
Thank you. I'll read your post again in a day or two, it will make more sense then. Just wanted to post quickly.

I looked at the man page for tar, and I *_think_* it is the same as the old MS-DOS "concatenate". That is, it removes the space between the end of the file and the end of the cluster, cramming the files together in one big file.
This is more like *nix command cat.

Quote:
Near the end of the Windows 3.1 era, when hard drives reached 500 megs or more, cluster sizes exploded. Tarring short files together would save a lot of slack space in this case.
Yes, the saving would be there, but most people use tar to manipulate a group of files as one. The allocation unit for mass storage in *nix is a block, so the most that one would waste would be a block less one byte (i.e. almost an entire block). Given the size of disks these days, no one I know of worries about that. I have a 4 GB pair of RAID disks on one of my servers, and, even with a lot of source, it's not even close to being used to capacity. It is claimed that some new filesystems are designed so that files can share a block: http://en.wikipedia.org/wiki/Block_%28data_storage%29 - however, I have not investigated those details.
Quote:
Have I described tar correctly?
Well, tar uses headers to describe the files in the archives that it creates. The length, owner - meta-data - are listed there, for example, http://en.wikipedia.org/wiki/Tar_%28...Format_details

Quote:
> So gzip and zip really do different things...

Are you saying that gzip will not handle the slack space correctly when compressing or decompressing? If so, this could throw off the comparison with checksums, "magic numbers" or uncompressed file sizes stored inside archives made with PKZIP, correct?
I don't know, but recall that gzip is only going to extract a maximum of one zip file:
Quote:
Files created by zip can be uncompressed by gzip only if they have a
single member compressed with the 'deflation' method. This feature is
only intended to help conversion of tar.zip files to the tar.gz format.
To extract a zip file with a single member, use a command like gunzip
<foo.zip or gunzip -S .zip foo.zip. To extract zip files with several
members, use unzip instead of gunzip.

-- from man gunzip
Best wishes ... cheers, makyo

( edit 1: unit of allocation )

Last edited by makyo; 10-04-2006 at 09:12 AM.
 
Old 01-18-2012, 10:44 PM   #9
gdawkins
LQ Newbie
 
Registered: Mar 2010
Posts: 1

Rep: Reputation: 0
Thumbs up EZ Solution

I found this thread since I was having the same problem and just wanted to state the ez solution that worked for me.

Filename: user_wordpress.01.18.2012.sql.ez (Gzipped export of a wordpress sql database)

Code:
gunzip -S ".ez" user_wordpress.01.18.2012.sql.ez
Resulting file: user_wordpress.01.18.2012.sql
An uncompressed SQL file.

Hope that helps someone else.
 
  


Reply

Tags
gzip, tar, zip


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Does anyone else think that FC2s Package manager in X stinks? SoupyC Fedora 3 10-15-2004 08:35 AM
Manually remounting cd-rom drive stinks... vi0lat0r Linux - General 2 04-07-2004 08:30 AM
burning cd's in linux stinks illtbagu Linux - Software 8 12-11-2003 02:50 PM
SBLive! 4 Speaker output stinks GreenerLinux Mandriva 4 11-19-2003 12:33 PM
http, red hat 9.0 stinks illtbagu Linux - Newbie 7 04-22-2003 11:27 AM


All times are GMT -5. The time now is 12:30 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration