LinuxQuestions.org
Help answer threads with 0 replies.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices



Reply
 
Search this Thread
Old 12-07-2006, 02:32 PM   #1
jim-j
LQ Newbie
 
Registered: Nov 2003
Location: Norfolk NE
Distribution: Fedora Core 6
Posts: 24

Rep: Reputation: 15
md5sum different on gzipped file, but same when gunzipped


I have a large file, approximately 8.5 GBs in size. Each time I gzip it the file and take a md5sum the checksum is different. If I gunzip the file and then take a md5sum of the gunzipped file it matches the original file's md5sum. So each gzipped file is valid yet has a different md5sum checksum. I.e. somehow gzipping the exact same file creates different, yet equally valid, gzip files. How can this be?

-I'm using the System Rescue CD 0.2.19 livecd (available here http://www.sysresccd.org/)
-The version of gzip on the System Rescue CD is 1.3.3.
-I also got the same results with gzip 1.3.5 on Windows.

The commands I run look like this:
gzip -c bigfile.txt >bigfile1.txt.gz
gzip -c bigfile.txt >bigfile2.txt.gz
gunzip -c bigfile1.txt.gz >bigfile1.txt
gunzip -c bigfile2.txt.gz >bigfile2.txt

In this example the md5sums of bigfile.txt bigfile1.txt and bigfile2.txt should and do match. However, the md5sums of bigfile1.txt.gz and bigfile2.txt.gz do not match!? Why not? The resulting md5sums are:

414d9c62127f3fcbd239bfedc412ab62 bigfile.txt
414d9c62127f3fcbd239bfedc412ab62 bigfile1.txt
414d9c62127f3fcbd239bfedc412ab62 bigfile2.txt
2ca22159a5dd3ddde674e5a4847f3873 bigfile1.txt.gz
460df7d60e6afc59a6f306bcee546d36 bigfile2.txt.gz

As you can see from the md5sums the gunzipped image files are identical even though their gzipped counterparts are different. Why is this happening? When I perform test gzips on small files I can't reproduce this behavior. When working with my large gzipped files, however, this same thing happens every time. Can someone shed some light on this for me?

TIA,
Jim
 
Old 12-07-2006, 03:04 PM   #2
matthewg42
Senior Member
 
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 63
Maybe the gzipped file contains a timestamp?
 
Old 12-07-2006, 10:41 PM   #3
jim-j
LQ Newbie
 
Registered: Nov 2003
Location: Norfolk NE
Distribution: Fedora Core 6
Posts: 24

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by matthewg42
Maybe the gzipped file contains a timestamp?
That's it! According to http://en.wikipedia.org/wiki/Gzip
"since the header of a gzip'd file contains a timestamp, it is necessary to use the utilities zcmp or zdiff to determine if two gzip'd files uncompress to the same data stream without getting false negatives."

Thanks for pointing me in the right direction.
 
  


Reply

Tags
gzip


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
mount gzipped hd image dohpaz Linux - Desktop 2 11-25-2006 01:58 PM
md5sum file satimis Fedora 2 02-24-2006 11:46 AM
gzipped log backups: clean up phats_O Debian 4 05-14-2005 01:39 AM
md5sum how do you check a file with it ? Joe47 Linux - Newbie 6 11-30-2003 09:02 PM
are my modules supposed to be gunzipped shanenin Linux - Software 1 10-17-2003 04:33 AM


All times are GMT -5. The time now is 10:40 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration