Share your knowledge at the LQ Wiki.
Go Back > Forums > Linux Forums > Linux - Software
User Name
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.


  Search this Thread
Old 12-07-2006, 01:32 PM   #1
LQ Newbie
Registered: Nov 2003
Location: Sioux Falls SD USA
Distribution: CentOS 7
Posts: 27

Rep: Reputation: 15
md5sum different on gzipped file, but same when gunzipped

I have a large file, approximately 8.5 GBs in size. Each time I gzip it the file and take a md5sum the checksum is different. If I gunzip the file and then take a md5sum of the gunzipped file it matches the original file's md5sum. So each gzipped file is valid yet has a different md5sum checksum. I.e. somehow gzipping the exact same file creates different, yet equally valid, gzip files. How can this be?

-I'm using the System Rescue CD 0.2.19 livecd (available here
-The version of gzip on the System Rescue CD is 1.3.3.
-I also got the same results with gzip 1.3.5 on Windows.

The commands I run look like this:
gzip -c bigfile.txt >bigfile1.txt.gz
gzip -c bigfile.txt >bigfile2.txt.gz
gunzip -c bigfile1.txt.gz >bigfile1.txt
gunzip -c bigfile2.txt.gz >bigfile2.txt

In this example the md5sums of bigfile.txt bigfile1.txt and bigfile2.txt should and do match. However, the md5sums of bigfile1.txt.gz and bigfile2.txt.gz do not match!? Why not? The resulting md5sums are:

414d9c62127f3fcbd239bfedc412ab62 bigfile.txt
414d9c62127f3fcbd239bfedc412ab62 bigfile1.txt
414d9c62127f3fcbd239bfedc412ab62 bigfile2.txt
2ca22159a5dd3ddde674e5a4847f3873 bigfile1.txt.gz
460df7d60e6afc59a6f306bcee546d36 bigfile2.txt.gz

As you can see from the md5sums the gunzipped image files are identical even though their gzipped counterparts are different. Why is this happening? When I perform test gzips on small files I can't reproduce this behavior. When working with my large gzipped files, however, this same thing happens every time. Can someone shed some light on this for me?

Old 12-07-2006, 02:04 PM   #2
Senior Member
Registered: Oct 2003
Location: UK
Distribution: Kubuntu 12.10 (using awesome wm though)
Posts: 3,530

Rep: Reputation: 65
Maybe the gzipped file contains a timestamp?
Old 12-07-2006, 09:41 PM   #3
LQ Newbie
Registered: Nov 2003
Location: Sioux Falls SD USA
Distribution: CentOS 7
Posts: 27

Original Poster
Rep: Reputation: 15
Originally Posted by matthewg42
Maybe the gzipped file contains a timestamp?
That's it! According to
"since the header of a gzip'd file contains a timestamp, it is necessary to use the utilities zcmp or zdiff to determine if two gzip'd files uncompress to the same data stream without getting false negatives."

Thanks for pointing me in the right direction.



Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off

Similar Threads
Thread Thread Starter Forum Replies Last Post
mount gzipped hd image dohpaz Linux - Desktop 2 11-25-2006 12:58 PM
md5sum file satimis Fedora 2 02-24-2006 10:46 AM
gzipped log backups: clean up phats_O Debian 4 05-14-2005 12:39 AM
md5sum how do you check a file with it ? Joe47 Linux - Newbie 6 11-30-2003 08:02 PM
are my modules supposed to be gunzipped shanenin Linux - Software 1 10-17-2003 03:33 AM > Forums > Linux Forums > Linux - Software

All times are GMT -5. The time now is 08:56 PM.

Main Menu
Write for LQ is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration