[SOLVED] Is the result of MD5 calculation consistant?

laks · 05-02-2012, 09:19 AM

Hello All,

I am trying to generate MD5Hash of a large number of files(.7z files) from java. I am using MessageDigest to do the same. However, the problem I am facing is that the output MD5Hashes are not consistant;changing everytime it is running(for same files). This is being run on Linux suse 10. Could anyone tell me if MD5calculation is dependent on platform and can change with multiple iterations?

Regards,
Lakshmi

sundialsvcs · 05-02-2012, 09:21 AM

MD5 is a digital-signature algorithm. If the signatures are changing, the files are changing ... or, just as likely, there's a bug in your code.

laks · 05-02-2012, 11:00 PM

Files are changing but file size and contents remain same. This is what my program does .

1. Reads a set of .gz files.
2. Archives them to a set of 7z files.
3. Calculates MD5Hash of 7z files.

At the third step, I am getting different results for same set of .gz files.

I have executed my program for many other data sets, but it works for all of them and fails only for this particular set. I am almost stuck with no way out for further debugging.

NevemTeve · 05-03-2012, 02:49 AM

This is natural. Using the MD5 algorithm on files something.gz and something.7z (or .Z, .xz, .bz2 etc) will give different results, even if they are both the compressed versions of the same something file.

gnashley · 05-03-2012, 03:46 AM

I noticed recently that files compressed using xz (another lz/7z format) have an inconsistent size by a few bytes when repeatedly built. This may be the OP's problem.

sundialsvcs · 05-03-2012, 08:52 AM

File formats can also have some "unpredictable" bytes, and even one bit will completely change a signature.

laks · 05-03-2012, 10:30 PM

Hello,

Finally I found out the problem. Though the gz files remain the same, some of the gz files were having a different timestamp(Replaced or recreated by an external process) every time the process was ran. Hence MD5Hash was changing due to this change in timestamp of files.
Thanks to everyone for their valuable replies.

Marking this thread resolved.

Regards,
Lakshmi