Highly mystifying and extremely frustrating experience: MD5SUM changes all the time

jozmak · 10-04-2014, 11:03 AM

Can anyone explain this to me and give me some suggestions how to go about preventing this from happening?

Here are a few examples.

I download an ISO image, I check the md5sum, everything is fine. I reboot the computer, I check the md5sum of the iso it changed. I have to start the downloading the ISO all over again.

Another example: I download the ISO image. I use dd method to copy it into usb stick. The copy fails. I check the md5sum again, the dd copy function changed the MD5SUM code. I have to start downloading the ISO again.

I sometimes use the unebootin to write image to usb. The copy fails, the md5sum changes once again.

These are not occasional occurrences but happening regularly.

The other day I had to download the ISO image five times to be able to make a bootable usb file. Tremendous waste of resources and of course, tremendous source of frustration.

How come a copy function can changes the ISO image files? What can be done to prevent this from happening.

Thanks a lot for all suggestions.

bulliver · 10-04-2014, 11:39 AM

What is the exact dd command you are using?

yancek · 10-04-2014, 11:53 AM

It might help if you indicated which iso files you are downloading so someone could check.

jozmak · 10-04-2014, 12:07 PM

Quote:

Originally Posted by bulliver

What is the exact dd command you are using?

This is the cp code I usually use:

Code:

sudo dd if=....iso of=/dev/sdc

jozmak · 10-04-2014, 12:09 PM

Quote:

Originally Posted by yancek

It might help if you indicated which iso files you are downloading so someone could check.

Last time I tried ubuntu-gnome beta because I wanted to help with testing.

bulliver · 10-04-2014, 12:27 PM

Well, I dunno. dd or even cp should not be changing the md5sum (that implies changing the contents of the iso). Rebooting certainly shouldn't change it. Something else must be at work there...

As for making a bootable USB stick, there are a few things you can try.

Some BIOSs will balk if the stick isn't formatted FAT32:

Code:

$ sudo mkdosfs -n 'USB-Drive-Name' -I /dev/sdc -F 32

Your BIOS might balk unless you add a MBR sector to your ISO:

Code:

$ isohybrid filename.iso

then:

Code:

$ sudo dd if=filename.iso of=/dev/sdc bs=4k

umount is supposed to take care of this itself, but perhaps even do a sync before unmounting the stick.

jozmak · 10-04-2014, 12:51 PM

Quote:

Originally Posted by bulliver

Well, I dunno. dd or even cp should not be changing the md5sum (that implies changing the contents of the iso). Rebooting certainly shouldn't change it. Something else must be at work there...

As for making a bootable USB stick, there are a few things you can try.

Some BIOSs will balk if the stick isn't formatted FAT32:

Code:

$ sudo mkdosfs -n 'USB-Drive-Name' -I /dev/sdc -F 32

Your BIOS might balk unless you add a MBR sector to your ISO:

Code:

$ isohybrid filename.iso

then:

Code:

$ sudo dd if=filename.iso of=/dev/sdc bs=4k

umount is supposed to take care of this itself, but perhaps even do a sync before unmounting the stick.

Thanks bulliver,

I try these tips.

One question: I have a gpt partitioned harddrive. Do I still have to add the MBR sector you mentioned?

jozmak · 10-04-2014, 12:58 PM

Currently, I'm downloading ubuntu-gnome-beta agian. I try these tips and report back what happened.

jozmak · 10-04-2014, 02:19 PM

It happened again. I downloaded ubuntu-gnome-beta. I checked the sha256sum and everything was fine.

I rebooted the computer and I checked the sha256sum again. It was different.

This is the right sha256sum before reboot.

Code:

mak@debian:~/Downloads$ sha256sum ubuntu-gnome-14.10-beta2-desktop-amd64\(1\).iso 
396f6ace73d817bc6c15b9518b52a49cf10bb008308c7d07af75c3aa7fc2bf53  ubuntu-gnome-14.10-beta2-desktop-amd64(1).iso
mak@debian:~/Downloads$

A this is after reboot:

Code:

mak@debian:~/Downloads$ sha256sum ubuntu-gnome-14.10-beta2-desktop-amd64\(1\).iso 
714dff477cb40a60fd73cb03ab1e08f4db48a65fde29b852ac0d8161c0e8f421  ubuntu-gnome-14.10-beta2-desktop-amd64(1).iso
mak@debian:~/Downloads$

My computer is in pretty good shape. It is about 8 years old.

CPU: AMD Dual Core
Motherboard: Asus
Memory: 6 Gig
Video Card: Nvidia GeForce 8400 GS
Harddrive: 1 SSD and 1 regular hd

jefro · 10-04-2014, 02:32 PM

Any file should not change for no reason. Virus or malware may cause this. Data problem like bad drive or cpu or memory might be an issue.

I'd download it and change permissions to only root. Hopefully you are not running under root ever. If so then you need to consider malware.

It could be possible that by some oddity you are testing the download in once case as a single file and in the next a mounted fileystem of an iso. Kind of doubt that.

The only other idea I'd have would be some cache issue.

Forget about bios issues. The only thing here is a file seeming to change. There is no other concern at this point.

jozmak · 10-04-2014, 02:53 PM

Quote:

Originally Posted by jefro

Any file should not change for no reason. Virus or malware may cause this. Data problem like bad drive or cpu or memory might be an issue.

I'd download it and change permissions to only root. Hopefully you are not running under root ever. If so then you need to consider malware.

It could be possible that by some oddity you are testing the download in once case as a single file and in the next a mounted fileystem of an iso. Kind of doubt that.

The only other idea I'd have would be some cache issue.

Forget about bios issues. The only thing here is a file seeming to change. There is no other concern at this point.

Thanks jefro,

No, I never ever use it as root.

What can I do now? How to go about troubleshooting it?

Could it be the harddrive. I downloaded on a regular harddrive.

Should I download again on the SSD to see?

Or what else can I do?

jozmak · 10-04-2014, 03:02 PM

Currently, I downloading the iso again on the SSD harddrive to see what happens.

I report back as soon as I finish.

jozmak · 10-04-2014, 03:42 PM

Sadly, I have to say that the same problem exists with the SSD drive as well. It seems that the problem is not the drive because it is unlikely the both mulfunctioning at the same time.

Otherwise, my computer is in perfect shape I've never experienced anything suspicious other then this.

On the regular harddrive I have 8 gpt partitions with 8 different distros installed and all work fine.

What can I do now?

How can I troubleshoot the machine?

howzer · 10-04-2014, 05:17 PM

I believe that jefro was on the right thank when he mentioned memory as a possible cause. From what I understand, when you do a checksum, the original file is copied into memory for the hash to be created. So, if your memory is bad, you can get a different result every time you do it.

I'd try running memtester or memtest86.

howzer · 10-04-2014, 05:25 PM

I couldn't put this in my first comment, but I learned this a while back from

http://www.cyberciti.biz/tips/howto-...ory-check.html

Basically, if a small portion of you ram is bad, you probably won't have problems with most tasks, but if you're working with larger files, you're more likely to hit that bad spot, and if the file is the same size as your RAM, you'd be guaranteed to hit it.