LinuxQuestions.org
Latest LQ Deal: Complete CCNA, CCNP & Red Hat Certification Training Bundle
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Newbie
User Name
Password
Linux - Newbie This Linux forum is for members that are new to Linux.
Just starting out and have a question? If it is not in the man pages or the how-to's this is the place!

Notices


Reply
  Search this Thread
Old 10-13-2009, 08:41 AM   #1
jhsu
Member
 
Registered: Mar 2007
Posts: 35

Rep: Reputation: 15
Why does the MD5SUM of a file change?


I'm trying to move a file from my desktop computer to my laptop computer (both with Puppy Linux) by putting it on a USB drive (FAT32 format). When I first copy the file from my desktop to the USB drive, the MD5SUM remains the same. I then unmount the USB drive from the desktop and mount it on the laptop. However, I now find that the MD5SUM of the file on the USB drive is now different from the original MD5SUM. And when I unmount the USB drive from the laptop and remount it to the desktop, the MD5SUM doesn't change back.

What exactly causes the MD5SUM of a file to change?
 
Old 10-13-2009, 09:28 AM   #2
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670
What kind of file is it. How long is it? How did you copy it?

If the file is over 2 GB, it was truncated. Fat32 isn't a good medium to transfer files if ownership and permissions need to be preserved. These permissions won't effect the md5sum however. If it was a sparse file, the copy could have had zeroes filling in the gaps. If the fat32 filesystem is mounted with the "conv=auto" or "conv=text" option, you could have had newlines converted to cl-lf's.

The most likely answer is that the file was corrupted.
 
Old 10-13-2009, 02:46 PM   #3
jhsu
Member
 
Registered: Mar 2007
Posts: 35

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jschiwal View Post
What kind of file is it. How long is it? How did you copy it?

If the file is over 2 GB, it was truncated. Fat32 isn't a good medium to transfer files if ownership and permissions need to be preserved. These permissions won't effect the md5sum however. If it was a sparse file, the copy could have had zeroes filling in the gaps. If the fat32 filesystem is mounted with the "conv=auto" or "conv=text" option, you could have had newlines converted to cl-lf's.

The most likely answer is that the file was corrupted.
The file is an ISO file around 650 MB in size (Debian ISO). I used the GUI copy command in Puppy Linux.

What format should I be using in the USB card?
 
Old 10-13-2009, 05:11 PM   #4
jefro
Moderator
 
Registered: Mar 2008
Posts: 15,388

Rep: Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199Reputation: 2199
There are only two reasons a file can change. One is there is some damage and the other could be how the system stored the file. In this case it should not have changed.

Might try to zip or otherwise store the file.

See the cp man page

"--sparse=WHEN
A `sparse file' contains `holes' - sequences of zero bytes that do not occupy any physical disk blocks; the `read' system call reads these as zeroes. This can both save considerable disk space and increase speed, since many binary files contain lots of consecutive zero bytes. By default, cp detects holes in input source files via a crude heuristic and makes the corresponding output file sparse as well. "
 
Old 10-13-2009, 07:15 PM   #5
jhsu
Member
 
Registered: Mar 2007
Posts: 35

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by jefro View Post
There are only two reasons a file can change. One is there is some damage and the other could be how the system stored the file. In this case it should not have changed.

Might try to zip or otherwise store the file.

See the cp man page

"--sparse=WHEN
A `sparse file' contains `holes' - sequences of zero bytes that do not occupy any physical disk blocks; the `read' system call reads these as zeroes. This can both save considerable disk space and increase speed, since many binary files contain lots of consecutive zero bytes. By default, cp detects holes in input source files via a crude heuristic and makes the corresponding output file sparse as well. "
So how do I preserve the MD5SUM value of a file as I transfer it from one computer to another through a USB drive?
 
Old 10-14-2009, 02:55 AM   #6
jschiwal
LQ Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670Reputation: 670
How did you calculate the md5sum? Did both use the `md5sum' command? Run the command on the source file and the copied file using the `md5sum' command. Look at "ls -l <filename>" on both. Did one get truncated?

For an ISO file, the permissions aren't important and you won't run into a problem with fat32's. It isn't a sparse file so that isn't the problem.

I don't think that using the 'cp' command or fat32 will result in padding the file.
If you calculated the md5sum in a CD burner program (like K3b), It might be possible that the actual file has some padding at the end, beyond what is part of the ISO filesystem, and that the burner program only includes the bytes that are part of the filesystem.

I doubt that two versions of gnu's md5sum would produce different results unless the files are different. They should comply with RFC 1321.

Use the "cmp" which does a byte by byte comparison. Compare the original with the pendrive copy. Compare the pendrive copy with the copy on the second computer.

Last edited by jschiwal; 10-14-2009 at 02:59 AM.
 
Old 10-14-2009, 03:26 AM   #7
Forrest Coredump
Member
 
Registered: Oct 2009
Location: Southwestern United States
Distribution: Redhat Enterprise Linux 4-5 (Current RHCE), Fedora Core 11 (FC11), Arch Linux, BT3 (Current GCIH)
Posts: 42

Rep: Reputation: 16
When you mount the fat32 partition on your laptop, make sure that the mount options on both systems match, i.e, issue the "mount" command and note the arguments. Most important here, is the length of the filename itself could be subject to change based on the fat32 mount option, "shortname=mixed".
 
Old 10-14-2009, 07:33 PM   #8
jhsu
Member
 
Registered: Mar 2007
Posts: 35

Original Poster
Rep: Reputation: 15
Quote:
Originally Posted by Forrest Coredump View Post
When you mount the fat32 partition on your laptop, make sure that the mount options on both systems match, i.e, issue the "mount" command and note the arguments. Most important here, is the length of the filename itself could be subject to change based on the fat32 mount option, "shortname=mixed".
Thanks, Forrest Coredump. The length of the filename was the issue. The MD5SUM value was preserved in the transfer when I shortened the filename.
 
  


Reply

Tags
md5sum


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
file utility md5sum slackamp Slackware 11 06-04-2007 09:36 PM
md5sum different on gzipped file, but same when gunzipped jim-j Linux - Software 2 12-07-2006 10:41 PM
md5sum file satimis Fedora 2 02-24-2006 11:46 AM
cksum & md5sum fingerprints change after backup/restore to st cougar694u Linux - Software 3 06-22-2004 10:38 AM
md5sum how do you check a file with it ? Joe47 Linux - Newbie 6 11-30-2003 09:02 PM


All times are GMT -5. The time now is 12:59 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration