[SOLVED] md5sum sha1sum are different after copy file/s by cp command, krusader or nautilus
Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
md5sum sha1sum are different after copy file/s by cp command, krusader or nautilus
Hi,
I have a problem when I made copy of big file/s. I have been trying to check their md5 and sha1 (md5sum -b file.vdi , sha1sum -b file.vbi) but copied file differs against the original file. Files are copied between ext4 partitions.
I have tried to copy file by cp command, krusader or nautilus - it has always different md5 or sha1.
Could anybody please explain what am I doing wrong and what should I do that copied file should have same content ?
This behavior can be caused by hardware failure, I would recommend to check the health of the RAM (using Memtest86+) and the disk(s) (using the manufacturers diagnosis tool).
Something like that may also be caused by a not stable running overclock, does your system run within its specs?
Thanks for quick reaction. I have checked before many times memory by memetest86+ - it is ok according to this test.
I have many SATA III/II hard drives but in windows 7 this errors does not exist even if I made copy from different partition/s BUT between same disk/s even if partitions are ntfs. And this behaviour is only with files bigger that tens of gigabytes (vrtualbox vdi files).
When copy ISO files ( up to 4.7Gbytes) - it is correct with nautilus but it is not correct with krusader for same files, cp terminal command I did not check so many times md5 or sha1 so I cannot say for now.
I did no check all hard drives which I am actively using with manufacturer diagnostic tools yet because those disks are correct in windows. So , I am not assuming that only ext4 or btrf partitions have problems and ntfs partitions are correct on the same hard drives :-) . But I am going to try it some night/s.
According to those it seems to be only problem because of overclock.
I have maximus iv extreme R3, i7-2600k(3.4GHz) - OC:5.1GHz , memory OC:1600MHz , 9-9-8-24-1 - it should not be problem , graphic card is not OC.
Which is strange as well because win764bit works more or less properly - not with such problems (have another :-( ).
P.S. would it be possible that antivirus is doing this things (I have avg and clamav) ?
(Hope it is not some kind of malware )
I have a problem when I made copy of big file/s. I have been trying to check their md5 and sha1 (md5sum -b file.vdi , sha1sum -b file.vbi) but copied file differs against the original file. Files are copied between ext4 partitions.
I have tried to copy file by cp command, krusader or nautilus - it has always different md5 or sha1.
Could anybody please explain what am I doing wrong and what should I do that copied file should have same content ?
thank you.
try using dd and see if you still have the same problem.
PHP Code:
dd if=[inputfile] of=[outputfile]
hopefully you don't have to do this for small files but dd is a better choice for large files.
I have maximus iv extreme R3, i7-2600k(3.4GHz) - OC:5.1GHz , memory OC:1600MHz , 9-9-8-24-1 - it should not be problem , graphic card is not OC.
Which is strange as well because win764bit works more or less properly - not with such problems (have another :-( ).
You say that it should work and that Windows works more or less, so I assume that your overclock is not stable. Linux is not Windows, it may act different to such an unstable system.
Try it again without the overclock and see if that works. If it works your overclock is not stable and causing your issues.
You say that it should work and that Windows works more or less, so I assume that your overclock is not stable. Linux is not Windows, it may act different to such an unstable system.
Try it again without the overclock and see if that works. If it works your overclock is not stable and causing your issues.
I setup BIOS to manufacturer settings and it did not helped. It behaves like following:
I have checked sha1sum of tens of .ISO files and .vdi files - NOT COPIED. It looks like files bigger than 3.8(3.9)Gbytes have always different sha1 result measured by sha1sum.
You say that it should work and that Windows works more or less, so I assume that your overclock is not stable. Linux is not Windows, it may act different to such an unstable system.
Try it again without the overclock and see if that works. If it works your overclock is not stable and causing your issues.
I have really fun with this:
I have tried to use groovy (version 1.7.4) source code taken from: http://groovy.codehaus.org/Calculati...or+large+files
and
I tried openssl sha1 file.vdi (version OpenSSL 0.9.8o 01 Jun 2010)
and guess what ?
It behaves exactly the same way (sha1sum or sha1deep) as I described:
for files bigger than ~4Gbytes it had different sha1.
I did not copy files - just repeat the same command on same file.
I do not have to copy file,
It have happened if I run twice or more times same check-sum command on the same file.
File has to be bigger than approximately 3.9 (3.8-4.2) Gigabytes.
Tested with sha1sum -b , md5sum -b , sha1deep , md5deep , openssl sha1 , groovy sha1 algorithm.
I made test on four EXT4 partitions - they are located on 4 different hard drives (2 external and 2 internal HDDs/USB flash drive) on one computer (other 3 computers does not have this issue).
NTFS partitions DO NOT have these check-sum problems measured on the same hard drive where is located ext4 partition as well (3 HDDs). ... (This I assume eliminates possibility of hardware error/s of hard drive/s)
On BTRFS partition it is not possible to check because of I/O problems:
md5sum -b ubu1104.vdi
# md5sum: ubu1104.vdi: Input/output error
openssl sha1 ubu1104.vdi
# Read Error in ubu1104.vdi
# 5210:error:0200B005:system library:fread:Input/output error:bss_file.c:202:
# 5210:error:20082002:BIO routines:FILE_READ:system lib:bss_file.c:203:
sha1sum -b ubu1104.vdi
# sha1sum: ubu1104.vdi: Input/output error
Run sha1sum from live-cd ubuntu 11.04 64bit have same results.
No overclocking.
Tested on linux kernel from 3.0.1 to 3.0.4 versions compiled by myself with ubuntu patches, config changes only core2 and mtune=native march=native
tested on (3.0.0-8 - 3.0.0-12 kernels compiled by ubuntu team)
I have no idea what causing this issue? Could it be some kind of malware ?
Please any help , advice I really much appreciate.
Thank you,
Check your harddisks with the manufacturer's diagnosis tools, also have a look for harddisk errors in the logs.
If it is not the disks and not the RAM and also happens with different versions of Linux I would assume that it is a hardware error, reseat all cables, maybe replace them, if that doesn't help I would think it is some serious failure, maybe motherboard or CPU.
Check your harddisks with the manufacturer's diagnosis tools, also have a look for harddisk errors in the logs.
If it is not the disks and not the RAM and also happens with different versions of Linux I would assume that it is a hardware error, reseat all cables, maybe replace them, if that doesn't help I would think it is some serious failure, maybe motherboard or CPU.
Thanks for advice (I already changed data cables before 2 times, power source it should not be - corsair AX1200) I am going to test it anyway ,
but how you can explain that it DID NOT happened on ntfs partition ?
but happened only on ext4 partitions ?
it is the same command , same hard disk , same file but located on ntfs partition.
how it could be hardware failure ?
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.