Linux - NewbieThis Linux forum is for members that are new to Linux.
Just starting out and have a question?
If it is not in the man pages or the how-to's this is the place!
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
a crc32 value is 8 digits long... cksum gave me two values.. a 10 digit value and a 9 digit value.. niether one came close to the crc32 value of a known file i checked
upon cksum --help execution.. i get
or: cksum [OPTION]
Print CRC checksum and byte counts of each FILE.
--help display this help and exit
--version output version information and exit
.. if this a crc checking option.. why am i not getting an 8 digit number? i'm confused :/
educate this ignorant newbie
Sorry about that. I thought I remembered that the last byte of a crc was the parity or some other type of property. That's why I suggested it (10 hex digits = 5 bytes - 1 byte = 4 bytes = 32 bits)
I'm also a bit fuzzy about calculating CRC. I believe it's possible to calculate it in a number of different ways. For the files you have, did the website you downloaded them from provide information on how/what was used to calculate them? Or if you got it from another individual, you might be able to ask them?
The only checksum tools I'm aware of are md5sum, cksum, and plain old sum. I checked the man pages for md5sum, and it doesn't seem like there are options to "downshift" it to 32 bits. And I think sum reports a 16-bit CRC.
<edit>
When talking about CRC and saying things like crc16, crc32, and crc64, the number represents the size of each "block" of data used in calculating the crc. In other words, for crc16, the routine will split the file up into 16-bit chunks to work with. Similarly, it will be 32-bits for crc32, and so on.
</edt>
Last edited by Dark_Helmet; 06-30-2004 at 07:09 PM.
I also did some other checking, and noticed that the library zlib has a crc32 calculator in it. So I wrote a quickie program to use it. You can try it if you like, but i make no guarantees. If you read through the website above, it will explain how calculating a CRC can differ from one implementation to the next (specifically the polynomial chosen). Using the routines in zlib gives a little reassurance that the polynomial wasn't just plucked out of the air.
If you want to give my program a try, type this up (or copy-&-paste) it into a file. Save it as something like simple_crc.c. Actually, you can name it whatever you like, just make sure to use ".c" for the end of the filename.
Code:
#include <stdio.h>
#include <zlib.h>
#include <unistd.h>
int main( int argc, char *argv[] )
{
FILE *input_file;
char *file_buffer;
unsigned long file_size;
unsigned long bytes_read;
unsigned long crc;
unsigned long buffer_index;
if( argc < 2 )
{
printf( "I need a filename to calculate crc on\n" );
return 1;
}
input_file = fopen( argv[1], "r" );
if( input_file == NULL )
{
printf( "Could not open %s for reading.\n", argv[1] );
return 2;
}
fseek( input_file, 0, SEEK_END );
file_size = ftell( input_file );
fseek( input_file, 0, SEEK_SET );
if( file_size == 0 )
{
printf( "File size is calculated to be 0. Cannot calculate CRC\n" );
return 3;
}
file_buffer = (char *)malloc( file_size );
if( file_buffer == NULL )
{
printf( "Unable to reserve enough memory for file buffer.\n" );
return 4;
}
bytes_read = read( (int)input_file, file_buffer, file_size );
if( bytes_read < file_size )
{
printf( "Did not read entire contents of the file into buffer.\n" );
printf( " number of bytes read: %d\n", bytes_read );
free( file_buffer);
return 5;
}
crc = crc32( 0L, Z_NULL, 0 );
crc = crc32( crc, file_buffer, file_size );
printf("Calculated CRC: %0X\n", crc );
free( file_buffer );
return 0;
}
At a command line, issue this command: gcc -o simple_crc -lz simple_crc.c
If everything went smoothly, you have a new file simply names "simple_crc" in the current directory. To use it: ./simple_crc filename
If the program spits out a message starting with anything other than "Calculated CRC:", then there was a problem. Since this is a quickie program, all you can really do is try again or verify the filename you gave is correct. In other words, it's not very robust when it comes to handling unexpected conditions.
thanks for sticking with me this long and no, i haven't found a solution to my problem yet...
i've tried the program you gave me however i'm getting different results for all my files... eg:
_____________________________________________________________________________________
[apostasy@localhost hack--DUSK]$ dir
(B-A)Hack_Legend_of_the_Twilight_-_01_(E48F18B2).mkv
(B-A)Hack_Legend_of_the_Twilight_-_02_(5023D2C9).mkv
(B-A)Hack_Legend_of_the_Twilight_-_03_(0875FB91).mkv
(B-A)Hack_Legend_of_the_Twilight_-_04_(CE26F317).mkv
[apostasy@localhost hack--DUSK]$ simple_crc \(B-A\)Hack_Legend_of_the_Twilight_- _01_\(E48F18B2\).mkv
Calculated CRC: 785D3BFE
[apostasy@localhost hack--DUSK]$ simple_crc \(B-A\)Hack_Legend_of_the_Twilight_- _02_\(5023D2C9\).mkv
Calculated CRC: E71E2733
[apostasy@localhost hack--DUSK]$ simple_crc \(B-A\)Hack_Legend_of_the_Twilight_- _03_\(0875FB91\).mkv
Calculated CRC: 87AD85D5
[apostasy@localhost hack--DUSK]$ simple_crc \(B-A\)Hack_Legend_of_the_Twilight_- _04_\(CE26F317\).mkv
Calculated CRC: 41A82D34
[apostasy@localhost hack--DUSK]$
_____________________________________________________________________________________
i have many files like this (with crc32 in their name) and unless they are all corrupted, i am not getting what i'm looking for.
the reason i need to get matching crc values is that i have many files which i do not know the value of... and need to know for database purposes.
I will ask around to see how these people are getting these values (what tools they are using)
btw simple_crc is awesome.. if this all gets sorted out i will be using it often
yea i had the same problems with that code.. didn't seem to work, could be a couple reasons.. first of all we're CRCin files that are at least a couple hundred megs, no need to load the whole thing into memory :-/..
anyway i'm werking on a better version, that might actually work ? not sure
[apostasy@localhost Kiddy Grade]$ crc32 \(B-A\)Kiddy_Grade_-_01_\(892D29A1\).mkv
crc32.c by elektron
File Name: (B-A)Kiddy_Grade_-_01_(892D29A1).mkv
File Size: 145790583
Calculated CRC: E4C83678
i'm getting different results for all my files
<EDIT>after thinking about it some i decided to try hasing some .avi files... your program works like its supposed to... but once i start hashing 'container' formats like .ogm or .mkv; that's where i am running into problems.. for some reason the mentioned below app does not give me the same problem</EDIT>
A friend pointed me to http://calcchecksum.sourceforge.net and i'm using this app for now, yet i'm still searching for a command-line oriented tool. I've emailed the author of calcchecksum and he told me that there are plans to implement command-line functions into this, yet he is currently not actively working on the project.
Code:
[apostasy@localhost Puni Puni Poemy]$ crc32 Puni.Puni.Poemy.01.Poemi.Is.In.A.Bad.Mood.\[AXP.DVDRip\].\[Dual.Audio\].\[7A9F8C2A\].ogm
crc32.c by elektron
File Name: Puni.Puni.Poemy.01.Poemi.Is.In.A.Bad.Mood.[AXP.DVDRip].[Dual.Audio].[7A9F8C2A].ogm
File Size: 366681173
Calculated CRC: D9D18910
Code:
[apostasy@localhost DayDream]$ crc32 \[A-Keep\]_DayDream_OVA_01_\[F978C89B\].avi
crc32.c by elektron
File Name: [A-Keep]_DayDream_OVA_01_[F978C89B].avi
File Size: 241700864
Calculated CRC: F978C89B
wierd... not familiar with 'container' formats like .ogm or .mkv, but the source should werk for computing correct crc32 values for any file. maybe there's a more obscure bug. according to http://www.afterdawn.com/glossary/terms/container.cfm avi is also a 'container' format. /me looks into this
Tinkster:
You and I think alike I already checked md5sum for its available options, but couldn't find anything telling it to downshift and calculate a 32-bit CRC. It spits out a full 128-bit hash which which is a bit more than Apostasy needs
elektron:
Take that code and run with it
Just a pointer: I noticed you swapped some things around a bit; All those "return X" statements for unexpected error conditions were to give a non-zero exit result for the program (just to follow traditional *nix guidelines). In your computeCRC32 function, you return 0 on those same error conditions. The only problem I see is that value gets assigned to the crc value. In theory, 0x00000000 is a valid CRC 32 value. It might be rare (specifically a 1 in about 4 billion chance), but in such a case, you wouldn't be able to distinguish whether it's a legitimate CRC or an error.
Also, you have a slight memory leak... Here's the sequence:
1. The malloc() in generateCRC32() goes off without a hitch
2. Suppose the open() call fails (returns -1)
In that scenario, you return from the generateCRC32() without freeing the memory for file_buffer. If the user supplied more than one filename on the command line, the possibility exists that you could chew through memory quickly.
Apostasy:
Did you ever find out what program the people used to calculate their CRC value? I trust the guys that wrote the zlib library, and that means the key portion of elektron's program is solid. If the value it spits out does not match what you have, then[list=1][*]There is a different polynomial used between the original CRC calculation and the one in zlib[*]The files actually are corrupt[*]The CRC does not apply to the "container" file[/list=1]
For #3, I'm suggesting something along the lines of a single zipped file. Say you had a gargantuan text file. You could calculate the CRC on the text file, then compress it. The CRC of the compressed file would not match the CRC of the uncompressed file. This would be a very, very odd way of distributing a CRC value, but it is possible.
As a side note, from the perspective of CRC, it should not matter whether a file is a "container", text, movie, or whatever. A CRC calculator simply looks at a file as long string of bytes; it doesn't know and doesn't care what those bytes are used for. If there is a CRC calculator that changes the calculation based on the type of file, then that's a crappy and completely untrustworthy program.
<edit>
One other thing to consider. These container formats you mentioned, are they anything like the relationship between wav files and mp3s? What I'm getting at is this (along the lines of #3 again). Say you downloaded a wav file with the crc embedded in the filename ("Sir_Mix_A_Lot-Baby_Got_Back(12AB34CD).wav"). If you encode that wav file to mp3, usually, only the extension changes ("Sir_Mix_A_Lot-Baby_Got_Back(12AB34CD).mp3"). However, the CRC value for the mp3 will not be the same as the CRC for the wav file; meaning the filename is giving misleading information. So, if the container files you have went through some sort of similar process, that can explain why you get different CRC values.
</edit>
Last edited by Dark_Helmet; 07-09-2004 at 01:39 AM.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.