[SOLVED] How can I md5 recursively and place a checksum file in each directory?
 

Hi All,

I would like to know how to md5 recursively and place the checksum file in each directory, consisting only off checksums from that directory. as if I were to run md5sum in every subdirectory one by one manually.
I tried google for 1 or 2 days but it is difficult to get google understand what exactly I want.

At the moment I use the following:

$ find . -type f 2>/dev/null -exec md5sum {} \; >checksum.txt
(I found this by googling )

This md5 recursively BUT place all the checksums and file names (including path from starting directory) in one file (in this case 'checksum.txt') in the directory where I execute the command from. Problem with this method is if I were to copy a subdirectory to some were I have to manually edit the checksum file and remove all entries that are not relevant to that subdirectory.

I also want to use sha1, 256 & 512 and any other checksum generating programs so a method were I can substitute md5sum with sha1sum and so on will be great.

My shell scrip experience is quite basic.

Thank you for your help,

Peter.



-Zenwalk6, K=2.6.28.7 (stock)

You could search for directories instead and run a script or use printf to create a script. For searching for files in a single directory, look at `-maxdepth 1'.

Thanks for the pointers....

I had a look at 'man find' and found a way to look for subdirectories.

I used the following command,
$ find . -type d 2>/dev/null >subdirectories.list

this is what is inside file 'subdirectories.list'

./dir1
./dir2
./dir3
./dir4/dir7
./dir4/dir7/dir8
./dir4/dir7/dir8/dir9


1st half of the problem is solved (getting a list of subdirectories)

------------------

2nd half of the problem (executing md5sum in each subdirectories or giving the subdirectory path to md5sum)

Method 1
manually editing the 'subdirectories.list' file and add 'md5sum' in front off every directory and '/*' at the end of every directory , this is ok for small number but can take a while when there are lot of subdirectories.

Is there a script that can do it (my script knowledge is at the level off 'echo hello world' level)?


Method 2 (executing 'md5sum' on the same line as 'find' command using option '-execdir')

I tried:
$ find . -type d 2>/dev/null -execdir md5sum >chsum.md5

I get a 0kb file called 'chsum.md5', on the directory where the command was executed but did not find 'chsum.md5' in any subdirectories. so I assume 'md5sum' is executed but the subdirectory paths are not pass on to 'md5sum' or md5sum was not executed at all.

then I tried:
$ find . -type d 2>/dev/null -execdir md5sum * >chsum.md5

Thinking md5sum did not do anything since it was not told to do so, same result as above
0kb 'chsum.md5' file in the directory where the command was executed.

If any one can help with the 2nd half of the problem, Method 1 or 2 would appreciate it very much.

Thank you for your time.

Peter.

I also read about options '-print' and '-printf' (under 'find') but could not figure out how to use it.

Short and sweet. Sorry for lack of any explanation, but it will handle spaces in the filenames and keep the sums files tidy. It depends on the output of file to be consistent. file can be rather inconsistent though. Should probably replace it with ls -l or stat or wc -c. meh.

:D

Thank you very much for your input 'mweed' :hattip: it works like a charm, one liner that will create md5sum file in every subdirectories below the starting dir.


:newbie:
if it is not asking too much, can you or anyone else tell me what dose, so I can understand :study: and it and apply in other placese.

Tankyou for your time,

Peter

As soon as I figure out how to thank 'mweed' ( I'll add 'Solved' to the title). Bytheway how do you thank some some in LQ ?? :scratch: :newbie:

I'll have a go.....

list all the directories under the current one and pipe the list to the while loop (which will continue to the end of the list)

Start the command secion

Echo the first directory name to the output

change to that directory, and push the current directory on to the stack (for use later)

Put nothing into the md5sums file. Could also have just been '> md5sums' ;)

list all the filenames in the current directory only, and run md5sum on them

pipe the output of md5sum through grep to do something I don't quite understand, which is to take the lines that don't match (-v) the extended (-E) expression, which is any line which starts with 32 alphnumeric characters, followed by space dot/md5sums and the end of line. Ah, got it now, this removes the md5sum of the actual 'md5sums' file. This could be made redundant by using the -not option of find:

find ./ -type f -not -name "md5sums" -maxdepth 1 -exec md5sum {} \;

delete the md5sums file if it's empty. Could also have been
'[ -s md5sums ] || rm -f md5sums'
Not sure how platform dependant that is though. I'm using bash.

go back to the previous directory on the stack

finish the command loop and go back for the next line


Just say thank you! Or click on the 'Thanks' button on the post you liked. Or click on the 'This post was helpful' link.... or all three!


That was fun! Thanks, mweed!

Thank you for the explanation bakdong,:hattip: thing make sense.

With the help off your explanation I made some changes to the command given by 'mweed'

I removed:

#did not see why create file 'md5sums' in every dir (maybe I am wrong)

and the corresponding removal of empty 'md5sums' file part

do you (or any one else) see any thing wrong with removing the above parts??

final command after removing bits

Looking forward to your comment,

Peter

The idea behind the '> md5sums' bit is to ensure that old 'md5sums' files that may exist in that dir are initialised. You could eliminate a complete line of grep by including that test in the second find, and I'm not sure about your '2nd' suffix on the md5sums filenames.

I see, I did not think off that.....:doh:

I ran the command that 'mweed' gave and I got md5sums in all the subdirectories with files in (as expected) , and after reading your (bakdong) explanation I was looking for ways to make the command shorter, and removed

and I added the suffix "2nd" to "md5sums" so I can distinguish between md5sum files created from 'mweed' command and my modified command, it serve no other funtion (I cut and past it from command line and forgot to remove the suffix).:redface:

I am just testing my new signature. :D

another way
 

find path/folder -type f -iname '*' -exec md5sum {} > checksums.txt \;