LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices


Reply
  Search this Thread
Old 04-11-2014, 03:49 PM   #1
Slim Backwater
Member
 
Registered: Nov 2005
Distribution: Slackware 10.2 2.6.20
Posts: 68

Rep: Reputation: 15
Generating md5sum checksum files in each directory


I'm trying to recursively make md5sum checksum files in each directory, for the files in that directory. Basically, automate the process of cd'ing to each directory and running "md5sum * > @md5Sum.md5"

So far this is the command I have gotten to work:

find -type f ! -name "@md5Sum.md5" -execdir sh -c "echo \"{}\"; md5sum \"{}\" >>@md5Sum.md5" \;

However, I don't think that will handle unusual filenames, and I think it creates a new shell per file and I suspect that there's a better way (xargs ?) to do this.

I would check the md5sum with:
find -type f -name "@md5Sum.md5" -execdir md5sum -c {} \;

In the end I might just live with md5deep's "-rl" single output for the entire tree but I found this problem interesting. I started with "find -type d" but couldn't get anything I tried with that to work.

Thanks.
 
Old 04-14-2014, 02:33 PM   #2
Beryllos
Member
 
Registered: Apr 2013
Location: Massachusetts
Distribution: Debian
Posts: 529

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
Here is a short command that does most of what you asked for:
Code:
find ! -name @md5sum.md5 -type d -execdir /bin/bash -c 'cd "$0"; md5sum * > @md5sum.md5' {} \;
Edit: Um, I guess since we have -type d we don't need ! -name @md5sum.md5, so this should work the same...

Code:
find -type d -execdir /bin/bash -c 'cd "$0"; md5sum * > @md5sum.md5' {} \;
It is not quite perfect, for at least two reasons: (1) md5sum gives an error message when its argument includes a directory, and (2) if there are no regular files in the directory, md5sum runs anyway and creates an empty .md5 file, and (3) it doesn't exclude @md5sum.md5. But it does handle unusual filenames. I tested it with filenames containing spaces and newline characters, and it worked fine, so I think it should work for filenames containing any special characters.

Just for fun and for learning, I tried a true recursive method. As a wise person once said:
Quote:
"If you want to do something recursively, try recursion."
At first, I had no idea how to do it, but I googled around and tried some methods to see where they failed, and repeated that process until it worked the way I wanted.

It works better than the short command above, in that it does not pass directory names to md5sum, so we avoid those pesky error messages, and it does not run md5sum on empty directories, so there are no zero-length .md5 files, and it correctly excludes @md5sum.md5.

Yes, I know recursion can use up a lot of resources, in this case creating subshells within subshells, but at least it can be an interesting and fun programming exercise. Here's what I got:
Code:
#!/bin/bash

md5sum_recursive() {
    find -maxdepth 1 -mindepth 1 \! -name "@md5sum.md5" -type f -execdir /bin/bash -c 'md5sum "$0" >> "@md5sum.md5"' {} \; -o -type d -execdir /bin/bash -c 'cd "$0"; md5sum_recursive' {} \;
}

export -f md5sum_recursive

md5sum_recursive
In this script, we define a function that calls itself.

It does that when it finds a directory and needs to descend one level into it.

When it finds a regular file, it executes md5sum on it.

-maxdepth 1 restricts the find command to the current directory only. In effect, this takes all the recursion out of the find command, so we have to set up the recursion ourselves.

-mindepth 1 prevents the find command from finding the current directory, which would lead to an infinite recursion.

\! -name "@md5sum.md5" excludes the md5sum report files.

Problem: If @md5sum.md5 already exists, lines will be appended to it, probably leading to much duplication.

-execdir /bin/bash is necessary because we need a shell in order to define and use a shell function. It may be possible to implement the same logic by a recursive shell script rather than a recursive shell function, but I haven't tried that.

The '... "$0" ...' {} arrangement (the use of "$0" inside the single quotes, with the brace pair outside of the single quotes) is necessary in order to handle filenames with special characters.

Last edited by Beryllos; 04-14-2014 at 08:40 PM.
 
Old 04-15-2014, 01:57 AM   #3
myatthu
Member
 
Registered: Jan 2014
Distribution: CentOS, Fedora, Ubuntu
Posts: 108

Rep: Reputation: 18
You can try md5deep and switch with -r for recursive.
Hope that help.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
md5sum, checksum - quick questions mpc8250 Linux - Newbie 1 01-07-2013 07:24 AM
How to create md5sum for a directory? SentralOrigin Linux - Software 13 04-09-2012 12:14 PM
Difference in MD5 Checksum - RPM Vs MD5SUM sumeetn Linux - Security 2 07-07-2009 08:08 AM
generating a checksum in a terminal phantom_cyph Linux - General 9 02-18-2007 11:50 AM
md5sum on a directory with lots of files. solarmax Linux - Software 5 08-11-2005 05:22 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - General

All times are GMT -5. The time now is 10:00 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration