LinuxQuestions.org
Share your knowledge at the LQ Wiki.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - General
User Name
Password
Linux - General This Linux forum is for general Linux questions and discussion.
If it is Linux Related and doesn't seem to fit in any other forum then this is the place.

Notices

Reply
 
Search this Thread
Old 11-10-2003, 03:13 AM   #1
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 13.1 / 12.3_64-KDE, Ubuntu 14.04, Fedora 20, Mint 17, Chakra
Posts: 3,650

Rep: Reputation: Disabled
I admit to lazyness: Can I "fork" a stream of data (dual use)?


This weekend I used Steve Litt's rawread-scheme to duplicate SuSE 8.2 .

I also followed his advice, and compared the md5sums of source and target CD-ROMs. That means reading five disks three times (once getting the data, then getting the original md5sum, then after burning the new CD). Well, to do that 15 times to a CD ...

So here is my question(s): When I copy the source-CD with something like
Code:
dd if=/dev/cdrom of=/my/temporary/iso bs=n cont=m
how can I use the output to the hard-disk of dd (like "of=/my/temporary/iso") to be used simultaneously by md5sum? I mean, why reading the source-CD twice, once for the temporary image and once for md5sum?

I was wondering, whether there is something like "||" for two parallel / diverging pipes or any possibility to influence stdout (1>) to do that?

Actually I'd imagine that md5sum could also work in-between if= and of= of the dd-statement, but that would bust dd ... so I need probably a "cloned" data-stream for md5sum but how do I get that?

P.S.: While I'm at it, Is there a utility to compare a set (source and target) of md5sums?

Last edited by JZL240I-U; 11-10-2003 at 03:15 AM.
 
Old 11-10-2003, 08:54 AM   #2
stonux
Member
 
Registered: Oct 2003
Distribution: SuSE, RH
Posts: 54

Rep: Reputation: 15
After dumping the ISO image, just enter the following command: md5sum /my/temporary/iso Making the md5sum of the ISO image from harddisk is much faster than from CD.
the magic command to use the standard output of a program twice is tee . To send output of dd to stdout, omit the of= option.

from man md5sum

md5sum [OPTION] --check [FILE]

DESCRIPTION
Print or check MD5 (128-bit) checksums. With no FILE, or when FILE is -, read standard input.

-c, --check
check MD5 sums against given list

so: create a file with file names and md5sums (i.e. the "given list") and issue the
md5sum program with the options above.


Apparently, you like parallel processes. try the following:



dd if=/dev/cdrom bs=n cont=m | tee /my/temporary/iso | md5sum

Last edited by stonux; 11-10-2003 at 09:00 AM.
 
Old 11-11-2003, 02:08 AM   #3
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 13.1 / 12.3_64-KDE, Ubuntu 14.04, Fedora 20, Mint 17, Chakra
Posts: 3,650

Original Poster
Rep: Reputation: Disabled
Quote:
Originally posted by stonux
After dumping the ISO image, just enter the following command: md5sum /my/temporary/iso
Making the md5sum of the ISO image from harddisk is much faster than from CD.
But that's so ... so ... so inartistic .

And it still means reading data twice.

Moreover, it doesn't discover errors during transfer from the CD to the hard-disk .

Quote:
Originally posted by stonux
the magic command to use the standard output of a program twice is tee . To send output of dd to stdout, omit the of= option.
Absolutely M*A*R*V*E*L*O*U*S

(read that as bold+italic+underlined+blinking+magenta on turquoise Background).

That's what I was looking for. And it works, yippeeee.

I was simply stumped, I didn't know where to look. Thanks a load.

This forum is fun .

Quote:
Originally posted by stonux
so: create a file with file names and md5sums (i.e. the "given list") and issue the
md5sum
program with the options above.
That's what I will try next. Thanks in advance.

Quote:
Originally posted by stonux
Apparently, you like parallel processes.
Who doesn't? Reading data once using them multiply -- that's it .
Quote:
Originally posted by stonux
Try the following:
dd if=/dev/cdrom bs=n cont=m | tee /my/temporary/iso | md5sum
Did that, and it worked, even with the bash script (rawread).

Last edited by JZL240I-U; 08-04-2005 at 06:30 AM.
 
Old 11-12-2003, 03:45 AM   #4
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 13.1 / 12.3_64-KDE, Ubuntu 14.04, Fedora 20, Mint 17, Chakra
Posts: 3,650

Original Poster
Rep: Reputation: Disabled
Quote:
Originally posted by stonux man md5sum
Good idea . Even more info gave info md5sum
Quote:
Originally posted by stonux so: create a file with file names and md5sums (i.e. the "given list") and issue the md5sum program with the options above.
That I did, with the "given list" file something like (example md5sum but in the format I used)
Code:
asfgjh276bd7bgbds67 - . /file/to/check/with/md5sum
and the command
Code:
md5sum -c /given/list/file
which produced an error. Since I found no example I don't see for the moment what went wrong (format wrong?). Any more hints / examples, anybody?

Last edited by JZL240I-U; 11-12-2003 at 06:36 AM.
 
Old 11-12-2003, 04:42 AM   #5
yapp
Member
 
Registered: Apr 2003
Location: Netherlands
Distribution: SuSE (before: Gentoo, Slackware)
Posts: 613

Rep: Reputation: 30
perhaps the tee command is what you're looking for.
I guess you can use this in a pipe...

dd if=/dev/cdrom | tee some.iso | md5sum -

?

but I'm not sure how efficient this really is. dd seams to write large blocks at once, and perhaps tee doesn't. Use the "time" command to find out what is faster.

md5sum -c is used the check a file, it requires an .md5 file as input. This file contains some master list of all files, and their md5 sums.

Last edited by yapp; 11-12-2003 at 04:47 AM.
 
Old 11-12-2003, 06:33 AM   #6
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 13.1 / 12.3_64-KDE, Ubuntu 14.04, Fedora 20, Mint 17, Chakra
Posts: 3,650

Original Poster
Rep: Reputation: Disabled
Yes, it was tee, and it is definitely faster (subjectively at least). Remember, that dd is reading by DMA slowly from a CD-ROM and writing by DMA to the disk, while md5sum hashing is done by the CPU. I'd guess, in this case there is little competition for resources.

Quote:
This file contains some master list of all files, and their md5 sums.
Yes, but in what format? You can see in my last post what I used -- without success.
 
Old 11-12-2003, 08:27 AM   #7
yapp
Member
 
Registered: Apr 2003
Location: Netherlands
Distribution: SuSE (before: Gentoo, Slackware)
Posts: 613

Rep: Reputation: 30
the format of the md5 file is simply a list of md5 hashes, and file names. Other text may be included, and doesn't seam to bother "md5sum"

Code:
112f9e2ae9cfbaebfb2137fa2292ab54  ./ChangeLog.txt
e759f1122fc9680a447941d188083a88  ./FAQ.TXT
f46bf6c3d1c925a05de879a4ed5d994f  ./RAWRITE.EXE
1d0954d71851e55ffa27bc6107ee95eb  ./RAWRITE12.DOC
7dac6ea496014d32dbce6c50feb254ca  ./RAWRITE12.EXE
this is the kind of output you should expect:
Code:
diederik@hal9000 zipslack $ md5sum -c CHECKSUMS.md5
./ChangeLog.txt: OK
./FAQ.TXT: OK
./RAWRITE.EXE: OK
./RAWRITE12.DOC: OK
 
Old 11-13-2003, 02:17 AM   #8
JZL240I-U
Senior Member
 
Registered: Apr 2003
Location: Germany
Distribution: openSuSE 13.1 / 12.3_64-KDE, Ubuntu 14.04, Fedora 20, Mint 17, Chakra
Posts: 3,650

Original Poster
Rep: Reputation: Disabled
Well, I got it working with the results as indicated by yapp. The reason for my problems was too much reading on my part . I used
Code:
asfgjh276bd7bgbds67 - . /file/to/check/with/md5sum
as instructed by
Code:
info md5sum
There it says one has to specify the file type (i.e. "." for text): Note, that after the md5sum-result comes
Code:
<blank><minus><blank><dot><blank><file name>
Obviously, that stuck in md5sum's gullet, so first I left out the <blank><dot>.

The next problem was the minus sign after the md5sum-result. That was part of the output of md5sum to the screen (stdout). When redirected to a file, the output was <md5sum-result><blank><filename>, which I could use directly as input for the next run of md5sum.

So, thanks a lot, stonux and yapp, for a most instructive thread.

Last edited by JZL240I-U; 11-13-2003 at 03:32 AM.
 
  


Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Is there any codecs or player to play the stream of "rtsp://" youngpie Linux - Software 3 10-30-2005 11:03 PM
Platform-independent "/dev/null" stream redirection? wapcaplet Programming 1 08-12-2005 10:11 PM
SpamAssassin Error "spamd: no response to STREAM cmd, Success" and other errors PlorkZ Linux - Software 1 11-18-2004 04:16 PM
"Undeleting" data using grep, but get "grep: memory exhausted" error SammyK Linux - Software 2 03-13-2004 03:11 PM
dual booting winME with RedHat 7.2 "HOW TOOO""HELP" jdbarrow Linux - Software 1 03-11-2002 05:33 PM


All times are GMT -5. The time now is 01:43 PM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration