LinuxQuestions.org
View the Most Wanted LQ Wiki articles.
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Software
User Name
Password
Linux - Software This forum is for Software issues.
Having a problem installing a new program? Want to know which application is best for the job? Post your question in this forum.

Notices

Reply
 
LinkBack Search this Thread
Old 12-09-2008, 06:03 PM   #1
nonoitall
Member
 
Registered: Feb 2005
Posts: 74

Rep: Reputation: 16
Burning DVD and verifying - no ISO


I am trying to burn the contents of a directory to a DVD and verify the integrity of the burned data afterwards. But I don't want to have to create an intermediate ISO on the hard disk. (Also this has to be accomplished with command line tools.) I was hoping to pipe together mkisofs and growisofs to burn the DVD, and then pipe together mkisofs and dd into cmp to verify the burned data, but the image produced by mkisofs differs each time I create it (timestamps are different, I'm assuming). How can I burn and verify a DVD without having to waste the time and hard disk space to create an intermediate ISO?
 
Old 12-09-2008, 07:01 PM   #2
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Why not compare the files that were burned to DVD with the same files in the source directory.
 
Old 12-09-2008, 07:53 PM   #3
nonoitall
Member
 
Registered: Feb 2005
Posts: 74

Original Poster
Rep: Reputation: 16
It struck me as being a bit less efficient for the tons of little files that I have to backup. (In some ways this is a bit of a proof-of-concept thing for me too.)
 
Old 12-10-2008, 05:21 AM   #4
jschiwal
Guru
 
Registered: Aug 2001
Location: Fargo, ND
Distribution: SuSE AMD64
Posts: 15,733

Rep: Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654Reputation: 654
Are these files such as mp3 files where permissions aren't important? If not, you should use an archiving program and burn the archive file. Besides ownership and permissions, there are also acls and extended attributes. There can also be incompatibility problems with filenames.

Besides tar & star, I like dar for backups. It can create backups in slices, and is written with CD and DVD backups in mind. It also will delete files that should be deleted between incremental backups. There is a KDE front end called kdar. It may not be in your current distro, and you may need to copy an older library version before kdar will work. You can export a backup or restore job to a bash script. This can make it easier to design your own scripts, learning from a model.

If your system aliases mkisofs for the genisoimage command, then there is an option to allow piping the output of tar to genisoimage:
Code:
       -stream-media-size #
              Select streaming operation and set the media size to # sectors.  This allows you to pipe the output of the  tar(1)  pro-
              gram into genisoimage and to create an ISO9660 filesystem without the need of an intermediate tar archive file.  If this
              option has been specified, genisoimage reads from stdin and creates a file with the name STREAM.IMG.  The  maximum  size
              of  the  file  (with padding) is 200 sectors less than the specified media size. If -no-pad has been specified, the file
              size is 50 sectors less than the specified media size.  If the file is smaller, genisoimage will write padding. This may
              take awhile.

              The  option  -stream-media-size  creates  simple ISO9660 filesystems only and may not used together with multisession or
              hybrid filesystem options.
 
Old 12-10-2008, 05:00 PM   #5
nonoitall
Member
 
Registered: Feb 2005
Posts: 74

Original Poster
Rep: Reputation: 16
It's just a smattering of many different documents. So is there no way to verify the actual image? I also tried:

Terminal 1:
Code:
mkfifo pipe.iso
md5sum -b pipe.iso
Terminal 2:
Code:
mkisofs -J -l -R -V <volume name> | tee pipe.iso | dvdrecord -dao dev=2,0,0 -
I hoped to just calculate the MD5 hash as the image was being burned, and then compare that to what finally got written to the disc. However, as soon as I execute the command in the second terminal, md5sum in the first terminal returns immediately, before burning even begins. (And this is a >4GB image - definitely not something that can be entirely buffered in memory and instantly hashed.)
 
Old 12-11-2008, 02:50 AM   #6
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
What I have tried using in the past was this program:
http://unix.freshmeat.net/projects/checkcrc/

It checks CRCs for things being piped through it, so it will work. The only problem was that it was difficult to get accurate results. When writing the ISO to disk, it is sometimes padded with zeros at the end, so this changes the hash. Thus the only plausible way is to know the exact number of bytes that were written without this padding. I tried finding a solution, but gave up so far.
 
Old 01-09-2009, 11:04 PM   #7
alphaniner
LQ Newbie
 
Registered: Jan 2009
Distribution: Arch
Posts: 18

Rep: Reputation: 1
md5deep

I've been looking to do something similar for a while. I use a GUI burner but it pretty much always produces false positives when it checks the disc for errors. I think I've found a reasonable method of checking the files through the terminal, but I pretty much just came up with it so I haven't tested it yet.

slackware isos* contain a text file that contains md5sums for all files in the iso. It looks something like this:

Quote:
These are the MD5 message digests for the files in this directory.
If you want to test your files, use 'md5sum' and compare the values to
the ones listed here.

To test all these files, use this command:

md5sum -c CHECKSUMS.md5 | less

'md5sum' can be found in the GNU coreutils package on ftp.gnu.org in
/pub/gnu, or at any GNU mirror site.

MD5 message digest Filename
305df2c17a09f28bcd40b68c865d1ccb ./ANNOUNCE.12_1
c08333e6569a6fd66fd59999bfa94eb5 ./BOOTING.TXT
3a902db03f5fc96a29488681552afeb8 ./CHANGES_AND_HINTS.TXT
18810669f13b87348459e611d31ab760 ./COPYING
d32239bcb673463ab874e80d47fae504 ./COPYING3
aadcb2dbade050dfafa993820b5b4c37 ./COPYRIGHT.TXT
199ecdab51fe5281f724eef3682cd053 ./CRYPTO_NOTICE.TXT
...
A single command is used to check the md5sums of all files on the disc against those stored in the text file. The problem is creating such a file, as md5sum does not work recursively. Enter md5deep. It can generate md5sums recursively and its output is identical to that in the slackware file.

So here's what I'm thinking. Assuming everything to be burned is in one directory, run the following command in that directory:

Code:
md5deep -rl .
the 'r' is for recursive and the 'l' is for enabling relative file paths. The output will be something like this:

Quote:
66dd47d55b72dc7a8e71927548ae4a6b ./VMware_Licenses
114849fd7b26188a6bae204b68a6d9a7 ./Free State Project.txt
ba6c2a36afcd38d25a94b11a058c5e1e ./bookmarks/bookmarks01.html
16ff51c168405e575d32bae001f280e4 ./pics/debian.jpg
...
Again, I haven't tested this, but conceivably you could pipe the output to a file in the directory to be burned, then after burning run

Code:
md5sum -c <file> | less
from the root directory of the disc.

*Actually this seems to be the means behind the "check disc" option in other distros as well.

---EDIT---

It works, but there's a few caveats.

First, if you pipe the output of the md5deep command into the directory you are 'md5deeping', you will end up with an entry for the piped file, which I think is guaranteed to be wrong. You may want to remove that entry, else you'll get a mismatch when running the md5sum -c command.

The md5deep command doesn't seem to check the files in any logical manner (ie. alphabetically/numerically) even if you have no sub-directories. That's only an issue if you are OCD like me though. It doesn't affect the functionality of the process.

Finally, there's the time factor. I burned 11 files totaling 3.7 GB (based on du -h). It only took about a minute to calculate the md5sums of the files on the HDD, but seven minutes to verify the files on disc.

Also, it is not necessary to actually burn the md5sum file to the disc. When you go to run the md5sum -c command, cd to the root directory of the disc and specify a path for the file, relative or absolute.

Last edited by alphaniner; 01-12-2009 at 10:45 PM.
 
Old 02-26-2009, 06:33 PM   #8
nonoitall
Member
 
Registered: Feb 2005
Posts: 74

Original Poster
Rep: Reputation: 16
Well I finally got around to messing with it long enough to come up with a satisfactory solution. For anyone who's interested, this is the script I came up with:
Code:
#!/bin/bash

#Usage: burndir <dir> <device>

#Make sure we are running as root (otherwise we can't burn later)
if [ `whoami` != "root" ]
then
echo "This script must be run as root."
exit 1
fi

DIRPATH="$1"
DEVICE="$2"

TMPDIR=`mktemp -d` || exit 1
ISOPIPE="$TMPDIR/pipe.iso"
MKISOERR="$TMPDIR/mkisoerr"

mkfifo "$ISOPIPE"

echo "Burning directory..."

growisofs -dvd-compat -Z "$DEVICE"="$ISOPIPE" &
BURNER=$!
MD5=`mkisofs -r -J -l "$DIRPATH" 2>"$MKISOERR" | tee "$ISOPIPE" | md5sum | egrep -o -i -e "^[0-9a-f]{32}"`

EXTENTS=`egrep -e "^[0-9]+ extents written " "$MKISOERR" | egrep -o -e "^[0-9]+"`

wait "$BURNER"

echo "Wrote $EXTENTS extents with a MD5 hash of $MD5."
echo "Cycling drive tray..."

eject "$DEVICE"
eject -t "$DEVICE"

echo "Verifying..."

DISCMD5=`dd if="$DEVICE" bs=2048 count="$EXTENTS" | md5sum | egrep -o -i -e "^[0-9a-f]{32}"`

echo "Disc has an MD5 hash of $DISCMD5."

if [ "$MD5" == "$DISCMD5" ]
then
echo "Disc appears to have burned successfully!"
else
echo "Disc appears to have burned with errors."
fi

rm -r "$TMPDIR"
There aren't really a whole lot of sanity checks, but it seems to work for me.
 
Old 02-27-2009, 04:22 AM   #9
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
Thanks, so does it actually work every time no matter in what mode you burn the disk in ? And how long does it take to get the checksum ? I wrote a script to burn CDs and DVD in a similar way (see my site) and I was wondering if I should add such a feature.
 
Old 02-27-2009, 05:25 AM   #10
nonoitall
Member
 
Registered: Feb 2005
Posts: 74

Original Poster
Rep: Reputation: 16
I'm a bit of a newbie so I'm not quite sure what type of modes you're referring to. As far as calculating the hash goes, it calculates the "correct" hash as the image is being generated and piped to growisofs. (Unless your processor is like ancient, I imagine just about any PC should be able to calculate MD5 fast enough to keep up with a DVD burner's speed.) After the disc is burned and the tray is ejected/closed, to verify the disc, I just used dd to read the contents of the disc and pipe that into md5sum again to calculate the hash of what was actually written to the disc. (Again, just about any processor should be able to keep up with a DVD drive's read speed.) So, all-in-all, it takes as long as you'd expect a burn/verify operation to take - however long it takes to burn the disc and then read back its contents. (A grand total of around 13 or 14 minutes for a full DVD on my burner.)

All that's created on the hard disk is one temporary directory, a named pipe, and a log of mkisofs's stderr output (to find out how many meaningful extents we're dealing with). I'm sure a more adept script writer could probably do away with the temporary directory completely, but that's not me. All I wanted was to be able to efficiently and thoroughly verify the disc without using up 4+ GB of space for an intermediate image file, and this did the trick.

Last edited by nonoitall; 02-27-2009 at 05:27 AM.
 
Old 02-27-2009, 05:57 AM   #11
H_TeXMeX_H
Guru
 
Registered: Oct 2005
Location: $RANDOM
Distribution: slackware64
Posts: 12,928
Blog Entries: 2

Rep: Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266Reputation: 1266
Alright, thanks, I'll try it out then.
 
  


Reply

Tags
burn, cd, command, dvd, iso, line, verify


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off
Trackbacks are Off
Pingbacks are On
Refbacks are Off


Similar Threads
Thread Thread Starter Forum Replies Last Post
Burning DVD from two iso files jr_linux Linux - Software 6 03-05-2008 01:14 PM
? Burning a DVD iso image RexCoeus Linux - Newbie 2 11-11-2007 03:16 PM
DVD iso burning Flab0y352 Ubuntu 2 08-09-2007 12:10 AM
Burning DVD ISO file to something besides an DVD Samoth Linux - General 4 07-04-2006 05:08 PM
Burning ISO image (DVD+R, external DVD writer) swan2925 Linux - Software 6 06-02-2005 09:49 AM


All times are GMT -5. The time now is 06:53 AM.

Main Menu
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
identi.ca: @linuxquestions
Facebook: linuxquestions Google+: linuxquestions
Open Source Consulting | Domain Registration