LinuxQuestions.org - Burning DVD and verifying

- Linux - Software (https://www.linuxquestions.org/questions/linux-software-2/)

- - Burning DVD and verifying - no ISO (https://www.linuxquestions.org/questions/linux-software-2/burning-dvd-and-verifying-no-iso-689438/)

Burning DVD and verifying - no ISO

I am trying to burn the contents of a directory to a DVD and verify the integrity of the burned data afterwards. But I don't want to have to create an intermediate ISO on the hard disk. (Also this has to be accomplished with command line tools.) I was hoping to pipe together mkisofs and growisofs to burn the DVD, and then pipe together mkisofs and dd into cmp to verify the burned data, but the image produced by mkisofs differs each time I create it (timestamps are different, I'm assuming). How can I burn and verify a DVD without having to waste the time and hard disk space to create an intermediate ISO?

Why not compare the files that were burned to DVD with the same files in the source directory.

It struck me as being a bit less efficient for the tons of little files that I have to backup. (In some ways this is a bit of a proof-of-concept thing for me too.)

Are these files such as mp3 files where permissions aren't important? If not, you should use an archiving program and burn the archive file. Besides ownership and permissions, there are also acls and extended attributes. There can also be incompatibility problems with filenames.

Besides tar & star, I like dar for backups. It can create backups in slices, and is written with CD and DVD backups in mind. It also will delete files that should be deleted between incremental backups. There is a KDE front end called kdar. It may not be in your current distro, and you may need to copy an older library version before kdar will work. You can export a backup or restore job to a bash script. This can make it easier to design your own scripts, learning from a model.

If your system aliases mkisofs for the genisoimage command, then there is an option to allow piping the output of tar to genisoimage:

Code:



      -stream-media-size #

              Select streaming operation and set the media size to # sectors.  This allows you to pipe the output of the  tar(1)  pro-

              gram into genisoimage and to create an ISO9660 filesystem without the need of an intermediate tar archive file.  If this

              option has been specified, genisoimage reads from stdin and creates a file with the name STREAM.IMG.  The  maximum  size

              of  the  file  (with padding) is 200 sectors less than the specified media size. If -no-pad has been specified, the file

              size is 50 sectors less than the specified media size.  If the file is smaller, genisoimage will write padding. This may

              take awhile.



              The  option  -stream-media-size  creates  simple ISO9660 filesystems only and may not used together with multisession or

              hybrid filesystem options.

It's just a smattering of many different documents. So is there no way to verify the actual image? I also tried:

Terminal 1:

Code:

mkfifo pipe.iso

md5sum -b pipe.iso

Terminal 2:

Code:

mkisofs -J -l -R -V <volume name> | tee pipe.iso | dvdrecord -dao dev=2,0,0 -

I hoped to just calculate the MD5 hash as the image was being burned, and then compare that to what finally got written to the disc. However, as soon as I execute the command in the second terminal, md5sum in the first terminal returns immediately, before burning even begins. (And this is a >4GB image - definitely not something that can be entirely buffered in memory and instantly hashed.)

What I have tried using in the past was this program:
http://unix.freshmeat.net/projects/checkcrc/

It checks CRCs for things being piped through it, so it will work. The only problem was that it was difficult to get accurate results. When writing the ISO to disk, it is sometimes padded with zeros at the end, so this changes the hash. Thus the only plausible way is to know the exact number of bytes that were written without this padding. I tried finding a solution, but gave up so far.

I've been looking to do something similar for a while. I use a GUI burner but it pretty much always produces false positives when it checks the disc for errors. I think I've found a reasonable method of checking the files through the terminal, but I pretty much just came up with it so I haven't tested it yet.

slackware isos* contain a text file that contains md5sums for all files in the iso. It looks something like this:

Quote:

These are the MD5 message digests for the files in this directory.
If you want to test your files, use 'md5sum' and compare the values to
the ones listed here.

To test all these files, use this command:

md5sum -c CHECKSUMS.md5 | less

'md5sum' can be found in the GNU coreutils package on ftp.gnu.org in
/pub/gnu, or at any GNU mirror site.

MD5 message digest Filename
305df2c17a09f28bcd40b68c865d1ccb ./ANNOUNCE.12_1
c08333e6569a6fd66fd59999bfa94eb5 ./BOOTING.TXT
3a902db03f5fc96a29488681552afeb8 ./CHANGES_AND_HINTS.TXT
18810669f13b87348459e611d31ab760 ./COPYING
d32239bcb673463ab874e80d47fae504 ./COPYING3
aadcb2dbade050dfafa993820b5b4c37 ./COPYRIGHT.TXT
199ecdab51fe5281f724eef3682cd053 ./CRYPTO_NOTICE.TXT
...

A single command is used to check the md5sums of all files on the disc against those stored in the text file. The problem is creating such a file, as md5sum does not work recursively. Enter md5deep. It can generate md5sums recursively and its output is identical to that in the slackware file.

So here's what I'm thinking. Assuming everything to be burned is in one directory, run the following command in that directory:

Code:

md5deep -rl .

the 'r' is for recursive and the 'l' is for enabling relative file paths. The output will be something like this:

Quote:

66dd47d55b72dc7a8e71927548ae4a6b ./VMware_Licenses
114849fd7b26188a6bae204b68a6d9a7 ./Free State Project.txt
ba6c2a36afcd38d25a94b11a058c5e1e ./bookmarks/bookmarks01.html
16ff51c168405e575d32bae001f280e4 ./pics/debian.jpg
...

Again, I haven't tested this, but conceivably you could pipe the output to a file in the directory to be burned, then after burning run

Code:

md5sum -c <file> | less

from the root directory of the disc.

*Actually this seems to be the means behind the "check disc" option in other distros as well.

---EDIT---

It works, but there's a few caveats.

First, if you pipe the output of the md5deep command into the directory you are 'md5deeping', you will end up with an entry for the piped file, which I think is guaranteed to be wrong. You may want to remove that entry, else you'll get a mismatch when running the md5sum -c command.

The md5deep command doesn't seem to check the files in any logical manner (ie. alphabetically/numerically) even if you have no sub-directories. That's only an issue if you are OCD like me though. It doesn't affect the functionality of the process.

Finally, there's the time factor. I burned 11 files totaling 3.7 GB (based on du -h). It only took about a minute to calculate the md5sums of the files on the HDD, but seven minutes to verify the files on disc.

Also, it is not necessary to actually burn the md5sum file to the disc. When you go to run the md5sum -c command, cd to the root directory of the disc and specify a path for the file, relative or absolute.

Well I finally got around to messing with it long enough to come up with a satisfactory solution. For anyone who's interested, this is the script I came up with:

Code:

#!/bin/bash



#Usage: burndir <dir> <device>



#Make sure we are running as root (otherwise we can't burn later)

if [ `whoami` != "root" ]

then

echo "This script must be run as root."

exit 1

fi



DIRPATH="$1"

DEVICE="$2"



TMPDIR=`mktemp -d` || exit 1

ISOPIPE="$TMPDIR/pipe.iso"

MKISOERR="$TMPDIR/mkisoerr"



mkfifo "$ISOPIPE"



echo "Burning directory..."



growisofs -dvd-compat -Z "$DEVICE"="$ISOPIPE" &

BURNER=$!

MD5=`mkisofs -r -J -l "$DIRPATH" 2>"$MKISOERR" | tee "$ISOPIPE" | md5sum | egrep -o -i -e "^[0-9a-f]{32}"`



EXTENTS=`egrep -e "^[0-9]+ extents written " "$MKISOERR" | egrep -o -e "^[0-9]+"`



wait "$BURNER"



echo "Wrote $EXTENTS extents with a MD5 hash of $MD5."

echo "Cycling drive tray..."



eject "$DEVICE"

eject -t "$DEVICE"



echo "Verifying..."



DISCMD5=`dd if="$DEVICE" bs=2048 count="$EXTENTS" | md5sum | egrep -o -i -e "^[0-9a-f]{32}"`



echo "Disc has an MD5 hash of $DISCMD5."



if [ "$MD5" == "$DISCMD5" ]

then

echo "Disc appears to have burned successfully!"

else

echo "Disc appears to have burned with errors."

fi



rm -r "$TMPDIR"

There aren't really a whole lot of sanity checks, but it seems to work for me.

Thanks, so does it actually work every time no matter in what mode you burn the disk in ? And how long does it take to get the checksum ? I wrote a script to burn CDs and DVD in a similar way (see my site) and I was wondering if I should add such a feature.

I'm a bit of a newbie so I'm not quite sure what type of modes you're referring to. As far as calculating the hash goes, it calculates the "correct" hash as the image is being generated and piped to growisofs. (Unless your processor is like ancient, I imagine just about any PC should be able to calculate MD5 fast enough to keep up with a DVD burner's speed.) After the disc is burned and the tray is ejected/closed, to verify the disc, I just used dd to read the contents of the disc and pipe that into md5sum again to calculate the hash of what was actually written to the disc. (Again, just about any processor should be able to keep up with a DVD drive's read speed.) So, all-in-all, it takes as long as you'd expect a burn/verify operation to take - however long it takes to burn the disc and then read back its contents. (A grand total of around 13 or 14 minutes for a full DVD on my burner.)

All that's created on the hard disk is one temporary directory, a named pipe, and a log of mkisofs's stderr output (to find out how many meaningful extents we're dealing with). I'm sure a more adept script writer could probably do away with the temporary directory completely, but that's not me. All I wanted was to be able to efficiently and thoroughly verify the disc without using up 4+ GB of space for an intermediate image file, and this did the trick.

Alright, thanks, I'll try it out then.