Linux - ServerThis forum is for the discussion of Linux Software used in a server related context.
Notices
Welcome to LinuxQuestions.org, a friendly and active Linux Community.
You are currently viewing LQ as a guest. By joining our community you will have the ability to post topics, receive our newsletter, use the advanced search, subscribe to threads and access many other special features. Registration is quick, simple and absolutely free. Join our community today!
Note that registered members see fewer ads, and ContentLink is completely disabled once you log in.
If you have any problems with the registration process or your account login, please contact us. If you need to reset your password, click here.
Having a problem logging in? Please visit this page to clear all LQ-related cookies.
Get a virtual cloud desktop with the Linux distro that you want in less than five minutes with Shells! With over 10 pre-installed distros to choose from, the worry-free installation life is here! Whether you are a digital nomad or just looking for flexibility, Shells can put your Linux machine on the device that you want to use.
Exclusive for LQ members, get up to 45% off per month. Click here for more info.
Distribution: openSUSE, Raspbian, Slackware. Previous: MacOS, Red Hat, Coherent, Consensys SVR4.2, Tru64, Solaris
Posts: 2,803
Rep:
Detecting File Corruption
This question isn't about looking for a definitive answer. It's more of a "how would you do this?" post.
The reason I'm asking:
Rather than stream music the other day, I went old-school and pointed an MP3 player at a filesystem containing a slew (several tens of GB worth) of MP3 and Ogg files I've either collected or ripped from my personal CD collection over time. I ran into a problem that I've experienced (thankfully, infrequently) in the past: Every once in a while I'll find that a music file has developed problems where it's skipping (almost like a CD will do when it's gotten dirty) or playback has some other form of distortion that I hadn't heard before. I figure that the file has become corrupted in some way and try to make a mental note to re-rip that track -- or. more likely, that CD -- in my spare time (that stuff we never seem to have).
The music files that got me to thinking further about this are on an older (ext3) filesystem built on top of an md mirror that's been migrated from a couple of other systems as they were replaced. (I wasn't thinking about individual files getting clobbered. More concerned about losing all of them.)
For a time I've wondered if there was a way to detect whether file corruption has taken place. One method I was thinking of was to generate a simple file of checksums in each directory -- I have each artist in their own directory with subdirectories for each CD (with additional subdirectories for multi-disc releases) -- that would be generated when the Ogg or MP3 was generated. Then, each night a process would run to re-compute the checksum for each file and compare it to the original checksum, adding any discrepancies to a report that I'd get via email.
Unfortunately, that doesn't help me to figure out which of the existing files have already gotten corrupted but it will help in the future (I'm hoping).
Thoughts?
How would you detect file corruption?
Is there some filesystem feature I've overlooked all this time that would alert me when files have been dinged?
The system that's hosting the music filesystem is due for an OS upgrade (currently looking at Labor Day weekend to do this). Anyone have experience that btrfs is a better at preventing file corruption? I'm betting that the answer is 'yes', especially compared to ext3, but would love to hear first-hand experience.
Is it possible, despite being informed of "success" at the time the CD was ripped, that the files used to create the MP3 were themselves corrupted? (I.e., that icedax lied to me and created broken WAV files? :^/ )
File corruption without some underlying problem is rare. Check the system log for disk errors, check the SMART stats for each drive, and check the md mirror for mismatches.
rnturn, there was some problem with mp3 and variable bitrate (which has been added to mp3 later on iiuc) versus fixed bitrate and some header information that might or might not be present.
which might result in playback problems.
so the files wouldn't be actually corrupted but rather outdated or wrongly encoded, or the player not be able to deal with these discrepancies.
just my 2ct.
All seem to be Windows-only and, thus, not useful to me. But that's a category I can exercise The Google with.
Oops! I didn't realize that the links in that blog article all point to Windows versions, but some of those tools are open source and available for Linux.
LinuxQuestions.org is looking for people interested in writing
Editorials, Articles, Reviews, and more. If you'd like to contribute
content, let us know.