LinuxQuestions.org
Download your favorite Linux distribution at LQ ISO.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Linux Forums > Linux - Server
User Name
Password
Linux - Server This forum is for the discussion of Linux Software used in a server related context.

Notices


Reply
  Search this Thread
Old 07-27-2016, 11:45 AM   #1
rnturn
Senior Member
 
Registered: Jan 2003
Location: Illinois (SW Chicago 'burbs)
Distribution: openSUSE, Raspbian, Slackware. Previous: MacOS, Red Hat, Coherent, Consensys SVR4.2, Tru64, Solaris
Posts: 2,803

Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
Detecting File Corruption


This question isn't about looking for a definitive answer. It's more of a "how would you do this?" post.

The reason I'm asking:

Rather than stream music the other day, I went old-school and pointed an MP3 player at a filesystem containing a slew (several tens of GB worth) of MP3 and Ogg files I've either collected or ripped from my personal CD collection over time. I ran into a problem that I've experienced (thankfully, infrequently) in the past: Every once in a while I'll find that a music file has developed problems where it's skipping (almost like a CD will do when it's gotten dirty) or playback has some other form of distortion that I hadn't heard before. I figure that the file has become corrupted in some way and try to make a mental note to re-rip that track -- or. more likely, that CD -- in my spare time (that stuff we never seem to have).

The music files that got me to thinking further about this are on an older (ext3) filesystem built on top of an md mirror that's been migrated from a couple of other systems as they were replaced. (I wasn't thinking about individual files getting clobbered. More concerned about losing all of them.)

For a time I've wondered if there was a way to detect whether file corruption has taken place. One method I was thinking of was to generate a simple file of checksums in each directory -- I have each artist in their own directory with subdirectories for each CD (with additional subdirectories for multi-disc releases) -- that would be generated when the Ogg or MP3 was generated. Then, each night a process would run to re-compute the checksum for each file and compare it to the original checksum, adding any discrepancies to a report that I'd get via email.

Unfortunately, that doesn't help me to figure out which of the existing files have already gotten corrupted but it will help in the future (I'm hoping).

Thoughts?

How would you detect file corruption?

Is there some filesystem feature I've overlooked all this time that would alert me when files have been dinged?

The system that's hosting the music filesystem is due for an OS upgrade (currently looking at Labor Day weekend to do this). Anyone have experience that btrfs is a better at preventing file corruption? I'm betting that the answer is 'yes', especially compared to ext3, but would love to hear first-hand experience.

Is it possible, despite being informed of "success" at the time the CD was ripped, that the files used to create the MP3 were themselves corrupted? (I.e., that icedax lied to me and created broken WAV files? :^/ )

What other aspects might I have overlooked?

Any comments are greatly appreciated.

TIA...

--
Rick
 
Old 07-27-2016, 01:58 PM   #2
smallpond
Senior Member
 
Registered: Feb 2011
Location: Massachusetts, USA
Distribution: Fedora
Posts: 4,149

Rep: Reputation: 1264Reputation: 1264Reputation: 1264Reputation: 1264Reputation: 1264Reputation: 1264Reputation: 1264Reputation: 1264Reputation: 1264
File corruption without some underlying problem is rare. Check the system log for disk errors, check the SMART stats for each drive, and check the md mirror for mismatches.
 
Old 07-31-2016, 01:38 PM   #3
Beryllos
Member
 
Registered: Apr 2013
Location: Massachusetts
Distribution: Debian
Posts: 529

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
I found this blog item:

Verifying or Validating MP3 Files: MP3-Check, MP3val, MP3 Diags, Foobar2000, and Others

But haven't actually read or used it.
 
Old 08-04-2016, 09:34 AM   #4
sundialsvcs
LQ Guru
 
Registered: Feb 2004
Location: SE Tennessee, USA
Distribution: Gentoo, LFS
Posts: 10,671
Blog Entries: 4

Rep: Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945Reputation: 3945
I would not presume that the files are actually damaged unless and until I ran a file-content validator (such as the above) on each of them.

There are many plausible reasons why a file might "skip" on playback.

Naturally, you should use the chmod command on the directory to prevent the files from being casually tampered-with by making them "read only."

However ... this is the reason why I still buy my music on shiny plastic (and, big dark vinyl) disks.
 
Old 08-05-2016, 11:01 AM   #5
rnturn
Senior Member
 
Registered: Jan 2003
Location: Illinois (SW Chicago 'burbs)
Distribution: openSUSE, Raspbian, Slackware. Previous: MacOS, Red Hat, Coherent, Consensys SVR4.2, Tru64, Solaris
Posts: 2,803

Original Poster
Rep: Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550Reputation: 550
Quote:
Originally Posted by Beryllos View Post
I found this blog item:

Verifying or Validating MP3 Files: MP3-Check, MP3val, MP3 Diags, Foobar2000, and Others

But haven't actually read or used it.
All seem to be Windows-only and, thus, not useful to me. But that's a category I can exercise The Google with.
 
Old 08-05-2016, 03:21 PM   #6
ondoho
LQ Addict
 
Registered: Dec 2013
Posts: 19,872
Blog Entries: 12

Rep: Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053Reputation: 6053
rnturn, there was some problem with mp3 and variable bitrate (which has been added to mp3 later on iiuc) versus fixed bitrate and some header information that might or might not be present.
which might result in playback problems.
so the files wouldn't be actually corrupted but rather outdated or wrongly encoded, or the player not be able to deal with these discrepancies.
just my 2ct.
 
Old 08-05-2016, 08:34 PM   #7
Beryllos
Member
 
Registered: Apr 2013
Location: Massachusetts
Distribution: Debian
Posts: 529

Rep: Reputation: 319Reputation: 319Reputation: 319Reputation: 319
Quote:
Originally Posted by rnturn View Post
All seem to be Windows-only and, thus, not useful to me. But that's a category I can exercise The Google with.
Oops! I didn't realize that the links in that blog article all point to Windows versions, but some of those tools are open source and available for Linux.
 
  


Reply



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
File System Corruption shahidmgh Red Hat 2 07-12-2010 01:26 AM
file corruption lcslouis Linux - Networking 3 06-26-2010 02:30 PM
file corruption!! vishamr2000 Linux - General 5 05-12-2006 01:45 AM
File Corruption in Red Hat 7.3 File Server dmurph2k2 Red Hat 3 10-09-2003 02:30 PM
File Corruption on Red Hat 7.3 File Server dmurph2k2 Linux - General 2 10-09-2003 12:38 PM

LinuxQuestions.org > Forums > Linux Forums > Linux - Server

All times are GMT -5. The time now is 12:18 PM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration