Bit Rot Detection

The Snakeoil Squad has created a quick Linux script that can test the integrity of your FLAC music library. The script is packed as a Snakeoil module and you can find this in the Snakeoil Technical Support Forums. Refer to the Snakeoil Technical Reference on how to install a Snakeoil module.

Once installed, you need to SSH into your Snakeoil machine and run the script like so:

cd /media/music
./testflac.sh

You can see a sample of the output here, corrupted files will be listed with the prefix of Error. Files that past the test will not be shown, so please be be patient and wait for the process to finish. Ideally you’d only want to see nothing at all pop up on the screen. At the end of the run the total number of FLAC files tested will be shown, along with the number of detected corrupted files. Like so:

You’d also get another file called testflac.log if there are any corrupted files. This file is located in your home directory and it contains a list of all the files that failed the integrity test. For example:

#cat ~/testflac.log
./Corrupted.flac
#

Bit rot should be a very rare occurrence. There should be no more than 1 file per 10,000 on average. If your total number of corrupted files exceed this ratio, please leave me a comment because I’d be interested to know your storage details.

Note that this script only works with FLAC, and it expects to have write permission to the /media/music folder. Expect some bugs as this script may need more finetuning to suit operating environments that is different to my setup.

As mentioned previously this script will not be able to test WAV formats as it do not support integrity verification. For other formats that do please modify the script as you see fit, and send me patches so I can update the module for every Snakeoil user to use.

Once you’ve verified the integrity of your existing files,  replacing any that were corrupted, you may want to consider moving your files to a more secure file system. According to the Internet, you’d need to use modern file systems with built in checksum support, e.g.

• ZFS
• Btrfs

Not entirely certain how true this claim actually is as I’m mid-way into this myself (I’m using FreeNAS with ZFS). It’d be a few years before I can say for certain bit rot is a thing of the past. Alternatively, you can perform regular backups and verify the backed up copy against the working copy.

Disclaimer

This test script scans your file system and test the integrity of all the FLAC files it can find. This is a non-invasive test. Having said that, the increased I/O may cause a HDD/SSD on it’s last legs to die. The Snakeoil Squad is not responsible for any bad thing that happens after you run this script. If in doubt, maintain a regular backup schedule before embarking on this, or simply do not execute this.

Pages: 1· 2· 3· 4· 5· 6· 7· 8· 9· 10· 11· 12· 13