HDD bad sectors causing visual artefacts on playback

[ben_uk]

I’ve spent a lot of time ripping my BD collection mostly to make watching movies easier and to free up some space in the house, most of my discs are now in storage in the cold, damp garage, I never thought I’d be needing them again.

However I have recently noticed that when I played back a film there was visual break up for a fraction of a second. I looked into it and then ran some diagnostics on the HDD, it would appear it had about 100-200 bad sectors. I’m not the worlds best expert of crystaldisk but it said “caution” on the drive status, however confusingly it had less bad sectors than another drive I have that it said was “good” health wise so I’m not really understanding that.

I then ran an app I found called video corruption detector or something, it scanned my drive and I reckon about 15-20 films were flagged as being corrupt.

I guess my point of the post is, I presume there is no actual recovery of this and you have to either put up with the flaws or re-rip the movie? I’m just a bit gutted really as I don’t even know if the discs will play any more.

I’m shocked really to be honest, I didn’t buy cheap drives (all are WD USB Elements external / portable) but it seems virtually all of them have bad sectors, re-allocated sectors and unrecoverable etc. None of the drives are older than 5 years.

[Woodstock]

All I can say is, "Be happy you don't have to rip EVERYTHING again!" I'd recommend copying as much of the drive as is good to another, before the drive starts getting really bad.

Once I could afford it, I went to using a multi-drive server for storing all my video. A single-drive failure is inconvenient, but recoverable, by simply replacing the drive and letting the system cook for a bit. I can still play anything, but it may be slow as it rebuilds the redundancy. You can also set up most of the larger units with two-drive redundancy (it's just a few hundred dollars, right?), which makes it automatic.

In days gone by, drives were small enough that a single bad sector would be flagged immediately. Nowadays, they have entire tracks set aside to move bad sectors to, and drive logic to do it automatically. The problem is, they often won't TELL you that the drive is failing, until you run a special utility that tells about how it's protecting you...

(working on box 2 of over 30 that need to be re-ripped. having separate media servers doesn't help when EVERYTHING is stolen, except the media itself!)

[flojo]

... re-rip the movie?

Yep :-/. Par2 could help. I used to use something like the below to create a small recovery set (and still do for photographs).

Code: Select all

# Creates a 1% recovery set using the filesystems sector size * 1024 for "blocksize". You should get file size and make adjustments for the 1024 multiplier (block count must be <= 2^16 / 2 )
par2 c -s$(expr $(stat -fc %s .) \* 1024) -r1 "movie.mkv"

You could choose to use a more rebust storage system, but you really have to go "all-in" with like 8+ drives for things like RAIDZ-3 to work.

But yeh... re-rip :-/

[dcoke22]

I agree with the others. Copying the data off to another drive is probably valuable.

For recovery, you might try SpinRite. (https://www.grc.com/sr/spinrite.htm) I use SpinRite to exercise any new drives I get before I put them into use. It occasionally finds trouble before it becomes a problem. It is written in x86 assembler, which makes it somewhat difficult to run in 2025. It wants to boot an x86 computer into FreeDOS to run. And it might take days to chew through a big hard drive. I actually have an x86 single board computer that I use just for this purpose so that it can sit in a corner and run until it is done. The last drive I used SpinRite on was a 24 TB drive. I had SpinRite set to its most thorough mode and it took just over 11 days to finish on that 24TB drive.

[flojo]

I should also add that you can verify the streams with hashes if you like, or at least if the video is 1 big .m2ts on the disc (you can also do it if it's chunks of .m2ts files but that involves scripting):

Code: Select all

# You need a hasher, this example uses "rhash" https://github.com/rhash/RHash
# Get the hash of the h264 stream in the file you just made.
ffmpeg -i movie.mkv -map 0:v:0 -c:v copy -f h264 pipe:1 | rhash --printf '%{sha1}\n' -
# Get the hash of the h264 stream in the .m2ts and make sure they both hashes match.
ffmpeg -i 00010.m2ts -map 0:v:0 -c:v copy -f h264 pipe:1 | rhash --printf '%{sha1}\n' -
# Replace "h264" for other codecs, eg. "hevc" (to find more, use: ffmpeg -codecs).

[ben_uk]

Thanks for the replies.

If I wanted to invest in something storage wise with a level of redundancy as suggested, what should I be looking at? Ideally I’d like a semi portable solution with USB interface.

[flojo]

... with USB interface.

With USB, at most a 2 drive RAID mirror, so use 2 20TB drives in a mirror to make 1 20TB. However I have heard bad things about RAID on USB.

You really should look at something like Par2 for this being that you're dealing with extremely low file counts and very large archive sources. This way you could buy a 20TB drive but keep a 2TB for recovery data.

Also you could try SnapRAID with USB, but I've never tried it https://www.snapraid.it/ There's also the option of using MergerFS and simply doubling every file you have (like a RAID mirror), but then you might of well use 2 drives JBOD style. I'm not sure I'd combine SnapRAID and MergerFS on USB, but I've never tried it.