Strange corruption issue - MakeMKV's fault or bad HDD?

Please post here for issues related to DVD discs
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

Billycar11 wrote:
Tue May 14, 2024 4:28 am
i write 1.2TB over usb a day
Hmm interesting. That is a lot of video and there is no way you could be watching 1.2TB a day so maybe you should scan the files for errors with ffmpeg afterwards.

It's a similar situation where you buy a DVD box set and think you've got it safely but until you actual play every disc you could have a bad disc in there and not know it until a year later when you finally got around to watching/ripping that episode.

Billycar11 wrote:
Tue May 14, 2024 4:28 am
but its only using about 106MB/s
Hmm well my third party AsMedia ports which seem to be immune to corruption will only do 90MB/sec (180MB/sec total read+write) whereas the Intel ports does about 130MB/sec (260MB/sec total read+write).

So just to clarify, I've got my DVD .ISO file on the USB drive, and I'm extracting it to mkv files with the destination being the same USB drive. I figured if MakeMKV is reporting 130MB/sec writing to mkv files, that must mean it's also reading at 130MB/sec from the ISO file, making the total 130+130=260 is that correct?

Back in 2014 when this motherboard was new, were people even connecting SSD's over USB 3.0 and getting these kinds of fast speeds like 260MB/sec? Maybe that's why it wasn't detected in testing or reported by many people until recently where more people are actually trying to utilise USB 3.0 to its full potential and seeing that it actually breaks when you stress it too hard.
Billycar11
Posts: 3905
Joined: Sun Aug 24, 2014 5:49 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by Billycar11 »

pneumatic wrote:
Tue May 14, 2024 5:16 am
Billycar11 wrote:
Tue May 14, 2024 4:28 am
i write 1.2TB over usb a day
Hmm interesting. That is a lot of video and there is no way you could be watching 1.2TB a day so maybe you should scan the files for errors with ffmpeg afterwards.

It's a similar situation where you buy a DVD box set and think you've got it safely but until you actual play every disc you could have a bad disc in there and not know it until a year later when you finally got around to watching/ripping that episode.

Billycar11 wrote:
Tue May 14, 2024 4:28 am
but its only using about 106MB/s
Hmm well my third party AsMedia ports which seem to be immune to corruption will only do 90MB/sec (180MB/sec total read+write) whereas the Intel ports does about 130MB/sec (260MB/sec total read+write).

So just to clarify, I've got my DVD .ISO file on the USB drive, and I'm extracting it to mkv files with the destination being the same USB drive. I figured if MakeMKV is reporting 130MB/sec writing to mkv files, that must mean it's also reading at 130MB/sec from the ISO file, making the total 130+130=260 is that correct?

Back in 2014 when this motherboard was new, were people even connecting SSD's over USB 3.0 and getting these kinds of fast speeds like 260MB/sec? Maybe that's why it wasn't detected in testing or reported by many people until recently where more people are actually trying to utilise USB 3.0 to its full potential and seeing that it actually breaks when you stress it too hard.
its all drive testing im that guy selling flashed and tested UHD drives
Buy a UHD drive from the guide and how to video maker: https://www.makemkv.com/forum/viewtopic ... 20&t=17831
UHD Drives Guide: https://www.makemkv.com/forum/viewtopic ... 16&t=19634
Auto flash kit $25 Email me for one Billycar5924@gmail.com
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

Also I checked S.M.A.R.T attributes in CrystalDiskInfo many times and it never says anything is wrong. So the SSD is blissfully unaware that it's writing garbage to the cells, more reason to think it's happening downstream at the USB stage.

I had seriously considered whether it's the USB to SATA controller's fault but I've had the same corruption on 2 different drives that use different USB to SATA controllers so I figured it's unlikely (albeit not impossible - maybe they both use some dodgy write caching strategies)
dcoke22
Posts: 2693
Joined: Wed Jul 22, 2020 11:25 pm

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by dcoke22 »

Have you thought about a USB PCIe card?
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

Billycar11 wrote:
Tue May 14, 2024 5:52 am
its all drive testing im that guy selling flashed and tested UHD drives
Oh I see. Can I ask what software you use for testing? I was looking at this one called FreeFileSync to verify the actual contents of files are the same. I think first it checks to see if the number of bytes are the same, and if they are then it will start comparing contents but I couldn't find a clear answer on that. My corrupt files have the same number of bytes but different checksums.
Last edited by pneumatic on Tue May 14, 2024 10:15 am, edited 1 time in total.
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

dcoke22 wrote:
Tue May 14, 2024 5:58 am
Have you thought about a USB PCIe card?
Yes, that may be a good option but I'm going to try the SATA rack bay thing first as well.
Billycar11
Posts: 3905
Joined: Sun Aug 24, 2014 5:49 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by Billycar11 »

pneumatic wrote:
Tue May 14, 2024 5:58 am

Oh I see. Can I ask what software you use for testing? I was looking at this one called FreeFileSync to verify the actual contents of files are the same between 2 discs. I think first it checks to see if the number of bytes are the same, and if they are then it will start comparing contents but I couldn't find a clear answer on that. My corrupt files have the same number of bytes but different checksums.
just makemkv i do full backup of UHD blurays on blurays it has a hash table and checks the files are correct as it copies. not sure if it checks before or after it writes to disk i asked once but dont think ill get an answer.
not sure if it does that on dvds but you saw makemkv said something when it saw weird junk
Buy a UHD drive from the guide and how to video maker: https://www.makemkv.com/forum/viewtopic ... 20&t=17831
UHD Drives Guide: https://www.makemkv.com/forum/viewtopic ... 16&t=19634
Auto flash kit $25 Email me for one Billycar5924@gmail.com
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

Billycar11 wrote:
Tue May 14, 2024 6:09 am
not sure if it does that on dvds but you saw makemkv said something when it saw weird junk
Makemkv never gives me any error messages. Anyway screw these USB ports I'm done wasting my life on them I'll get a PCI-E card with hopefully 8 working USB 3.2 ports which might be even faster than 3.0, now that I think about it I should have done this ages ago.
Radiocomms237
Posts: 377
Joined: Mon Oct 18, 2021 12:23 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by Radiocomms237 »

Billycar11 wrote:
Tue May 14, 2024 6:09 am
~ on blurays it has a hash table and checks the files are correct as it copies. not sure if it checks before or after it writes to disk ~
The hash table can only verify the integrity of the files on the optical disc itself. It cannot be used to check for file copy errors once the content has been decrypted and muxed into an .mkv file.
Billycar11 wrote:
Tue May 14, 2024 6:09 am
~ not sure if it does that on dvds ~
AFAIK there is no way to detect errors from DVDs unless it finds a sector of the disc is unreadable.
Billycar11
Posts: 3905
Joined: Sun Aug 24, 2014 5:49 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by Billycar11 »

Radiocomms237 wrote:
Tue May 14, 2024 7:52 am
Billycar11 wrote:
Tue May 14, 2024 6:09 am
~ on blurays it has a hash table and checks the files are correct as it copies. not sure if it checks before or after it writes to disk ~
The hash table can only verify the integrity of the files on the optical disc itself. It cannot be used to check for file copy errors once the content has been decrypted and muxed into an .mkv file.
just adding some real world experience still dont know exactly where and when it happens but when a stick of ram went bad in my test pc i was getting hash errors on pioneers and well thats not possible they dont have that jamless play junk lg has so makemkv was catching it somewhere between it decrypting it in ram and after writing it to disk/hdd it may be catching it in ram or it may catch it after it writes it and then compares not sure.

mike has eluded to adding checking after the fact eventually.
dcoke22 linked it a few days ago
viewtopic.php?p=119692#p119692
Buy a UHD drive from the guide and how to video maker: https://www.makemkv.com/forum/viewtopic ... 20&t=17831
UHD Drives Guide: https://www.makemkv.com/forum/viewtopic ... 16&t=19634
Auto flash kit $25 Email me for one Billycar5924@gmail.com
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

It seems a lot of these third party PCIe USB expansion cards could be cheaply made junk prone to failure and corruption as well. Not sure I want to risk 2 years of archiving to some $30 PCB made to a price. Maybe I just have to get used to the idea of scanning everything with ffmpeg. Maybe that is just the reality if you want data integrity.
dcoke22
Posts: 2693
Joined: Wed Jul 22, 2020 11:25 pm

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by dcoke22 »

pneumatic wrote:
Tue May 14, 2024 10:24 am
It seems a lot of these third party PCIe USB expansion cards could be cheaply made junk prone to failure and corruption as well. Not sure I want to risk 2 years of archiving to some $30 PCB made to a price. Maybe I just have to get used to the idea of scanning everything with ffmpeg. Maybe that is just the reality if you want data integrity.
I don't really know all that much about USB chipsets, but something from Sonnet (https://www.sonnettech.com/product/allegro-pro-usb31-pcie.html) or another company serving the video editor market seems like a reasonable bet. That market generally prioritizes reliability above all else since time is money for those folks.
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

dcoke22 wrote:
Tue May 14, 2024 2:19 pm
pneumatic wrote:
Tue May 14, 2024 10:24 am
It seems a lot of these third party PCIe USB expansion cards could be cheaply made junk prone to failure and corruption as well. Not sure I want to risk 2 years of archiving to some $30 PCB made to a price. Maybe I just have to get used to the idea of scanning everything with ffmpeg. Maybe that is just the reality if you want data integrity.
I don't really know all that much about USB chipsets, but something from Sonnet (https://www.sonnettech.com/product/allegro-pro-usb31-pcie.html) or another company serving the video editor market seems like a reasonable bet. That market generally prioritizes reliability above all else since time is money for those folks.
That looks better but costs more than a new motherboard.

Saw this Inateck brand one on Amazon that seems to have good reviews but 2 of them costs more than a new motherboard.

There's another generic popular one using a Renesas UPD720201 controller that only costs $30 so I'll try one of those and see if I can reproduce any corruption on it. But one of the amazon reviews is saying it uses an Asmedia controller so maybe there's some chip lottery going on.

I was thinking about how the corruption manifests as copying wrong blocks of memory and what could be deduced from that. I think that could mean the data itself isn't getting corrupted because the corrupted data is MPEG2 video from another DVD but still in tact because the media player can still decode it into video and audio which seems to imply it's simply reading from the wrong address rather than the data getting scrambled.
pneumatic
Posts: 114
Joined: Sat Apr 01, 2023 8:09 am

Re: Strange corruption issue - MakeMKV's fault or bad HDD?

Post by pneumatic »

Got the unbranded PCI USB card off ebay which looks like this

Image
https://www.amazon.com.au/Expansion-Exp ... B0B5M1J3LY

They've scrubbed out the chip numbers for whatever reason but they are Renesas UPD720201 and UDP720210.

There is no driver support for Windows 7, only 8+ so I had to hunt down the drivers from the support page of another branded version of it (Startech - https://www.startech.com/en-au/cards-adapters/pexusb3s7).

The drivers are ancient from 2012 but the release notes have some interesting things to say about various USB bugs they found which were causing BSODs and such.

Speed is on par with my good mobo ports (Asmedia) - 90MB/sec (180MB/sec total read+write). Intel ports which suffer corruption do 260MB/sec so maybe that is a good sign they don't runf aster.

No corruption yet but we shall see.

In the meantime I have uninstalled the Intel Rapid Storage Technology driver in case that was causing the issue, as it is another common denominator in all the corruption scenarios. USB speed not affect after uninstall.

Spotted another issue that was giving me false *negatives* for corruption - Windows seems to cache files in RAM so when I run ffmpeg.exe to check the mkvs for errors, the USB drive light was not flashing. I take this to mean the result is invalid as it must be getting the mkv data from system RAM and not actually checking the bytes on disk. This is consistent with a scenario where a mkv file had no corruption according to ffmpeg.exe but then had corruption after copying it elsewhere.
Post Reply