makemkv producing .mkvs with video corruption

[whiskeyalpha82]

Hi,

I've been an avid makemkv fan since "day dot" and chose to demonstrate my support in the not too distant past by purchasing a license.

However, I noticed that a few of my recent blu-ray rips seem to be exhibiting problems with small glitches or video corruption, which are reproducible on various machines (eliminating the player or device being to blame). Most of these issues were resolved by re-ripping, although the problem seems to have become more common with the more recent versions of makemkv.

My immediate assumption was that perhaps my trusty Pioneer BDC-203 has finally started to show its age. However, I think the following scenario suggests that the problem may lie with the application itself:

The film in question is Toy Story 3. After first using the "backup" option (and hence, I assume, ensuring that the m2ts files were CRC checked) to a folder "ts3", I then selected the "Open Files" option and pointed it to "\ts3\BDMV\index.bdmv". After selecting the desired video stream, DTS-HD audio and subs, I proceed to rip the content to a Ubuntu fileserver (via a Samba share).

The image below shows the issue with the resulting file:


To be clear, this is the only place in the entire video where I noticed the issue but I luckily hadn't deleted the m2ts streams from my local disk from the original full-disc backup. I tracked down this particular scene to the corresponding m2ts stream and the issue was not there (even when played with the same player).

I guess that leaves either makemkv (or somehow the transport to a Samba share) to blame.

Does anybody have any thoughts? I'd really like to get this problem fixed so that I can keep ripping and stop worrying

Thanks

[setarip_old]

Hi!

To be clear, this is the only place in the entire video where I noticed the issue

Have you subsequently tried to playback this problematic file - and does it still show the same minor glitch?

I ask because sometimes the smallest speck of detritus (or the smallest wobble) can cause such a momentary glitch - Not likely, but possible...

[mike admin]

Firstly, let's rule out "glitch" case. Can you please make another MKV file from saved backup, this time to the local disc, and see if video will be corrupted at that same place again? Better yet, demux video from "old" mkv file and "new" one and compare files - if they are not byte-to-byte identical then indeed there was some sort of one-time glitch and "old" mkv file got corrupted. If files turn to be identical and both exhibit the issue then at least we'll have a stable problem reproduction steps.

[whiskeyalpha82]

Hi!

To be clear, this is the only place in the entire video where I noticed the issue

Have you subsequently tried to playback this problematic file - and does it still show the same minor glitch?

I ask because sometimes the smallest speck of detritus (or the smallest wobble) can cause such a momentary glitch - Not likely, but possible...

Hi setarip_old,

Thanks for the reply.

Yes, the problem can be reproduced using a variety of different media players on various computers so it is definitely a flaw in the final video.

Firstly, let's rule out "glitch" case. Can you please make another MKV file from saved backup, this time to the local disc, and see if video will be corrupted at that same place again? Better yet, demux video from "old" mkv file and "new" one and compare files - if they are not byte-to-byte identical then indeed there was some sort of one-time glitch and "old" mkv file got corrupted. If files turn to be identical and both exhibit the issue then at least we'll have a stable problem reproduction steps.

Hi mike,

Thanks for the reply.

First things first, let me congratulate you on a fantastic piece of software. It's been worth every penny!

Now the issue at hand:

I've made another mkv (from the local backup) to local disk and the glitch is not present here. Interestingly, it has a different byte count to the one that I ripped directly to the samba share. I'm beginning to think that the issue might be samba. I'd assume if makemkv was really producing these glitches/artifacts then it would lead to a larger number of such issues mentioned on the forum here.

As a sys admin though, I do love a methodical approach to troubleshooting and can't leave a problem as "resolved" until I've found the root cause.

I think I'll produce 5 local rips and see if they all have an identical byte count, followed by 5 rips directly to the samba share. If the local copies come back identical and the samba versions show irregularity or variances, I'll be more comfortable that we've found the culprit.

Thanks for the help.

[whiskeyalpha82]

Hi mike,

Just one other thing I wanted to check:

Am I right in thinking that there is some sort of hash table present on a blu ray disc which allows you to verify that the m2ts streams are backed up perfectly (via a check-sum) when the "backup" option is selected in makemkv?

If so, does ripping the "main-movie only" provide the same kind of protection? (i.e. are check-sums still used to verify the integrity of the copied streams?)

It'd be great to get a bit more understanding around this. If the question has already been asked or this is covered in the documentation then my apologies.

Thanks.

[whiskeyalpha82]

I've been digging a little further tonight. Examine this image from a mkv rip of "The Thing (1982)":



Again, this is the only place in the entire film where we see this glitch but it is reproducible every time, with multiple players, on multiple systems. This was a rip I'd made a little while ago, directly to Samba, main-movie only.

I re-ripped, making the entire process local disk (no Samba share involved). The new file did not have the issue in the scene above.

What's interesting is that the file size in this example appears identical:


What's even more interesting is that pulling a video only makemkv rip from each of these files shows that the video itself is identical in size, yet one has corruption, where the other does not:


Not sure if this helps identifying a cause but it means that my method of using file size as an indication of a successful/consistent rip is not reliable.

[manfriday]

Glad (not really) that it is not just me having issues with video corruption. My corruption is noticeable straight away by a grey bar through the middle third of the screen, then at points throughout the movie typically at change of scene. I can reproduce it and has now occurred on 3 BD. Any input is appreciated.
Cheers
MF

[whiskeyalpha82]

Glad (not really) that it is not just me having issues with video corruption. My corruption is noticeable straight away by a grey bar through the middle third of the screen, then at points throughout the movie typically at change of scene. I can reproduce it and has now occurred on 3 BD. Any input is appreciated.
Cheers
MF

Hey manfriday,

Sorry to hear you're also having a problem. Hopefully mike can provide with some more info on a possible cause when he has time.

Can you provide any more info on your issue so we can begin to see if there is any pattern:

- Does the "corruption" appear in the same place every time the file is played back?
- Have you tried the affected file on other players and machines?
- What's your ripping process? (e.g. backup full Blu ray to local hard disk, then extract main movie to NAS)
- Have you tried re-ripping and does this sometimes solve the issue?
---- if not, is the corruption evident in the same place every time you rip it?
---- if so, is there a difference in file size between the "good" rip and the "bad" one?

Thanks

[manfriday]

Follow up to previous post. It appears to be the Drive. Either dirt or use but I retried the rip on an older system and it worked fine. Call off the dogs

[mike admin]

Am I right in thinking that there is some sort of hash table present on a blu ray disc which allows you to verify that the m2ts streams are backed up perfectly (via a check-sum) when the "backup" option is selected in makemkv?

Yes. This hash table is present for every M2TS file but does not cover few kilobytes at the very end of M2TS file. MakeMKV checks these during backup.

If so, does ripping the "main-movie only" provide the same kind of protection? (i.e. are check-sums still used to verify the integrity of the copied streams?)

No, hashes are checked only in backup mode.

[whiskeyalpha82]

Thanks mike,

I suspected this was the case which is why I started using the full backup to local disk, before extracting the movie only to the samba share.

Unfortunately, in the case of both Toy Story 3 and The Thing examples I gave above, this was the approach I took and both of these glitches were introduced in the second stage (local disk to samba, using makemkv). In the case of The Thing, both files (good and bad) have exactly the same byte count (though I appreciate this doesn't mean the same content).

I think I'll stick to doing everything locally and then transport the final file to the samba share.

Any thoughts?

[mike admin]

As suggested before, please try making another MKV file on local drive and compare it byte-for-byte with "glitchy" one. If files are identical, then it's a bug in MakeMKV. Else it's a glitch (because a bug in MakeMKV or somewhere else).

[whiskeyalpha82]

Thanks mike,

I think that I have conclusive proof that this is in fact a Samba issue:

I created several local rips of Toy Story 3, along with several straight to the Samba share. Byte counts were identical on all files and I was unable to find a scene showing corruption (though clearly, I couldn't watch every scene). Once I was finished, I deleted all but one local "good" copy and then copied that back to the Samba share.

Now the interesting part: Whilst I was watching the film (over the network) ~45mins in, I got corruption in the video just like above. Just to be sure, I went back and looked at the original file on my local disk - as expected, it had an identical modified time stamp, identical byte count etc. to the one that I was viewing from the file server. However, the "master" on my local disk did not have the corruption.

As the only difference between the two was that it had been copied via the network to a samba share, I can only conclude the fault lies there.

EDIT:

Just to confirm I took a small text file with little content on my local "H:\" drive and created an MD5 hash:

Code: Select all

Checksums generated by ExactFile 1.0.0.15

H:\ts3.txt
857 bytes

MD5: d4eae8bcfa1c1429c66d75210ddeb836

I then copied the folder to my linux fileserver (samba share mounted as "Z:\") and hashed it again:

Code: Select all

Checksums generated by ExactFile 1.0.0.15

Z:\ts3.txt
857 bytes

MD5: d4eae8bcfa1c1429c66d75210ddeb836

So far so good, that's what we'd expect, both hashes are identical.


Now onto the mkvs. This is a known good copy on my local drive:

Code: Select all

Checksums generated by ExactFile 1.0.0.15

H:\Video\ts3\Toy_Story_3_GOOD.mkv
21394456214 bytes

MD5: 1f4d3648803d64179725e37670c9c348

I then copied the file to a location on the file server and hashed again:

Code: Select all

Checksums generated by ExactFile 1.0.0.15

Z:\Movies\Blu-ray\Toy Story 3 (2010)\Toy_Story_3_GOOD.mkv
21394456214 bytes

MD5: 2f69a231903672ba5824c7dc4e5b38aa

What do you know! The byte count stays identical but hashing the file reveals that the file is not exactly the same as the one on the local disk (i.e. corruption has occured)!

I copy over the file again and recreate the hash to see if it is a "one-off":

Code: Select all

Checksums generated by ExactFile 1.0.0.15
http://www.exactfile.com

Z:\Movies\Blu-ray\Toy Story 3 (2010)\Toy_Story_3_GOOD.mkv
21394456214 bytes

MD5: d9d66342b54b19f8ed4fff8bcb7aa663

Nope! There's definitely an issue of some kind, which looks to be Samba at this stage. In order to rule out the file system (ext4) being at fault, I'll try copying using scp and see what results I get.

EDIT2:

This copy made using scp also doesn't match the original hash:

Code: Select all

Checksums generated by ExactFile 1.0.0.15

Z:\Movies\Blu-ray\Toy Story 3 (2010)\Toy_Story_3_GOOD.mkv
21394456214 bytes

MD5: 3c8ae1f71d6c172658a8e95567c5b32e

Now I'm suspecting file system issues. I'm using mdadm software raid with ext4 as the file system so I'll need to do some digging there I think.

[Drag0nFly]

Just noticed your post from a Google search. I'm having this exact same issue show up in all the .mkv's I've ripped using MakeMKV (which is currently at version 1.8.2). I'm on OS X. The issue is present on several players – XBMC, VLC, MplayerX, etc.

The problem is also present on both BD-ROM drives I've tried (one a LiteOn iHBS2) and the other a brand-new Asus BW-14D1XT drive. I am not using Samba.
What is curious is that the discs are ripping just fine. Neither one of the glitched Blu-Rays have any problems with the media (neither from visual inspection nor from testing the glitched scenes on two standalone players.)

So something screwy is going on with MakeMKV. I will try the backup option once I upgrade my storage to see if that makes any difference.

[Romansh]

Do you have the same issue as whiskeyalpha82 where the file size for the video track is identical?