Wrong Number of Frames?

[KYThrill]

I have had trouble with glitches in the output of MakeMKV 1.6.8 and 1.6.10. There are glitches in almost everything I ripped in May and June (BD and DVD). In the process of trying to track down the source of these glitches, I noticed that if I use eac3to to extract the H264 stream from a mkv made by MakeMKV, it reports an incorrect number of frames.

For example, if I use a blu-ray of 30 Days of Night and make a MKV of the video and lossless audio, then use eac3to to demux out the video stream, eac3to reports that video stream has 325389 frames. If I use eac3to to extract the original video stream from the disc, it reports 162695. This is basically 1/2 of what eac3to reports the MakeMKV video stream to be.

Why are these numbers so different. Is MakeMKV doing some type of frame doubling or interpolation?

[Romansh]

Most likely, eac3to is guessing the number of frames from the framerate. I've noticed that FFmpeg tends to report some MakeMKV rips as being 59.94 fps (as opposed to the actual framerate, which is usually either 23.976 or 29.97 fps). My guess is that whichever MKV demuxer is used by eac3to has the same issue.

No idea of whether it's a MakeMKV or eac3to issue, however.

[KYThrill]

Most likely, eac3to is guessing the number of frames from the framerate. I've noticed that FFmpeg tends to report some MakeMKV rips as being 59.94 fps (as opposed to the actual framerate, which is usually either 23.976 or 29.97 fps). My guess is that whichever MKV demuxer is used by eac3to has the same issue.

No idea of whether it's a MakeMKV or eac3to issue, however.

When eac3to scans the MKV, it lists the streams (1,2,3, etc.). In this list, it correctly identifies the frame rate of the video stream. So if eac3to can correctly identify it for the purpose of listing the streams within the MKV, then their shouldn't be an error with count of the total frames, correct?

[mike admin]

Why are these numbers so different. Is MakeMKV doing some type of frame doubling or interpolation?

No, MakeMKV puts source stream as is. Moreover, when you extract streams with eac3to from MKV or from disc, the files should be idenical - they should have same size and same md5 sum. do they?

[KYThrill]

Why are these numbers so different. Is MakeMKV doing some type of frame doubling or interpolation?

No, MakeMKV puts source stream as is. Moreover, when you extract streams with eac3to from MKV or from disc, the files should be idenical - they should have same size and same md5 sum. do they?

When I extract the H264 video streams from disc and from my mkv file made by MKV, they do have slightly different file sizes. The stream extracted from the MKV is usually a few hundred kb more than the stream extracted from the disc. So naturally, the checksum is different. eac3to also reports the video stream from mkv as having nearly 2X as many frames (but file size doesn't reflect this).

And this is true on rips made with 1.6.5, 1.6.8, and 1.6.10 of MKV. Video streams extracted from the MKV do not match video streams extracted from the disc, on anything I've checked. File size is always slightly different and naturally checksums do not match.

On the discs I've tested, I've extracted the video stream directly from the Blu-ray disc. I have then used DVD FAB to create an ISO on the HDD. I then extract the video stream from the ISO on HDD. In all instances, the file size, frame count, and MD5 checksum have been identical when comparing the disc video stream and the HDD ISO video stream. I then use MakeMKV to create a MKV with the same streams as the ISO (same video, audio, and subtitle). Extracting the video stream always results in a file size, frame count, and MD5 checksum that is different that what I get from the disc and HDD ISO. I thought it could perhaps be something to do with MakeMKV's method of handling AACS, so I tried the same test with anyDVD HD enabled. It does not change the situation. With MakeMKV 1.65, I get an identical video stream with anyDVD is enabled or disabled, with file sizes, total frames, and MD5 matching in both cases.

Now, the situation is more complex when I try this with 1.6.8 and 1.6.10 because of the other problem with me not be able to get consistent rips with those versions.

All of this extraction is fairly time consuming to do for troubleshooting. Let me look at the discs I have and see if I can't find a feature that is shorter (maybe some type of half hour to hour long animated feature). This would let me speed up my troubleshooting and maybe allow you to perform the same tests to see if you get identical results. We should both be able to extract the same video stream, get the same file sizes, frame counts, and MD5 checksums, and they should match what is on the original disc, if we are truly getting a 1:1 copy.

[KYThrill]

Okay. Here is what I have. I will be doing testing using Mater's Tall Tales. It is about a 35 min. feature, so everything goes relatively quickly. I am using eac3to v 3.24.I used the 00801.mpls, which is the 439.m2ts file. AACS v17. For each test I ripped the AVC H264 video stream and the DTS-HD MA audio stream. MD5 Summer was used to calculate the checksums. This testing is ongoing, so I will be editing this post to include all new data and details as testing is performed.

First I extracted the streams directly from the Blu-ray disc.

H264 Video is:
File Size: 6,496,873 KB
Total Frames: 51,246
MD5 Checksum: 970496cd9d40c995d964777f5283ff2c

DTS-HD MA Audio is:
File Size: 1,087,786 KB
MD5 Checksum: bc8ae42b41ac37bbf1b0fe87eb83fb9b

Then I ripped the disc to HDD and extracted the streams from the HDD.

H264 Video is:
File Size: 6,496,873 KB
Total Frames: 51,246
MD5 Checksum: 970496cd9d40c995d964777f5283ff2c

DTS-HD MA Audio is:
File Size: 1,087,786 KB
MD5 Checksum: bc8ae42b41ac37bbf1b0fe87eb83fb9b

So disc and rip of disk to HDD are identical. Next, I use MakeMKV 1.6.5 to create a MKV and extract the streams.

H264 Video is:
File Size: 6,497,181 KB
Total Frames: 102,493
MD5 Checksum: f947e3dac63ebdc94acab012916b94a4

DTS-HD MA Audio is:
File Size: 1,087,786 KB
MD5 Checksum: bc8ae42b41ac37bbf1b0fe87eb83fb9b

So you can see that MakeMKV made an identical copy of the audio, but the H264 stream is different. So next, I use 1.6.10 to create a MKV and extract the streams.

H264 Video is:
File Size: 6,497,181 KB
Total Frames: 102,493
MD5 Checksum: f947e3dac63ebdc94acab012916b94a4

DTS-HD MA Audio is:
File Size: 1,087,786 KB
MD5 Checksum: bc86843d8ea1d9e4ab73b78d0d7c3663

As you can see with 1.6.10, the video stream is identical to the video stream from 1.6.5, but both are different than the stream on the original disc. However, the file size for the audio stream is the same, but the checksum is different. This hits on my other thread about inconsistent output from MakeMKV 1.6.10. I believe the checksum is different on the audio from 1.6.10 because some random errors are introduced by MakeMKV. If I run MakeMKV again, I will get different checksums. Sometimes the errors show up in video, sometimes audio, and sometimes both places. Ths problem goes away when I roll back to 1.6.5. But with 1.6.10 and 1.6.8, I can't run the program two times and get the same output both times.

So I see two errors, one where random corruption of the streams creeps in, and another where the video stream seems to be altered from the original stream on the disc. I've seen this same error on every title I've tested.

[KYThrill]

And just to prove that it wasn't anything inherent to the MKV container, I used eac3to to extract video, audio, subtitles, and chapters from the original disc. I then remuxed them using mkvmerge (specifying 24000/1001 for the H264 frame rate). The mkv played perfectly. I then extracted the video, audio, and subtitle streams from that mkv and compared them to the original streams from the disc. All had the same file sizes, same total frames on the video. and matching checksums.

So mkvmerge was able to place the streams into a MKV container without altering them any. This would lead me to conclude that there is a bug(s) in MakeMKV preventing users from getting a true 1:1 copy in the MKV container.

[KYThrill]

I just tested a DVD as well. I used Devil's Advocate. I used DVD Decrypter to extract the M2V video stream from the disc. I then used MakeMKV 1.6.5 to create an MKV of the disc. I then extracted the M2V stream from the resulting MKV. This time the MakeMKV M2V was 2,027 KB smaller than the M2V stream from the disc. The duration differed by 2 seconds.

[KYThrill]

I played around some more last night with DVD's and MakeMKV. I used both DVD Decrypter and PGC Demux to extract the m2v files on three different DVD's. In all three cases the file size, duration, and checksum between the DVD Decrypter and PGC Demux outputs were the same. However, they did not match file size, duration, or checksum with the m2v streams extracted from the MKV files made by MakeMKV and extracted with eac3to.

I guess my next step is to use MKV Merge and put together an MKV with one of the m2v files I extracted using PGC Demux. Then use eac3to to extract the m2V back out of the MKV. If the m2v matches before and after, then it would mean MKV Merge and eac3to can mux and demux m2v without altering it, which would just leave MakeMKV to blame for the difference I'm seeing it the output now.

But without doing this check, I can't be sure if it is MakeMKV or eac3to which is altering the m2v, and eac3to doesn't work directly on DVD VOB's.

[KYThrill]

In troubleshooting this problem with Mike, he discovered that MakeMKV was inserting two headers on each frame for H264 streams instead of one, which was doubling the frame count when extracted by eac3to (it counts headers to determine the number of frames).

I did some more testing last night and if you remux the MKV with MKV Merge, it will remove the extra header and reduce the file size slightly.

Just an FYI.

[gmd024]

Thanks for the update -- is Mike still working on this from his/MakeMKV's side of things

[crowfax]

Thanks for the update -- is Mike still working on this from his/MakeMKV's side of things

I believe so. He posted yesterday saying the seeking issue would be fixed in about a week, I expect this issue will also be fixed as they both appear to be related to the new multiplexing engine.

[mike admin]

In troubleshooting this problem with Mike, he discovered that MakeMKV was inserting two headers on each frame for H264 streams instead of one, which was doubling the frame count when extracted by eac3to (it counts headers to determine the number of frames).

I did some more testing last night and if you remux the MKV with MKV Merge, it will remove the extra header and reduce the file size slightly.

Just an FYI.

It turned out to be a bit more complex. MakeMKV was always preserving the stream as is, but it was mkvmerge who always removed a certain header. Since mkvmerge is a "matroska specification", the author of eac3to implemented a hack - eac3to adds this header to each frame if you extract h.264 stream from MKV file. So, when eac3to extracts stream from an MKV file produced by MakeMKV, the header is inserted twice - once from original stream and one by eac3to. Try demuxing the MKV file by mkvextract, and you'll get output without two headers. And yes, this header accounts for an extra frame. To add more confusion, the next version of MakeMKV will also strip this header like mkvmerge.

[gmd024]

Thanks for the update Mike