Audio drops out of sync in transcoded mkv files

[simbakeeper]

Hello =)

I've been transcoding my video files I made using makemkv. My encoding script can be found in this forum, too. Now I've been experiencing audio/video sync issues. The audio drops out of sync exponentially as the video progresses towards the end.

I have checked the original mkv files using mediainfo and they do not contain audio delay. Also I believe makemkv corrects audio delay to mkv files. I use the "expert mode" and "show AV sync issues" options in makemkv, but I never get any messages about them in the output.

The next step for me was to extract the audio and video timecodes from the original mkv file and mux it into the final mkv file I create using mkvmerge. Here is how I extract and mux them:

Code: Select all

Extract:
mkvextract timecodes_v2 "video.mkv" 1:"video-timecodes.txt" \
    2:"audio-timecodes.txt"

Muxing:
mkvmerge --default-language eng -o "video-muxed.mkv" \
    --language 0:eng --compression 0:none --timecodes 0:"video-timecodes.txt" -d 0 --no-chapters -A -S "video.h264" \
    --language 0:eng --compression 0:none --timecodes 0:"audio-timecodes.txt" -a 0 --no-chapters -D -S "audio.m4a" \
    --chapter-language eng --chapters "chapters.txt"

But this produced an error in the muxing process:

Code: Select all

mkvmerge v6.7.0 ('Back to the Ground') 64bit built on Jan  9 2014 18:03:17
'video.h264': Using the demultiplexer for the format 'AVC/h.264'.
'audio.m4a': Using the demultiplexer for the format 'QuickTime/MP4'.
'video.h264' track 0: Using the output module for the format 'AVC/h.264 (unframed)'.
'audio.m4a' track 0: Using the output module for the format 'AAC'.
The file 'video-muxed.mkv' has been opened for writing.
'video.h264' track 0: Extracted the aspect ratio information from the MPEG-4 layer 10 (AVC) video data and set the display dimensions to 720/405.
Warning: 'audio.m4a' track 0: The number of external timecodes 945 is smaller than the number of frames in this track. The remaining frames of this track might not be timestamped the way you intended them to be. mkvmerge might even crash.
mkvmerge: /build/buildd/libmatroska-1.4.1/make/linux/../../src/KaxCluster.cpp:279: int16 libmatroska::KaxCluster::GetBlockLocalTimecode(uint64) const: Assertion `TimecodeDelay >= int16(0x8000) && TimecodeDelay <= int16(0x7FFF)' failed.
./mux.sh: line 4: 12944 Aborted                 mkvmerge --default-language eng -o "video-muxed.mkv" --language 0:eng --compression 0:none --timecodes 0:"video-timecodes.txt" -d 0 --no-chapters -A -S "video.h264" --language 0:eng --compression 0:none --timecodes 0:"audio-timecodes.txt" -a 0 --no-chapters -D -S "audio.m4a" --chapter-language eng --chapters "chapters.txt"

Then I left out the audio timecodes file and it worked out fine. I do not know yet if my new files are in perfect sync, I will post more about it when I watch more transcoded material. My question is, should I even use an audio timecode file? And how do other people do this? How do you get the audio and video in perfect sync?

[ndjamena]

When you remux with mkvmerge, make sure you explicitly set the frame rate to 24/1.001. I got that problem with the first episode of Star Trek TOS, it turned out the frame rate was 23.9700, which was wrong (but close enough that mediainfo reported it as the standard frame rate).

[simbakeeper]

Thank you very much, ndjamena. I didn't know that. I'll give that a try tonight

Last night I tried to watch one of the videos which I remuxed like I said in the first post, including the timecodes. It was terrible. The video was fast forwarding like someone had pressed the "fast forward" button

So, do I even need the original timecodes for the video/audio tracks or should I just use the option for both tracks what ndjamena suggested, instead?

Code: Select all

--default-duration 0:24000/1001fps

[ndjamena]

AAC audio frames have a different duration than DTS (10.666_ ms) or AC3 (32ms), therefore the audio time codes in the original file are useless. The fast forwarding is odd, but if it's one of the files with the old audio time codes it's likely the difference between the time code durations and actual frame durations is causing the problem. Otherwise it's possible you double rate deinterlaced the video and that's the cause.

[simbakeeper]

AAC audio frames have a different duration than DTS (10.666_ ms) or AC3 (32ms), therefore the audio time codes in the original file are useless.

Understood. Are the video timecodes useless too? I also re-encoded the video.

The fast forwarding is odd, but if it's one of the files with the old audio time codes it's likely the difference between the time code durations and actual frame durations is causing the problem.

Maybe this occured because I used the old timecodes with the re-encoded video? I don't know, I have to try it out.

Otherwise it's possible you double rate deinterlaced the video and that's the cause.

I don't even know how to do this, so yes, it might be that I did this accidentally. What I do know is that the video is progressive, so no deinterlacing was done.

[ndjamena]

If the video is still 24000/1001 fps then the frame durations will be the same and the time codes should be fine.

-Edit- Wait, UNLESS IT'S INTERLACED VC1, which separates the fields when it encodes them which are then considered indiuvidual frames as far as Default Duration is concerned.

-Edit 2- Is this a DVD or Blu Ray?

-Edit 3- Sorry, I'm just waking up after being stuck on an iphone for a few days.

[simbakeeper]

Source is blu-ray. The video is 24000/1001. If I set the default duration of the audio track using --default-duration 0:24000/1001fps, I could use the same for the video track. That would eliminate the need to use the original timecodes. So maybe the question now is: do I benefit anything if I use the original timecodes file?

[ndjamena]

Only if it's variable frame rate, which it's not.

-Edit- You don't set the default duration of the audio to 24/1.001, the audio frames have a different duration to the video frames.

[simbakeeper]

OK. Thanks for all your help, ndjamena =)

[simbakeeper]

-Edit- You don't set the default duration of the audio to 24/1.001, the audio frames have a different duration to the video frames.

Do I use the "default-duration" setting at all for the audio track? And if I should use it, what should be the value for the setting?

[ndjamena]

Audio frames are temporal and are made up of a certain number of temporal samples, so generally the duration shouldn't be altered for any reason. The only valid reason I'm aware of to attempt to alter an audio streams default duration is when you're adding a secondary audio stream to a blu ray rip from a sped up DVD rip or vice versa, although it may not play back properly if you try.

[simbakeeper]

ndjamena:
How did you find out the frame rate when you had that "Star Trek TOS" problem? I have now watched about 8 or so episodes that I remuxed using the help you gave and most of them seemed fine, but I thought I noticed some delay in two episodes... Or maybe I'm just imagining things. I used this to retrieve the frame rate info on all of my mkv files I have ripped using makemkv:

Code: Select all

find -name "*.mkv" -type f -exec ffprobe {} -v 0 -select_streams v -print_format flat -show_entries stream=r_frame_rate \;

Each one returned 'streams.stream.0.r_frame_rate="24000/1001"' as the result. So the frame rates should be the same and what I expected.

[ndjamena]

If I load the file into VirtualDub through an AVISynth script it shows the frame rate as 23.970fps, if I then save that as an avi and try to re-encode the avi with x264, x264 shows the frame rate as 2500000/104297, which comes out to about 23.970008725083175930276038620478fps. MediaInfo reports the avi as being 23.976fps.

Maybe it's a problem with AVISynth, VirtualDub, DirectShowSource or the avi container but I've heard of other people needing to set the frame rate during muxing to keep sync so I'm not the only one who's discovered it. Did it actually help with your episodes?

Is the out of sync audio just at the end or does it continue all the way through? If you're processing the audio and video separately, there's also the possibility you've run into the orphaned b-frames thing. If you look at the original MKV using MediaInfo does it report any kind of delay in the audio? (I noticed an audio delay in some of my Get Smart episodes at one point, but it turned out it was in the source files too, and on the original DVD.)

[simbakeeper]

If I load the file into VirtualDub through an AVISynth script it shows the frame rate as 23.970fps, if I then save that as an avi and try to re-encode the avi with x264, x264 shows the frame rate as 2500000/104297, which comes out to about 23.970008725083175930276038620478fps. MediaInfo reports the avi as being 23.976fps.

I tried linux mediainfo and windows MediaInfo (the one with the GUI). Both show no delays whatsoever in these episodes. I have delays in later episodes, but I'm not quite there yet.

Maybe it's a problem with AVISynth, VirtualDub, DirectShowSource or the avi container but I've heard of other people needing to set the frame rate during muxing to keep sync so I'm not the only one who's discovered it. Did it actually help with your episodes?

Yes I think it did. The sync was way off and very noticeable before.

Is the out of sync audio just at the end or does it continue all the way through? If you're processing the audio and video separately, there's also the possibility you've run into the orphaned b-frames thing. If you look at the original MKV using MediaInfo does it report any kind of delay in the audio? (I noticed an audio delay in some of my Get Smart episodes at one point, but it turned out it was in the source files too, and on the original DVD.)

What are orphaned b-frames and is there a way to fix those? There is no delay in these episodes I watched last night, but that is a good point, I should look the originals too if they have the same delay =)

[ndjamena]

Orphan b-frames occur when you cut an open gop video stream at an I-frame and manage to displace a b-frame from one of it's references in the process. MakeMKV seems to do it quite a lot. There's not much you can do about it, but when faced with orphaned b-frames some decoders simply skip those frames making the video a few frames shorter than it was in the container. AVISynth's MPEG2Source will replace the frames with copies of the first I-frame but FFVideoSource will just skip them and apparently append two frames to the end of the video, making everything 80ms out of sync (people can notice when audio is just 60ms off). I've been wondering how many of my FF problems might be caused by the codec pack I have installed, I might check next time I reformat my SSD and see if the bugs disappear. If you look at the files through MediaInfo it reports a negative delay in the audio of about 2 frames (-80ms for PAL), but that's just because at the moment it only checks the time code of the first frame it finds, in the next update that bug will be fixed and early b-frames in an open gop will be reported in their own field (early closed GOP b-frames will be ignored). Not all early b-frames are orphans, but you'd have to make an attempt to decode them to figure out if they are or not (which I guess is why MakeMKV leaves them there).

I'm not sure how prevalent the decoding problems are though.