I have 20TB of ripped and decrypted movies that I've been accumulating over my life. They were all ripped from purchased DVDs as main movie only, and all ripped movies/episodes have a standard DVD structure i.e. they contain a VIDEO_TS folder that holds multiple VOB files containing all audio and subtitle tracks that came with the original DVD. My source media is on one NAS box and my transcoded media is on another NAS box - all locally accessed over ethernet and connected to my 2019 MacBook Pro.
I'm using makemkvcon in a script to transcode from VOB to MKV while keeping only one audio track (typically the 5.1 English if available) and no subtitles. I've separated the stuff needing subtitles for later. During the transcoding I'll typically see quite a few lines similar to:
AV sync issue in stream 0 at 0:05:30.680 : video frame timecode differs by +3.483ms
Even though there's a lot of lines like that being produced as the script runs through my files, I'm ignoring them because I'd be hard pressed to notice a 5ms sync issue.
However, every now and then I'll see a bunch of lines like:
AV sync issue in stream 1 at 0:20:59.040 with duration of 32ms : broken timecode, apparent audio skew is -241.066ms
AV sync issue in stream 1 at 0:28:13.824 with duration of 32ms : broken timecode, apparent audio skew is -278.833ms
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
AV sync issue in stream 1 at 0:30:46.976 with duration of 32ms : broken timecode, apparent audio skew is -317.6ms
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
AV sync issue in stream 1 at 0:36:20.352 with duration of 32ms : broken timecode, apparent audio skew is -346.733ms
Program reads data faster than it can write to disk, consider upgrading your hard drive if you see many of these messages.
AV sync issue in stream 1 at 0:44:36.128 with duration of 32ms : broken timecode, apparent audio skew is -395.166ms
AV sync issue in stream 0 at 0:45:37.101 : video stream has 5295 frames with invalid timecodes
1 titles saved
Copy complete. 1 titles saved.
Output lines like the above are concerning because that amount of sync skew is definitely noticeable. I'll view the MKV file via VLC at the indicated time code and sure enough the audio/video sync is awful. So I went back to the original VOB structure and played the movie at the same time code via VLC and it was just fine! This is happening to a significant number of my transcodes, and I simply can't trust the MKV files being produced.
I'm not sure what's happening here and was hoping somebody could help me.
Transcoding VOB to MKV yielding bad sync issues
Re: Transcoding VOB to MKV yielding bad sync issues
I do have some further information. Quite some time ago I compressed the very same source material using HandBrake and that process yielded an MKV file, however those compressed MKV files do NOT exhibit the same audio/video sync as that the MakeMKV transcoding produced. It stands to reason that there is something about the transcoding process (as opposed to the compression process) that is problematic with respect to maintaining sync. Note that I did first try MKVToolNix to do the transcoding, but that tool was vastly worse with sync issues. When asking about the issue on their forums, I was bluntly told nobody had touched the VOB side of the program in eons and that there would be no further attention paid to that aspect. That led me to MakeMKV, where I have found the sync issues to be much better, although still present.
I do have plans to compress my media after all the MKV transcodings, however the tool I want to use doesn't accept VOB structures so that's why I'm going through the intermediate step of transcoding to MKV.
I do have plans to compress my media after all the MKV transcodings, however the tool I want to use doesn't accept VOB structures so that's why I'm going through the intermediate step of transcoding to MKV.
Re: Transcoding VOB to MKV yielding bad sync issues
Can be PTS issues within the VOBs. You can try VobBlanker: https://www.videohelp.com/software/VobBlanker
Re: Transcoding VOB to MKV yielding bad sync issues
Sorry for the long delay in getting back to this. I'm on a Mac so VobBlanker isn't going to work for me. Additionally it's abandonware and the last version is from back in 2007.
I don't know what a PTS is (although I'd guess that the TS bit is Time Stamp). That being said, if the VOB has some kind of time stamp error you'd think playing the VOB would also yield an audio/video sync issue - but it doesn't.
I subsequently thought the sync issue might be due to some kind of discontinuities between all the constituent 1GB VOB files so I tried:
cat *.VOB > VTS_01_1.VOB
That still played with perfect audio/video sync in VLC. So I then ran MakeMKV on the single VOB version and it produced the same really horrible audio/video sync issues as the scenario when everything was the original separate 1GB VOB files.
So I'm of the opinion that this is a bug in MakeMKV and would like to report it as such (how do I do that?). I'm happy to provide a link to a sample movie file on Dropbox if that would help, and I'd also be happy to beta test the fix prior to any subsequent release if that also helps.
I don't know what a PTS is (although I'd guess that the TS bit is Time Stamp). That being said, if the VOB has some kind of time stamp error you'd think playing the VOB would also yield an audio/video sync issue - but it doesn't.
I subsequently thought the sync issue might be due to some kind of discontinuities between all the constituent 1GB VOB files so I tried:
cat *.VOB > VTS_01_1.VOB
That still played with perfect audio/video sync in VLC. So I then ran MakeMKV on the single VOB version and it produced the same really horrible audio/video sync issues as the scenario when everything was the original separate 1GB VOB files.
So I'm of the opinion that this is a bug in MakeMKV and would like to report it as such (how do I do that?). I'm happy to provide a link to a sample movie file on Dropbox if that would help, and I'd also be happy to beta test the fix prior to any subsequent release if that also helps.
Re: Transcoding VOB to MKV yielding bad sync issues
I have a further thought about this, given that for the same input VIDEO_TS file structure MakeMKV yields audio/video sync issues when creating an MKV but HandBrake doesn't when compressing and converting to MKV. The sync issue in MakeMKV certainly doesn't happen all the time, but I do see it about 15% of the time. The tell-tale signature is what you start seeing lines like the following in the log:
AV sync issue in stream 1 at 0:44:36.128 with duration of 32ms : broken timecode, apparent audio skew is -395.166ms
If you never looked at the logs and just sanity-check played the MKV in a few spots you might not notice anything, but if you go to the vicinity of the time stamps where AV sync issues are reported in the >100msec range you'll definitely notice a problem. I've certainly had MKV conversions done where there have been no errors at all and that's obviously all good. I've had others where the worst sync duration I've seen is <10msec - playing back any of those will also effectively seem fine.
So here's my inexperienced take on this - it's totally hand-wavey, possibly just plain wrong and will likely have knowledgeable people doing facepalms. I'm guessing all the issues I'm seeing are for converted DVDs that had some kind of scratch or imperfection on the DVD before ripping. Just like on a VHS tape, a VOB file has the audio and video intertwined. A damaged disc will play with a hiccup where the scratches are but will inherently hold onto it's sync because that's just how the audio and video are put down. In an MKV the audio and video need to be extracted as separate files and then (somehow) kept in sync using time stamps when played. I'm guessing that when these glitches occur MakeMKV isn't so robust and you end up with these sync issues. MKVToolNix has the same issue, but it happens much more frequently and yields much worse sync divergences. When HandBrake compresses the VIDEO_TS to MKV, I believe it's also doing that on the fly (i.e. directly operating on the VOB as a stream and not converting to MKV first) and that's why it always seems to maintain sync.
It's feasible that the % of ripped DVDs converted to MKV by MakeMKV that have these sync issues correlates with % of DVDs that had visible scratches or blemishes.
Anyway, at this point I'll be happy if somebody just responds at all - even if just to tell me I'm an idiot...
AV sync issue in stream 1 at 0:44:36.128 with duration of 32ms : broken timecode, apparent audio skew is -395.166ms
If you never looked at the logs and just sanity-check played the MKV in a few spots you might not notice anything, but if you go to the vicinity of the time stamps where AV sync issues are reported in the >100msec range you'll definitely notice a problem. I've certainly had MKV conversions done where there have been no errors at all and that's obviously all good. I've had others where the worst sync duration I've seen is <10msec - playing back any of those will also effectively seem fine.
So here's my inexperienced take on this - it's totally hand-wavey, possibly just plain wrong and will likely have knowledgeable people doing facepalms. I'm guessing all the issues I'm seeing are for converted DVDs that had some kind of scratch or imperfection on the DVD before ripping. Just like on a VHS tape, a VOB file has the audio and video intertwined. A damaged disc will play with a hiccup where the scratches are but will inherently hold onto it's sync because that's just how the audio and video are put down. In an MKV the audio and video need to be extracted as separate files and then (somehow) kept in sync using time stamps when played. I'm guessing that when these glitches occur MakeMKV isn't so robust and you end up with these sync issues. MKVToolNix has the same issue, but it happens much more frequently and yields much worse sync divergences. When HandBrake compresses the VIDEO_TS to MKV, I believe it's also doing that on the fly (i.e. directly operating on the VOB as a stream and not converting to MKV first) and that's why it always seems to maintain sync.
It's feasible that the % of ripped DVDs converted to MKV by MakeMKV that have these sync issues correlates with % of DVDs that had visible scratches or blemishes.
Anyway, at this point I'll be happy if somebody just responds at all - even if just to tell me I'm an idiot...
Re: Transcoding VOB to MKV yielding bad sync issues
You must not have used MakeMKV initially to rip your DVDs? MakeMKV usually fails to rip a disc if it encounters an unrecoverable error while reading a disc. MakeMKV's goal is to create an exact copy of the data on the disc.
If that's the case, then what MakeMKV is seeing when it attempts to make a .mkv from one of your VIDEO_TS folders should also be seen by Handbrake; some of the data from the DVD is missing. It might manifest as an audio or video glitch at the same time as at MakeMKV sees a giant audio skew. It is unclear to me from reading your post if that's the case or not.
Asked more directly, does Handbrake always work (and produce glitch free output) in cases where MakeMKV seems to find a giant audio skew?
If that's the case, then what MakeMKV is seeing when it attempts to make a .mkv from one of your VIDEO_TS folders should also be seen by Handbrake; some of the data from the DVD is missing. It might manifest as an audio or video glitch at the same time as at MakeMKV sees a giant audio skew. It is unclear to me from reading your post if that's the case or not.
Asked more directly, does Handbrake always work (and produce glitch free output) in cases where MakeMKV seems to find a giant audio skew?
Re: Transcoding VOB to MKV yielding bad sync issues
I did not use MakeMKV to rip the dvd - I used MacDVDRipper Pro (or MDRP) to decrypt and extract the main movie while maintaining the original format (VIDEO_TS folder). Been doing that for around 15 years. I didn't want to alter the bits in the movie until years down the road when a worthy enough compression engine came along (in my case that's H.265).
MakeMKV's goal may very well be to create an exact copy of the disc, but if the disc has a scratch or whatever then something's bound to happen. Given the source disc is scratched anyway, I'd much rather have an annoying artifact in the movie (typically a momentary pixelated mess) and have it continue on in sync than have a glitch (or not) but continue on afterwards with an intolerable A/V sync issue afterwards. I don't why MakeMKV does this, but I know it's way better than MKVToolNix which consistently yields even worse A/V sync separations.
HandBrake does manifest a glitch in the compressed MKV output where ever the scratch is, but afterwards the audio and video continue to maintain sync (just like when playing the dvd in a dvd player). If I use MakeMKV to first turn my (corrupted) VIDEO_TS folder into an MKV then I get the A/V sync issue and any further compression by HandBrake simply maintains the sync issue during compression.
So you may be thinking, why not just use HandBrake if you're going to compress anyway. Well, that'd be an option but I see too many video compression artifacts on my 75" TV using HandBrake (despite much tweaking to prevent that and maintain a reasonable file size). I've found a different compression front end for ffmpeg that produces just wonderful results so I want to use that - but it ONLY accepts MKV inputs. My life would be wonderful if ffmpeg was able to take the collection of VOB and IFO files within a VIDEO_TS folder and compress that while it makes an MKV output. Annoyingly, those developers are paranoid about getting sued because (even though my files are already decrypted) they believe operating on a COLLECTION of VOBs and turning it into anything else is breaking the law (even if it's your own home movie taken from a camcorder). They'll happily turn *ONE* VOB into an MKV, but doing that on a sequence of VOBs has them spooked. I know many programs that operate on VOB sequences (HandBrake, MakeMKV, MKVToolNix, and zillions of others) and I've NEVER heard of anybody getting sued as a result. There's always something...
MakeMKV's goal may very well be to create an exact copy of the disc, but if the disc has a scratch or whatever then something's bound to happen. Given the source disc is scratched anyway, I'd much rather have an annoying artifact in the movie (typically a momentary pixelated mess) and have it continue on in sync than have a glitch (or not) but continue on afterwards with an intolerable A/V sync issue afterwards. I don't why MakeMKV does this, but I know it's way better than MKVToolNix which consistently yields even worse A/V sync separations.
HandBrake does manifest a glitch in the compressed MKV output where ever the scratch is, but afterwards the audio and video continue to maintain sync (just like when playing the dvd in a dvd player). If I use MakeMKV to first turn my (corrupted) VIDEO_TS folder into an MKV then I get the A/V sync issue and any further compression by HandBrake simply maintains the sync issue during compression.
So you may be thinking, why not just use HandBrake if you're going to compress anyway. Well, that'd be an option but I see too many video compression artifacts on my 75" TV using HandBrake (despite much tweaking to prevent that and maintain a reasonable file size). I've found a different compression front end for ffmpeg that produces just wonderful results so I want to use that - but it ONLY accepts MKV inputs. My life would be wonderful if ffmpeg was able to take the collection of VOB and IFO files within a VIDEO_TS folder and compress that while it makes an MKV output. Annoyingly, those developers are paranoid about getting sued because (even though my files are already decrypted) they believe operating on a COLLECTION of VOBs and turning it into anything else is breaking the law (even if it's your own home movie taken from a camcorder). They'll happily turn *ONE* VOB into an MKV, but doing that on a sequence of VOBs has them spooked. I know many programs that operate on VOB sequences (HandBrake, MakeMKV, MKVToolNix, and zillions of others) and I've NEVER heard of anybody getting sued as a result. There's always something...
Re: Transcoding VOB to MKV yielding bad sync issues
So Handbrake's output glitches at the same spot. This supports the theory that the file is damaged or improper in some way.
Since MakeMKV generally produces a perfect copy of the disc or nothing, it doesn't surprise me that it doesn't seem to be able to recover from an imperfect or improper VOB structure. There would seem to be little incentive to write code to try to deal with it.
If you have the DVDs handy, you could try ripping them again, perhaps with MakeMKV to see if you can get a clean rip (perhaps after cleaning the disc).
Since MakeMKV generally produces a perfect copy of the disc or nothing, it doesn't surprise me that it doesn't seem to be able to recover from an imperfect or improper VOB structure. There would seem to be little incentive to write code to try to deal with it.
If you have the DVDs handy, you could try ripping them again, perhaps with MakeMKV to see if you can get a clean rip (perhaps after cleaning the disc).