"Batman Begins" Blu-Ray ripping hangs in title 4

[henk84]

Hi

I’m trying to rip my BD of "Batman Begins" with makemkvcon on Debian stable.
I had previously ripped it with HandBrake but had mistakes in my process (I’m really new to the "ripping discs" thing), so I started over, now using MakeMKV for ripping the disc, then using HB to reencode to save space.
The point is: the disc seems to be fine as ripping with HB worked fine.

I started MakeMKV and after several hours noticed that it still wasn’t finished and was stuck in title04 at 64% and overall progress of 2%.
I now started it again with --debug and the same thing happened, see attached logfile.
Most relevant line might be

Code: Select all

001003:0020 DEBUG: Code 0 at ^aoCY_?pF2N[RTPtazEj{e:29393631

Full invocation:

Code: Select all

~/usr/bin/makemkvcon --debug=makemkv.debug.log --progress=-same --cache=1024 --minlength=0 mkv disc:0 all .

(I noticed that the log was written to ~/MakeMKV_log.txt instead of the location I specified but I’m not sure if that’s relevant for this issue. I will open a separate post for that.)

Can anyone help getting this to work?

I’m happy to provide more information if needed, I’m just not sure what’s relevant (as said: I’m new to makemkv, blu-rays, ripping, encoding, matroska, etc.) and I couldn’t find any post detailing which information I should be providing. If that exists, please link me to it or let me know how to find it and I will update this post.

Thanks!

[dcoke22]

The bottom of your log file:

Code: Select all

005014:0000 Saving 51 titles into directory file://.
005001:0308 File ./title_t00.mkv already exist. Do you want to overwrite it?
005001:0308 File ./title_t01.mkv already exist. Do you want to overwrite it?
005001:0308 File ./title_t02.mkv already exist. Do you want to overwrite it?
005001:0308 File ./title_t03.mkv already exist. Do you want to overwrite it?
005001:0308 File ./title_t04.mkv already exist. Do you want to overwrite it?
001003:0020 DEBUG: Code 0 at ^aoCY_?pF2N[RTPtazEj{e:29393631
AV sync issues in ./title_t04.mkv
AV sync in 1 at 0:00:27.127 (927.08s) : encountered overlapping frame, audio skew is +927.08s

Which doesn't really show anything. If the drive returns an error, MakeMKV usually prints that in the log. Then we look at it and tell people, see, the log says the drive couldn't read the disc properly likely because the disc is dirty, even if it seems clean and plays fine in your blu-ray player. Gently clean the disc and try again.

Since your log doesn't have that, all I can suggests is try power cycling the optical drive. Try gently cleaning the disc. Then try to rip again.


As a side note, your log suggests you're trying to rip 51 titles from the disc. There will likely be a lot of stuff that you'll delete afterwards, since much of that is probably FBI warnings, blu-ray menus, trailers for other movies and what not. This methodology is not unreasonable, but it is not the workflow I follow. As a caveat, I'm using the GUI for MakeMKV, not the command line. When I get a new disc, I first make a decrypted backup of the disc. This results in a folder that contains all the contents of the disc with the encryption removed. Then I open the folder with MakeMKV to see all the titles on the 'disc'. From there, I use MPV on the command line to play the relevant #####.m2ts files for the various titles. (Lots of players would work, I just like MPV.) From that I can sort out what's interesting and what isn't, then I rip the relevant titles. This ripping is as fast as my storage will go since it isn't coming from an optical disc.

[henk84]

The bottom of your log file:
…
Which doesn't really show anything. If the drive returns an error, MakeMKV usually prints that in the log. Then we look at it and tell people, see, the log says the drive couldn't read the disc properly likely because the disc is dirty, even if it seems clean and plays fine in your blu-ray player. Gently clean the disc and try again.

Since your log doesn't have that, all I can suggests is try power cycling the optical drive. Try gently cleaning the disc. Then try to rip again. :)

Thanks for your explanation, sadly that does not seem to have helped.
Since Handbrake was able to rip this disc just fine, I’m currently thinking that it must be an issue with MakeMKV (or one of the libs it uses).
OTOH it would be strange if I were the first to try to rip "Batman Begins" with MakeMKV. So it might be the combination of MakeMKV, this "Batman Begins" BD version (german, at least the cover, but the disc itself is probably also specific to some region, I’m guessing), and my disc drive (which is a PIONEER BD-RW BDR-209M)?
Could this be a libredrive issue?
How can I check the firmware of my drive?
Any advice what else to try is appreciated!

As a side note, your log suggests you're trying to rip 51 titles from the disc. There will likely be a lot of stuff that you'll delete afterwards, since much of that is probably FBI warnings, blu-ray menus, trailers for other movies and what not. This methodology is not unreasonable, but it is not the workflow I follow. As a caveat, I'm using the GUI for MakeMKV, not the command line. When I get a new disc, I first make a decrypted backup of the disc. This results in a folder that contains all the contents of the disc with the encryption removed. Then I open the folder with MakeMKV to see all the titles on the 'disc'. From there, I use MPV on the command line to play the relevant #####.m2ts files for the various titles. (Lots of players would work, I just like MPV.) From that I can sort out what's interesting and what isn't, then I rip the relevant titles. This ripping is as fast as my storage will go since it isn't coming from an optical disc.

AFAICT ripping a full BD to MKV only takes up to 30 minutes, but I haven’t stopped the time yet. Is a decrypted backup faster? Does it also filter out identical playlists (as ripping to MKVs seems to do)?
I’m not sure what advantage your method would give over mine, TBH, but, as I said, I’m still a total noob WRT DVD/BD, ripping, encoding, etc., so I’d be happy if you could explain the difference between our approaches!
You are right: there is a lot of stuff (for some BDs, for others there is almost nothing) that I have to wade through but isn’t that the case for both methods?

Thank you very much, your input is highly appreciated! (:

[dcoke22]

A feature of blu-ray is something called seamless branching. This is a feature where a player can switch from one MPEG-2 transport stream to another without the viewer noticing. These different streams are called segments. Those segments get assembled in a playlist that's stored in a .mpls file on the disc. Generally speaking, when you tell your blu-ray player to play a title, it starts playing the playlist for that title. In the simplest case, that playlist has a single segment that contains the whole title. But it might contain multiple segments that make up the title. This allows for multiple versions of a movie to be contained on a single disc.

For example, in North America, Pixar movies often have three versions of the movie on the disc. The difference between them is usually the opening and closing credits. One version has the credits in English, one in French, and one in Spanish. Movies with both the theatrical release and director's cut of the movie use the same technique.

There are some discs, usually from the studio Lionsgate, that use something called playlist obfuscation in an attempt to make the movie harder to rip. They arbitrarily chop the movie into a couple dozen segments and then make hundreds of playlists with all but one out of order or wrong in some way. Knives Out (2019) is one example. So, if you blindly rip every title on the disc, you'll get hundreds of 26GB+ .mkv files of Knives Out with no clue as to which one is right. (Just about every one of these discs has a thread about it on this forum; the Knives Out thread. See also What is the correct play list [insert title here]?)

Making a decrypted backup of a disc replicates the structure of the disc into a folder on your hard drive with the decryption removed. It doesn't turn individual titles on the disc into .mkv files. I choose which titles to turn into .mkv files in the second step of my workflow. It isn't necessarily any faster in most cases (although blindly ripping all titles on a disc like Knives Out will take many hours an terabytes of hard drive space).

[henk84]

Making a decrypted backup of a disc replicates the structure of the disc into a folder on your hard drive with the decryption removed. It doesn't turn individual titles on the disc into .mkv files. I choose which titles to turn into .mkv files in the second step of my workflow. It isn't necessarily any faster in most cases (although blindly ripping all titles on a disc like Knives Out will take many hours an terabytes of hard drive space).

Aaah, that makes sense! I had stumbled over the method you mention with one of my early ripping attempts: some Disney DVD seemed to do the same thing, there were like 80 titles on the disc of which >30 were the actual movie but with the chapters in different orders and only one was correct. Very annoying but good to know that with your technique at least the excessive disk space use can be avoided.
Thanks for elaborating!

And if you have any advice what to do about my ripping issue with "Batman Begins", whom/where else to ask, what to look at or try, that’d be highly appreciated (:
ATM my only option is using handbrake for ripping this disk. And I’d like to avoid that, if only for the reason that I don’t think that HB can _only_ rip, without encoding, and thus takes a really long time …

[dcoke22]

Do you have a different drive you can try? It has been my experience that some discs just don't work in some drives. I have three different drives I use with MakeMKV. When a disc doesn't work in my 'main' drive, I try it in my other drives. Often it just works in one of the other drives and I don't have to bother with trying to clean it.

In your case, do you know if title_t04 is even interesting to you? What about skipping that title? If there's a spot on the disc that your drive is having trouble with, maybe you'll get lucky and it'll be contained in parts of the disc that are uninteresting to you.

[henk84]

When I get a new disc, I first make a decrypted backup of the disc. This results in a folder that contains all the contents of the disc with the encryption removed. Then I open the folder with MakeMKV to see all the titles on the 'disc'.

Code: Select all

makemkvcon --progress=-same --cache=1024 --minlength=0 backup --decrypt disc:0 backup

I tried that now and 'backup' was actually able to backup the disc!
So in the next step I then tried to use makemkv to convert that backup to mkvs:

Code: Select all

makemkvcon --progress=-same --debug --cache=1024 --minlength=0 mkv file:backup all mkvs

but now _this_ command seemed to get stuck for hours.
strace (without any options other than -p to attach to the makemkvcon process) shows only nanosleeps, no other activity, but I have no idea whether it should show anything else.

So I think a problem with the BD drive can now be eliminated as the cause of the issue, right?

From there, I use MPV on the command line to play the relevant #####.m2ts files for the various titles. (Lots of players would work, I just like MPV.) From that I can sort out what's interesting and what isn't, then I rip the relevant titles. This ripping is as fast as my storage will go since it isn't coming from an optical disc.

I tried that but the quality seemed worse than playing the mkvs I had previously created, it looked kinda bad, possibly "telecined"? I’m not so sure about these terms … Every second line seems to be "delayed" or so when there is movement. Do you know anything about that?

Also: How do you get from a m2ts file you want to the playlist/title number you need to extract?
And how do you extract those titles then? With 'makemkvcon mkv'?

Do you have a different drive you can try? It has been my experience that some discs just don't work in some drives.

No, I’m not in the habit of having hardware lying around (; I guess I could try the drive that’s in my (currently defunct) desktop. It might be a 'Lite-On iHES112' but I can’t currently tell.
But the behaviour described above seems to indicate that the problem is not with the drive anyway, doesn’t it?

Right now I’ll just leave makemkv running on the 'backup' folder. It seems to be doing _something_ as 'htop' shows it using 100% CPU (of one of the two cores) even without it showing any progress.
I’ll report back when it’s finished or in 1-2 days, whichever comes first :D

In your case, do you know if title_t04 is even interesting to you?

I think I figured that out: it’s moderately interesting. I can easily live without it but since the issue seems to be with makemkv and could possibly reoccur with titles I really want to have (until it’s clear what’s causing this behaviour, I currently don’t want to just ignore the issue and move on.
But as I said: I’ll leave it running for 1-2 days and then report back.

[dcoke22]

I tried that now and 'backup' was actually able to backup the disc!

I agree, if MakeMKV was able to successfully make a backup of the disc, then your optical drive is fine.

From there, I use MPV on the command line to play the relevant #####.m2ts files for the various titles. (Lots of players would work, I just like MPV.) From that I can sort out what's interesting and what isn't, then I rip the relevant titles. This ripping is as fast as my storage will go since it isn't coming from an optical disc.

I tried that but the quality seemed worse than playing the mkvs I had previously created, it looked kinda bad, possibly "telecined"? I’m not so sure about these terms … Every second line seems to be "delayed" or so when there is movement. Do you know anything about that?

Batman Begins (2005) The review of the Batman Begins blu-ray has a section about the special features of the disc and indicates that some of those items are in standard definition. Lots of standard definition stuff is interlaced and if it came from film, is telecined. (I always have to lookup the differences.) If you use Plex or whatever to play the file on your TV, it will likely look better than on your computer monitor or if you post process the files in Handbrake or FFMPEG or whatever, there's almost certainly a 'deinterlace' option.

[dcoke22]

As for going from .m2ts file to playlist… I'm not sure. When I use the GUI, if I highlight a title, in a box on the right, there's a bunch of details about that title.

window.jpg (30.1 KiB) Viewed 9549 times

Those details contain the relationship between .m2ts file and .mpls playlist.

info.jpg (38.86 KiB) Viewed 9549 times

In this example, the playlist 00001.mpls (which lives in the BackupFolder/BDMV/PLAYLIST/ folder) has a single segment in it named 00018.m2ts (which lives in the BackupFolder/BDMV/STREAM folder). So, I'd just file up MPV and point it at 00018.m2ts and verify what that title is (which is obvious in this case). Unfortunately, the .mpls files are not vanilla text files, so you can't just look them to figure out the mapping between title and segment map.

Obviously MakeMKV can interpret the .mpls files. MediaInfo can read a .mpls but it doesn't always seem to tell you the name of the .m2ts files in the playlist (although sometimes it does). I've never had much of a reason to get at it from the command line, so I don't have a good answer.

[henk84]

So, since makemkv again managed to make some progress in the last hour or so, I thought I should post an update:

• I have made a "backup" of the disc, which took a reasonable amount of time (around 1h15m)
• I have then started converting all titles from this backup to mkvs using the following command

  Code: Select all

  makemkvcon --progress=-same --debug --cache=1024 --minlength=0 mkv file:backup all mkvs
  

This command was started 2 days ago and is making progress _very_ slowly, as can be seen when looking at the timestamps of the resulting files:

Code: Select all

% ls -l mkvs total 5437176
-rw-rw-rw- 1 henk multimedia  102147777 19. Okt 10:09 title_t00.mkv
-rw-rw-rw- 1 henk multimedia 1170097496 19. Okt 10:09 title_t01.mkv
-rw-rw-rw- 1 henk multimedia   95785403 19. Okt 23:55 title_t02.mkv
-rw-rw-rw- 1 henk multimedia 1215290770 19. Okt 23:55 title_t03.mkv
-rw-rw-rw- 1 henk multimedia  262780142 19. Okt 23:55 title_t04.mkv
-rw-rw-rw- 1 henk multimedia 1219597472 19. Okt 23:56 title_t05.mkv
-rw-rw-rw- 1 henk multimedia   77039005 21. Okt 13:48 title_t06.mkv
-rw-rw-rw- 1 henk multimedia 1174500982 21. Okt 13:48 title_t07.mkv
-rw-rw-rw- 1 henk multimedia  143229101 21. Okt 13:49 title_t08.mkv
-rw-rw-rw- 1 henk multimedia   73602114 21. Okt 13:49 title_t09.mkv
-rw-rw-rw- 1 henk multimedia   33554432 21. Okt 13:49 title_t10.mkv

Started two days ago, the first two titles were ripped relatively quickly, then makemkv needed more than 12h to be done with the next one (t02).
The next three titles (03-05) were quick again.
06 was _really_ slow, even though it’s rather small: 36 hours for a 74MB title.
When I checked it was using 100% of one CPU.

That does not seem normal.
What could I try?
What could I check?
What other info can I provide to help get this issue fixed? The backup of the blu-ray, so someone can test whether they see the same behaviour, maybe?
Thank you!

[henk84]

Batman Begins (2005) The review of the Batman Begins blu-ray has a section about the special features of the disc and indicates that some of those items are in standard definition. Lots of standard definition stuff is interlaced and if it came from film, is telecined. (I always have to lookup the differences.) If you use Plex or whatever to play the file on your TV, it will likely look better than on your computer monitor or if you post process the files in Handbrake or FFMPEG or whatever, there's almost certainly a 'deinterlace' option.
[/quote]

Ah, yes, interlaced is the term I was looking for, thanks!

In this example, the playlist 00001.mpls (which lives in the BackupFolder/BDMV/PLAYLIST/ folder) has a single segment in it named 00018.m2ts (which lives in the BackupFolder/BDMV/STREAM folder). So, I'd just file up MPV and point it at 00018.m2ts and verify what that title is (which is obvious in this case). Unfortunately, the .mpls files are not vanilla text files, so you can't just look them to figure out the mapping between title and segment map.

Obviously MakeMKV can interpret the .mpls files. MediaInfo can read a .mpls but it doesn't always seem to tell you the name of the .m2ts files in the playlist (although sometimes it does). I've never had much of a reason to get at it from the command line, so I don't have a good answer.

Getting from the segment.m2ts file to the playlist.mpls seems to be impossible. Other than scanning all the playlist files and making an index. Which I assume is what MakeMKV GUI is doing.
I have not tested a lot but MediaInfo seems to show playlist contents most times. I’ll see how that goes in the future. ATM I don’t need that method anyway, since ripping BDs to MKVs directly mostly works fine. Except for the current case "Batman Begins" (but the ripping never ends): it’s still running :D
I’ll probably find out with the next BD that’s giving me similar troubles.
Thanks so much for all your input!

[dcoke22]

Wow, 3 days and counting to make .mkv files from a single blu-ray has got to be some kind of record.

Nothing comes to mind for a reason why making a .mkv file from a decrypted backup would be going so slowly. When I do this on my systems, I'm almost always limited to disk I/O speeds. Maybe there's a weird thing going on with a nearly full filesystem or a hard drive that's about to crash? Is the source or target filesystem isn't on the network somewhere? A SMB share (or whatever) would add another variable into the mix. Is this running on a RAM constrained system and maybe the swap is killing things?

I suppose an even more fundamental question is does this only happen with this blu-ray or does it happen with others?

[dcoke22]

I just made a .mkv from a decrypted backup (using the GUI). makemkvcon used 125% of my 4-core Intel CPU. The kernel kicked in for another 50%, undoubtedly dealing with all the io requests for the 450M/s writes MakeMKV was making to my SSD.

For fun, I also ripped using the command line:

Code: Select all

/Applications/MakeMKV.app/Contents/MacOS/makemkvcon mkv file:/path/to/BACKUP all .

It went just as fast as the GUI, which is what I'd expect.

https://www.makemkv.com/developers/usage.txt

[henk84]

Wow, 3 days and counting to make .mkv files from a single blu-ray has got to be some kind of record.

Yay? (:

Maybe there's a weird thing going on with a nearly full filesystem or a hard drive that's about to crash? Is the source or target filesystem isn't on the network somewhere?

I doubt it, filesystem:

Code: Select all

/dev/mapper/nastix--data-media    1.6T  1.4T  198G  88% /media/multimedia

i.e. local, not full.

The disc is HDD, not SSD. But that can’t be the sole reason, because other BDs are usually done in less than an hour (with the 'mkv' command) and making a 'backup' (i.e. 'backup' command, not 'mkv' command) of this "Batman Begins" BD also finished in a very reasonable time of less than an hour IIRC. Only making MKVs of it is slow, no matter whether from a backup or directly from the disc.

A SMB share (or whatever) would add another variable into the mix. Is this running on a RAM constrained system and maybe the swap is killing things?

RAM, while this makemkvcon is still running and HandBrake is encoding something else:

Code: Select all

% free -m
               total        used        free      shared  buff/cache   available
Mem:            7773        2035         149          33        5587        5480
Swap:           7628         473        7155

I suppose an even more fundamental question is does this only happen with this blu-ray or does it happen with others?

It only happens with this one (so far). While this has been running I have been ripping (with makemkv and exactly the same commands, same drive, etc. "The Dark Knight" and "The Dark Knight Rises", i.e. the other two movies from the same trilogy, and 2 discs for each of them, and transcoding them with HandBrake ("Rises" is still running currently).
Before that I have been ripping 5 other movies where everything went well.

Current status: it has finished 13 tracks, currently stuck on the 14th at 60%, overall progress 10% …

Code: Select all

% ls -hl mkvs
total 6.1G
-rw-rw-rw- 1 henk multimedia  98M 19. Okt 10:09 title_t00.mkv
-rw-rw-rw- 1 henk multimedia 1.1G 19. Okt 10:09 title_t01.mkv
-rw-rw-rw- 1 henk multimedia  92M 19. Okt 23:55 title_t02.mkv
-rw-rw-rw- 1 henk multimedia 1.2G 19. Okt 23:55 title_t03.mkv
-rw-rw-rw- 1 henk multimedia 251M 19. Okt 23:55 title_t04.mkv
-rw-rw-rw- 1 henk multimedia 1.2G 19. Okt 23:56 title_t05.mkv
-rw-rw-rw- 1 henk multimedia  74M 21. Okt 13:48 title_t06.mkv
-rw-rw-rw- 1 henk multimedia 1.1G 21. Okt 13:48 title_t07.mkv
-rw-rw-rw- 1 henk multimedia 137M 21. Okt 13:49 title_t08.mkv
-rw-rw-rw- 1 henk multimedia  71M 21. Okt 13:49 title_t09.mkv
-rw-rw-rw- 1 henk multimedia  67M 22. Okt 04:16 title_t10.mkv
-rw-rw-rw- 1 henk multimedia 717M 22. Okt 04:16 title_t11.mkv
-rw-rw-rw- 1 henk multimedia  84M 22. Okt 04:18 title_t12.mkv
-rw-rw-rw- 1 henk multimedia  64M 22. Okt 04:18 title_t13.mkv

[henk84]

It has finished!
Entry from my (extended) shell history says that I started it on 2022-10-19 08:33 and it took 7717:24 (minutes:seconds) which is 5.35 days until it finished successfully.

Any suggestions what to do now?
If anyone is willing to test running makemkv on the backup files I created to see if it behaves the same for them (and hopefully even debug this, if it’s the case), please send me a message and I will provide a download link.
If anyone knows anyone who can likely help, please point their attention to this thread (:

Thanks!