Question to all: MakeMKV without GUI, console only

[NullNix]

I've written my own scripting language, specifically tailored to transcoding discs into mkv files. My automation goals seem different than most. Most people seem interested in quickly and easily extracting or transcoding Blu-rays or DVDs into media files. I'm more interested in repeatability. I realized long ago that there is a lot of research that goes into processing discs (determining title numbers, tracks, forced subtitles, etc.). I wanted to preserve this knowledge, so reprocessing discs in the future (e.g., errors, newer formats, change-of-mind, etc.) would be easy.

Ditto (mentioned now and again here before), though transcoding into media files is also crucial for me (I have a projector/rasbpi-kodi setup that is only wifi-connected, so a lot of artificial-noise-filled blurays *need* transcoding and often vapoursynthing before I can squeeze them into the available bandwidth: for simplicity I simply transcode everything). I'm not sure I can call what I wrote a scripting language so much as a transcoding script which takes a bunch of options and catalogue files that let you vary options on a track-by-track basis without having to repeat yourself too much. The script *can* take a directory-full of MKV files named as makemkv names them and work from those, but can also call makemkvcon to extract individual tracks or sets of tracks -- it would be nice to be able to say "give me tracks 2, 5, and 9" and get them out named with _02, _05 and _09 in the names without pyaing the substantial makemkvcon initialization cost more than once. (The "default mode" of my transcoding script just uses direct handbrake CLI invocations, but it *can* use makemkv, and for some workflows, e.g. when throwing things through vapoursynth, this is mandatory because I need the video stream before it is retranscoded by handbrake.)

My main wishes for MakeMKV are:

I agree with all of these (and though this doesn't cover my biggest single annoyance, that is unfixable: handbrake and makemkv simply use different track numbering schemes, and that's all there is to it). I might note that the makemkv drive number thing is a bit silly. I should not have to do this:

makemkvcon -r info | grep '^DRV' | sed 's,^DRV:,,; s,",,g' | awk -F, '{ if ($7 != "") { print $7 " " $1;} }'

to map a device file to a drive number! (Among other things this hangs if a drive is in exclusive use by something else, and takes forever with massive numbers of slow seeks if another makemkvcon instance is extracting from any drive, even if it's not the drive I was planning to extract from). Just take device file names on the command line like everyone else. Rolling one's own device naming scheme was annoyingly alien with cdrecord and is just the same here.

But really what I'd *expect* from experience with Unix tools is that anything you can do from the gui is also doable from the command-line tool, including things like picking specific things to extract within a given track, and that the gui uses the public options to the command-line tool to do what it does, so the options never rust and are automatically tested. makemkvcon is clearly capable of doing everything makemkv can since makemkv uses it, but it looks like it only knows how to do most of what it can do via the somewhat-opaque shared-memory protocol between makemkvcon and makemkv. If I was writing this I'd redo that protocol as a serialized command language rather than a bunch of shared memory stuff (maybe as simple as throwing some sort of instruction sheet at makemkvcon in JSON over stdin or something), then publicize that protocol so I didn't have to come up with a pile of horrible makemkvcon command-line options for obscure things like turning on track 4's second Spanish DTS track or whatever, and didn't have to worry about maintaining a pile of stuff that makemkv-the-GUI wasn't even going to use. But of course I'm not writing this so there's probably a very good reason why it's not done that way.

Oh, and a simple one: --help output which actually lists the valid command-line options. Half of them seem to be documented only in random forum posts, and since makemkvcon is closed-source I can't look and see what it can actually do...

[tstraus13]

Hello and Thanks for the Great application,

I use the makemkvcon for a project I am working on to do automated ripping on a standalone box with multiple disc drives. I love that MakeMKV can pretty much rip all relevant video based discs (DVD, BluRay, 4K BluRay).

One thing I would like to see is a way to generate the config file without having to open the GUI application. This is especially useful for setting the drive speed because the drives are added to the config when using the GUI. Maybe another solution would be to add the drive speed as a command switch.

As of now thats all I can really think of currently, but as I continue to develop my app/project I will add things I find. Thanks again!

[cpitchford]

What do I want as a user

I run MakeMKV on a remote server

And beyond this, as a user, I don't want to interact with or see MakeMKV at all!!

In fact, I have never used the GUI for MakeMKV. I'm entirely on-board with the command line access and I've built some some-what shameful but (for me) extremely functional scripts over the 6 years

My user experience for my set up is really supremely simple:

1. Wait for movie to arrive in the post
2. Unbox disc
3. Insert into server
4. Wait for drive to eject disc
5. Box disc for safe storage
6. Wait some hours for a notification from Plex to let me know there is a new movie added.

For me, that's the ultimate end user experience. The process of adding a new movie to my collection doesn't see me touch a command line, keyboard, or even a mouse.

What is my hardware setup

I have two headless servers at home:

1. My "ripper/transcoder" a HP Microserver Gen8 with a LU40N optical drive and the fastest CPU Xeon CPU the box will take.
2. An off the shell NAS on which I run Plex media server (this is where I store my music, movie and TV show rips)

I can't really do the transcoding/ripping on my NAS because its not fast enough. Good at being a NAS.

How do I do all this?

I started writing scripts back around MakeMKV 1.8. They whenever a disc is inserted into my server and deal with CD DVD BR and UHD (still working with this and want to support automating sending of unrecognised dumps)

1. React to a disc being inserted (udev rules)
2. Detect the disc type (CD / DVD / BR and now UHDBR)
3. Call the ripper to copy the content locally (MakeMKV for video)
4. Eject the disc
5. Call the transcoder (HandbrakeCLI for video)
6. Call the tagger (for audio)
7. Call the uploader to put the files into the appropriate place (via SFTP to my NAS running Plex)

The entire process runs in queues. I can rip multiple discs one after another and they will wait in a queue to transcode using Handbrake. I might take about an hour to rip 4 discs, but the transcoding won't finish for just over a day (my server is slow by today's standards)

How do I use MakeMKVcon

* Working with foreign language movies; I like the primary audio to be the original language but the subtitles to be English. I pull information from MakeMKV to alter the selection criteria

* Detecting and stopping when facing fake playlists. I detect if chunks are being re-used in multiple titles in differing orders. If the hit rate is too high, it will stop ripping (Then I use Windows + Process Explorer to find the right MPLS and use a link to set that back in the system for when the disc is re-inserted)

* I use disc information to chose the transcoding profile. Mainly for 4:3 versus 16:9 DVDs but I use it now with strange titles (like The Lighthouse which is in black & white)

* I use title information with Epguides to scrap episode names/numbers. The tagging part of my scripts ultimately chooses the file name 'Series - S01E01 Episode Name.mkv' is generated from

Code: Select all

"%{series_title}s - S%{season_num}02dE%{episode_num}02d %{episode_title}s.mkv"

What do I need in the future?

Not sure really because it works so well at the moment.

* Better understanding of the terms in the settings.conf; what they mean and how to use them (like app_DefaultSelectionString)
* Some clarity of the command line options that don't always appear in the "man page" like setting ripping speeds
* Some clarity over some of the awesome features (like the fake playlist work around and how recognise it has found a title)
* I'd love to be able to specify an MPLS file name instead of a title number (since the title number will change if the min-length is altered)
* I'd like to be able to specify a different settings file so I can build a custom settings file for a specific disc (I need to change the selection criteria some times and to do this I edit the settings.conf file which is not great)

To be honest for 99.9% of what I do, this setup is practically perfect. My wife can use it with confidence and has zero knowledge of what happens in the background.. put in disc, wait, watch on FireTV.

Having NOT seen the GUI, I feel I might be missing out!

[kevmitch]

I would love to see more functionality for makemkvcon, or even just documentation for the existing functionality like operating as a TCP server.

The basic problem with using the GUI is that for each disk it requires several steps:

0) insert disc
1) open makemkv
2) scan the disc (wait a minute or two)
3) look at all the titles choose which ones I want
4) initiate the ripping (wait 15-40 minutes for the rip to complete)
5) eject the disc

#2 is particularly problematic because the scan time is too short to go off and do something else, but long enough to be annoying.

With makemkvcon, there is only

0) insert disc
1) run makemkvcon to rip the whole disc
2) eject the disk

With a little scripting and a udev rule, this could even be brought down to

0) insert the disc, let udev run makemkvcon, and then automatically eject on completion

This makes a world of difference particularly when ripping a TV series with multiple discs. The GUI requires a shift of focus and a minute long wait time followed by some concentration on title selection, whereas for running a single command on the console, I don't even have to pause whatever I'm already watching.

The only downside to makemkvcon on its own is that it doesn't allow or #3 in the GUI workflow: choosing titles. However, most of time I find myself manually executing an algorithm to select all titles necessary to rip all content on the disk only once (or at least minimize duplication). This is a problem better solved by a computer anyway, so I have created a front-end script to makmkvcon to parse the disc contents and choose the optimal set of titles. This gives me the best of both worlds: single step ripping and optimal title selection. I only need to use the GUI for pathological discs.

The main problem is that given the TCP mode is undocumented and I haven't yet gone to the trouble of trying to reverse engineer it, makmkvcon must be run multiple times for each title selected and each time it must rescan the whole disc making the whole process take longer. But at least this is time spent by the computer and not me.

[drxenos]

I'm not sure I can call what I wrote a scripting language so much as a transcoding script which takes a bunch of options and catalogue files that let you vary options on a track-by-track basis without having to repeat yourself too much.

Mine is an actual interpreted language, I write scripts in. Each disc or disc set has it's own script. It allows me to streamline a lot of the drudgery of processing a disc (which preserving the necessary steps). I like special features, and they are sometimes a bear to deal it.

Running the script generates a queue file. I have an GUI tool that handles the queue, so I can more easily adjust the order of tasks, make quick fixes without rerunning the scripts, pause the processing, change cpu priority and processor affinity, set the number of parallel transcoding processes, etc.

I agree with all of these (and though this doesn't cover my biggest single annoyance, that is unfixable: handbrake and makemkv simply use different track numbering schemes, and that's all there is to it).

That's what I was trying to convey with most of my points (obviously, not very well). Every tool seems to want to do their own numbering. My language allows you to use whatever numbering is most convenient (mpls, m2ts, HandBrake's, MakeMkv's), and automatically converts when it needs to run a particular tools.

It would be awesome if they all just allowed you use the mpls.

DrX

[aidenn0]

I just recently started using makemkvcon to streamline my setup. The one improvement I would like to see is have something that extracts only all of the titles that would be selected by default in the GUI. Specifically what I've seen is if there are e.g. 3 titles that are different angles of the same TS, the GUI will select just one of them to extract, but if I do "makemkvcon mkv disc:0 all" then all of the angles will be ripped. maybe there should be a "makemkvcon mkv disc:0 almostall"

I will also note that I only use makemkvcon for blu-rays because too many DVDs have dozens of fake titles, totaling many hundreds of GB if all are extracted to mkv so manual intervention is typically required to find the correct title(s) to extract.

[keithhelms]

I use makemkvcon in a set of scripts that allow me to specify which titles to extract, and for each of those which audio tracks and which subtitle tracks. My scripts extract the title and then pass it to either handbrakecli or ffmpeg to transcode to x264 or x265. I write a control file for each disk specifying what to do with it.

The GUI is fine for backing up a disc or titles, but it doesn't have any "do this after extract" features (that I'm aware of). If I decide I want to change something and rerun, I can just modify my control file and rerun the script. For example, I might change the x265 CRF setting from 24 to 20 or switch the encoder from x264 to x265. With the scripts doing batch processing I don't have to go back into the GUI and respecify what titles to extract.

[drxenos]

I use makemkvcon in a set of scripts that allow me to specify which titles to extract, and for each of those which audio tracks and which subtitle tracks. My scripts extract the title and then pass it to either handbrakecli or ffmpeg to transcode to x264 or x265. I write a control file for each disk specifying what to do with it.

The GUI is fine for backing up a disc or titles, but it doesn't have any "do this after extract" features (that I'm aware of). If I decide I want to change something and rerun, I can just modify my control file and rerun the script. For example, I might change the x265 CRF setting from 24 to 20 or switch the encoder from x264 to x265. With the scripts doing batch processing I don't have to go back into the GUI and respecify what titles to extract.

There aren't many things more beautiful than a good script!

[elguapo]

I use Linux as my primary desktop, I prefer to not use the GUI. I did not even compile support for it. I do not use it because I can script makemkvcon so I can process my discs faster

[Zirias]

Please give me some feedback if you are using MakeMKV without GUI, only the console app. This applies to any OS, but most of you do this on Linux, so that's why the question is posted here.

Hi Mike,
just so it isn't forgotten: On FreeBSD, GUI just doesn't work, because 'makemkvcon' running in Linux binary compatibility environment fails with the 'guiserver' option.

[AlphaScoliosis]

Hello Mike,
thank you for this awesome software!

Just like many others, I have automated the ripping process.
I wrote an application that waits for a disc to be inserted, and then begins the ripping process.
This application contains a lua script-engine whose purpose is to implement the heuristics for track and stream selection.
It is based on rules. Each rule is a lua-file which has these basic functionalities:

• Use the discinfo (makemkvcon info) to determine whether the rule matches for this disc (e.g. heuristics to determine whether it's a movie or a series disc)
• If the rule matched: Select the tracks to rip.
• After each track has been ripped, select the streams in the corresponding track I want to keep, their order, as well as their default / forced / commentary flags.

The process works like this:

1. Read disc information using

   Code: Select all

   makemkvcon info dev:/dev/<devnode> --robot --progress=-stdout --messages=-stdout --debug

2. Send discinfo to rule engine, so it can find a matching rule
3. Send discinfo to the matching rule, so it can determine the list of tracks it thinks should be ripped
4. For all selected tracks:
   1. Rip n'th track using

      Code: Select all

      makemkvcon mkv dev:/dev/<devnode> <n> /path/to/output --robot --progress=-stdout --messages=-stdout --noscan 

   2. Parse the output of makemkvcon during the process. What's very important about this are the lines:

      Code: Select all

      subtitles track # turned out to be empty and was removed from output file

   3. Modify the discinfo parsed from the initial info run for the current track (removing the streams makemkv removed by itself). This is very important, so I can make sure the discinfo I have fits what was written to the resulting mkv file.
   4. Pass the modified discinfo to the matched rule via the rule engine, and ask it to give me an ordered list of streams to select, as well as the corresponding default / forced / commentary flags
   5. Generate mkvmerge commandline from it, and run that to generate a second (filtered) mkv from the original file ripped by makemkv

This process works quite well and is very dynamic in its possibilities, because the lua-based rules have all of the information about the disc available (language, duration, boolean flags such as author commentary, visual_impaired, etc. etc.).
Unfortunately though, since makemkvcon 1.16, something seems to have changed. I don't seem to get the "turned out to be empty and was removed from output file" messages anymore, which breaks my complete setup.

[OrdinaryCarrot16]

Hey, Mike -- love makemkv.

The box I rip on doesn't even have a keyboard/monitor connected to it normally. I run makemkvcon after connecting to the ripper via ssh, in a screen or tmux session that I can disconnect from while rips are ongoing.

I have a request, actually, and while I don't want to assume anything about how simple or complicated my request is, I don't think it's a huge thing. I'd love your feedback.

Often when ripping TV series, a disc will have a title for each episode, AND a title that it all episodes together. This is an excellent example of a use case that would benefit from what I'm about to request.

Currently, the command-line syntax I use is something like this:
makemkvcon mkv dev:$DEVICE $TITLE /output_dir

For $TITLE, a single number is allowed, or the word "all". I'd really like to be able to pass something like "1-5,8". What I need to do now is either pass "all" and rip a bunch of material I don't want or need, or run the command multiple times -- once for each title I want. Running multiple times is what really becomes inconvenient, especially when the disc has certain types of copy protection schemes ("Complex multiplex encountered").

What do you think? Is it a big ask for makemkvcon to be able to parse a title list and rip multiple titles without completely rescanning the structure every time?

Thanks for your time and your wonderful tool!

Rob

I'm wondering about this too, I am making some FFmpeg + MakeMKV conversion scripts that will automatically pick the titles with actual content, but I can only select one title, or all of them at a time currently (unless there is an undocumented feature, or I'm using it wrong).

[dball202]

It would be great if you could allow makemkvcon64.exe to extract a group of titles at once rather than 1 or ALL. Possibly allow support for a list of title numbers separated by commas ex: 1,2,3,4,5. In the case where I want to grab 10 titles I currently do it one at a time and incur the info scan overhead each time. Allowing multiple titles to be extracted at once would minimize the scan to a single time and speed up the process. Thanks for considering!

[musictrivianut]

Literally just started using the console version yesterday and just found this thread during a search for something.

I had been using the GUI for years (Ubuntu, at least 4.5 and purchased it a couple of years ago to support it), but recently got a second external DVD drive to go with the Bluray drive I already had, so that I could rip CDs at the same time I was making backups of various movies I own.

I took a break on the CDs and was wondering if I could use MakeMKV with both drives at the same time. Could not figure out how to start two copies of the GUI and came across a reference to the command line tool (unknown to me until yesterday). The documentation I found was extremely helpful and I am now sitting here, less than 24 hours later, making a backup of a Bluray and a DVD in two different external drives at the same time using makemkvcon. Even managed to figure out the track selection to match what I had in the GUI and created aliases for both in my .zshrc file. Pop a disc into the appropriate drive, run the alias, wait for a notification (noti). Easy peasy.

Thank you again for this fantastic bit of software! Really makes putting things on my media server *so* much easier.

[MickM]

I have thousands of VOB movies and TV episodes to convert to MKV and the only effective way is to do this in a script, hence the need for makemkvcon. Here's my wishlist:

1. I only care about forced subtitles in English e.g. when two Russians are talking to each other in an English spy movie. However, there is simply no good way to determine up front if a movie or TC episode has these in it. If not, I remove all subtitles, but if it does then I'll keep just the English forced subtitles. HandBrake can scan for a very sparse subtitle track, can MakeMKV do something similar? Presently there's simply no way to tell (aside from watching the movie first).
2. The forced selection rule doesn't appear to work - it always appears to evaluate to 0.