LINUX - ffmpeg - 4K HDR HEVC X265 - Plex

[plittlefield]

Hello All,

As I have benefited so much from MakeMKV and learned so much from this forum, I thought I would post my notes on creating 4K HDR movie files for Plex.

This is still a "work in progress" but believe I have the correct format of MKV for my Samsung TV so that I can experience 4K HDR from my Plex Media Server.

Here is the important hardware:-

* ASUS BW-16D1HT with v3.01 firmware and LibreDrive (4K UHD Friendly - thanks to user asmcom
* Samsung UE49KU6400 49" 4K Ultra HD Smart HDR LED TV
* Roku Streaming Stick+

Here is the important software:-
* MakeMKV
* Ffmpeg
* X265

Here is the possible winning command line:-

Code: Select all

ffmpeg -y -i "massive_4k_input_file.mkv" -t 00:10:00 -analyzeduration 100M -probesize 100M -max_muxing_queue_size 9999 -map 0:v:0 -map 0:a:1 -pix_fmt yuv420p10le -vcodec libx265 -preset medium -x265-params "keyint=60:bframes=3:vbv-bufsize=75000:vbv-maxrate=75000:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display='G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500)'" -acodec ac3 -ab 384k -ac 6 -map_metadata -1 -metadata title="Movie Name (2018)" -metadata:s:a:0 language=eng -map_chapters -1 -sn "Movie Name (2018) 4K ffmpeg.mkv"

Here is the mediainfo output on the file:-

Code: Select all

Format                                   : Matroska
Format version                           : Version 4
File size                                : 213 MiB
Duration                                 : 10 min 0 s
Overall bit rate                         : 2 973 kb/s
Writing application                      : Lavf58.27.100
Writing library                          : Lavf58.27.100
ErrorDetectionType                       : Per level 1

Video
ID                                       : 1
Format                                   : HEVC
Format/Info                              : High Efficiency Video Coding
Commercial name                          : HDR10
Format profile                           : Main 10@L5@High
Codec ID                                 : V_MPEGH/ISO/HEVC
Duration                                 : 10 min 0 s
Bit rate                                 : 2 530 kb/s
Width                                    : 3 840 pixels
Height                                   : 2 160 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Variable
Original frame rate                      : 23.976 FPS
Color space                              : YUV
Chroma subsampling                       : 4:2:0
Bit depth                                : 10 bits
Stream size                              : 181 MiB (85%)
Writing library                          : x265 3.0+28-gbc05b8a:[Linux][GCC 5.4.0][64 bit] 10bit
Encoding settings                        : cpuid=1064959 / frame-threads=2 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=6 / keyint=60 / gop-lookahead=0 / bframes=3 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / radl=0 / no-splice / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=crf / crf=26.0 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=75000 / vbv-bufsize=75000 / vbv-init=0.9 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=2 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=1 / overscan=0 / videoformat=5 / range=0 / colorprim=9 / transfer=16 / colormatrix=9 / chromaloc=0 / display-window=0 / master-display=G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500) / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / refine-ctu-distortion=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-analysis-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei / no-hevc-aq / no-svt / qp-adaptation-range=1.00
Default                                  : Yes
Forced                                   : No
Color range                              : Limited
Color primaries                          : BT.2020
Transfer characteristics                 : PQ
Matrix coefficients                      : BT.2020 non-constant
Mastering display color primaries        : Display P3
Mastering display luminance              : min: 0.0500 cd/m2, max: 1000 cd/m2

Audio
ID                                       : 2
Format                                   : AC-3
Format/Info                              : Audio Coding 3
Commercial name                          : Dolby Digital
Codec ID                                 : A_AC3
Duration                                 : 10 min 0 s
Bit rate mode                            : Constant
Bit rate                                 : 384 kb/s
Channel(s)                               : 6 channels
Channel layout                           : L R C LFE Ls Rs
Sampling rate                            : 48.0 kHz
Frame rate                               : 31.250 FPS (1536 SPF)
Bit depth                                : 16 bits
Compression mode                         : Lossy
Stream size                              : 27.5 MiB (13%)
Writing library                          : Lavc58.48.101 ac3
Language                                 : English
Service kind                             : Complete Main
Default                                  : Yes
Forced                                   : No

As you can see, I have only gone for AC3 audio for the time being, because I was more concerned about the picture being correct...

...and as far as I can tell (because I do not have a standalone consumer UHD player) the resulting picture is very nice indeed.

I needed a budget 4K HDR media player and I am very impressed with the Roku, in fact I still cannot believe it's able to stream 4K over wireless with DIRECT PLAY in Plex!

So, I welcome people comments on this, but wanted to thank the developers of MakeMKV, everyone on this forum and especially to the ebay user who sold me the correct drive and has excellent customer service... above and beyond the call of duty.



Regards,

Paully

[djtimyd]

I'd be curious to see if you have made any changes to this script? I ran it as you posted (with very minor tweaks) and the output was acceptable. Curious if you have made changes to make it more efficient or streamlined or better in some way?

[plittlefield]

I'd be curious to see if you have made any changes to this script? I ran it as you posted (with very minor tweaks) and the output was acceptable. Curious if you have made changes to make it more efficient or streamlined or better in some way?

Hello there!

No, I haven’t changed that command because I haven’t bought another 4K UHD since I wrote this thread.

What tweaks did you make to the command line and what was “acceptable” about the result?

Look forward to your reply.

Paully

[djtimyd]

here is how I have modified the script:

ffmpeg -y -i "\Video\Movie_t18.mkv" -map 0:v:0 -map 0:a:1 -pix_fmt yuv420p10le -vcodec libx265 -preset slow -x265-params "keyint=60:bframes=3:vbv-bufsize=75000:vbv-maxrate=75000:colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display='G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,500)'" -crf 22 -force_key_frames "expr:gte(t,n_forced*2)" -c:a ac3 -c:s copy "\Video\Movie.mkv"

* Your script only ran for the first 10 min of the file - great for testing!, but I removed that for my "final" script.
* I removed the analyze and probe and muxing parts because I have had good success with my FullHD videos using the defaults, and when running multiple tests of your scripts, I couldn't tell the difference with vs without.
* I changed the preset to slow - this I DID notice right away. For my FullHD videos I use veryslow but I also use 264. While my transcode times are acceptable for the FullHD (usually 20-30 fps depending on source), very slow brings my computer to its knees. Slow is a good compromise for me.
* I simplified the AC3 part (I also use AC3 as it is very universal, I don't have Atmos or anything more than 5.1 and doubt I ever will, but if I do, I'll have a video processor that would be smart enough to fake the Atmos for me).
* I force keyframes every 2 seconds because there are sometimes when the screen draw glitches and has missing data for parts of the screen that haven't been updated - by forcing a keyframe every 2 seconds, if that scenario happens, then it only lasts for a maximum of 2 seconds as the entire screen is redrawn every 2 seconds regardless of how the algorithm reads changes.
* I added the subtitle flag - I only ever rip the forced subtitle so this helps with mixed language movies. I haven't had success with this yet, and I haven't been too concerned with the testing part, but it will be fixed eventually.
* I removed the metadata flags. I choose my naming conventions on the file name level and carry that over to the metadata by default.
* I restored the chapter information from your script. I am not sure why you had it flagged to transcode as a flat file, but I like to be able to skip around in a movie.

I have snooped around and looked at other peoples scripts as well, and my next tweak to yours is the L parameter for the -x265 from your (10000000,500) to (12000000,200). Needless to say, I have many more tests to run. I have found that a good source file is very important - I am using one of the more recent F&F movies as there are lots of dynamic ranges available, as well as many flat scenes to use as references.

I'll post the script that I end up sticking with, but that will probably take weeks... ¯\_(ツ)_/¯

[plittlefield]

Hi djtimyd,

Thanks for your detailed reply and ffmpeg tweaks!

I shall invest in a new UHD 4K Blu-ray movie soon (I only have 2 right now... Blade Runner 2049 and Black Panther) and try out your settings.

I am particularly interested in the keyframes settings which will make a nice difference.

Also, I am concerned that my final file may not have quite the right HDR colour settings for my TV... the colours do sometimes look washed out and not as bright as the 1080p Blu-ray version.

I have really noticed this with Blade Runner 2049 but perhaps we can post some screenshot comparisons.

I do NOT have an actual 4K UHD Blu-ray player to use as a reference for that film, so I don't really know what the 4K should actually LOOK like when sent to my Samsung TV... do you follow me?

I use a Plex server --> Roku Streaming Stick+ --> Samsung TV --> Optical Out (DTS) --> Logitech 5.1 Speakers.

I look forward to your reply and your results!

Regards,

Paully

[mfraz74]

Found this in another website which may be of some help:
ffprobe -read_intervals "%+#2" -show_frames -show_entries side_data "video.mkv"
Also is it possible to add auto crop to your script?

[plittlefield]

Found this in another website which may be of some help:
ffprobe -read_intervals "%+#2" -show_frames -show_entries side_data "video.mkv"
Also is it possible to add auto crop to your script?

Interesting, what does that do?

Auto crop... yes. Here is a link to my Wiki which has a page about ffmpeg and cropping filter.

http://wiki.indie-it.com/wiki/FFmpeg_DV ... Black_Bars

Use 2 ffmpeg filters - crop first, then scale...

e.g.

-vf "${CROP}scale=${SCALEWIDTH}:-2"

I have another complex script that analyses the crop required then outputs that to a temporary text file then uses that to form the variable.

Shout if you want to see that.

Paully

[mfraz74]

Found this in another website which may be of some help:
ffprobe -read_intervals "%+#2" -show_frames -show_entries side_data "video.mkv"
Also is it possible to add auto crop to your script?

Interesting, what does that do?

It outputs the values required for HDR:

red_x=34000/50000
red_y=16000/50000
green_x=13250/50000
green_y=34500/50000
blue_x=7500/50000
blue_y=3000/50000
white_point_x=15635/50000
white_point_y=16450/50000
min_luminance=1/10000
max_luminance=10000000/10000

That I haven't been able to extract using mediainfo.

Auto crop... yes. Here is a link to my Wiki which has a page about ffmpeg and cropping filter.

http://wiki.indie-it.com/wiki/FFmpeg_DV ... Black_Bars

Use 2 ffmpeg filters - crop first, then scale...

e.g.

-vf "${CROP}scale=${SCALEWIDTH}:-2"

I have another complex script that analyses the crop required then outputs that to a temporary text file then uses that to form the variable.

Shout if you want to see that.

Paully

I have this https://github.com/tlaziuk/ffautocrop which I haven't tried yet.

[plittlefield]

It outputs the values required for HDR:

red_x=34000/50000
red_y=16000/50000
green_x=13250/50000
green_y=34500/50000
blue_x=7500/50000
blue_y=3000/50000
white_point_x=15635/50000
white_point_y=16450/50000
min_luminance=1/10000
max_luminance=10000000/10000

That I haven't been able to extract using mediainfo.

Woah, that is awesome!

So, are you saying...

...that you can then use these values from the source MKV file in the ffmpeg command to get the HDR settings exactly right in the target MKV file?

e.g. using your example above, the 'master-display' setting would be...

Code: Select all

master-display='G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(1,10000000)'

...is that right?

I take it the L figure is the Luminance?

Paully

[plittlefield]

e.g. using your example above, the 'master-display' setting would be...

Code: Select all

master-display='G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(1,10000000)'

I ran that ffmpeg command to analyse my MKV of a UHD...

Code: Select all

$ ffprobe -read_intervals "%+#2" -show_frames -show_entries side_data "input.mkv"
[SIDE_DATA]
side_data_type=Mastering display metadata
red_x=34000/50000
red_y=16000/50000
green_x=13250/50000
green_y=34500/50000
blue_x=7500/50000
blue_y=3000/50000
white_point_x=15635/50000
white_point_y=16450/50000
min_luminance=1/10000
max_luminance=10000000/10000
[/SIDE_DATA]

... and I've just read that the L Luminance figures are max,min (https://x265.readthedocs.io/en/default/ ... er-display so it should be 10000000,1 ...

Code: Select all

G(13250,34500)B(7500,3000)R(34000,16000)WP(15635,16450)L(10000000,1)

...which is kind of exciting because my transcodes of 2 films were not quite right, because I had 10000000,500 which may or may not be an important factor!?

Comments please...

Paully

[mfraz74]

I was hoping to extract these figures and put them straight into ffmpeg.

[plittlefield]

I was hoping to extract these figures and put them straight into ffmpeg.

Oh, no problem.

I can help with that tomorrow.

Cheers,

Paully

[plittlefield]

I was hoping to extract these figures and put them straight into ffmpeg.

OK, here is a quick BASH script to do what you want, just outputting 1 of the variables as a test...

Code: Select all

ffprobe -loglevel quiet -read_intervals "%+#2" -show_entries side_data "input.mkv" | grep -A 10 'side_data_type=Mastering display metadata' | tail -n10 >/tmp/variables.txt; source /tmp/variables.txt; echo "$red_x"

...then, you then use that in your main script where your ffmpeg command goes...

e.g.

Code: Select all

#!/bin/bash

# load HDR variables from input MKV file
ffprobe -loglevel quiet -read_intervals "%+#2" -show_entries side_data "input.mkv" | grep -A 10 'side_data_type=Mastering display metadata' | tail -n10 >/tmp/variables.txt && source /tmp/variables.txt

# use those variables in my ffmpeg command
ffmpeg ... bla bla ... master-display='G("$green_x","$green_y")B("$blue_x","$blue_y")R("$red_x","$red_y")WP("$white_point_x","$white_point_y")L("$max_luminance","$min_luminance")'

exit

Please let me know how you get on!

Regards,



Paully

[asmcom]

Hello Paully,

Just been reading through this thread. It is very interesting.

Amazing what you can do with Linux.

Hope you and your family or staying safe.

Hard times Paully.

Asmcom

[plittlefield]

Hope you and your family or staying safe.

Yes, we're alright for now... hope you are too.

My UHD Drive is still going strong thanks!



Kind regards,

Paully