Thanks to all of you for your diverse input. My wife was a deaf pro photographer, my daughter shot for NASCAR and I have grand champion photos. So, it looks like I will be amassing a mass of terabytes for my 5,000 plus video collection. So far they are averaging 29gb per .mkv file. As for Plex, I used it on my old video serving system. It worked, period. I am building a Kodi system since the price is right and it works on windows, Linux and Android. ( I threw all my apple j__k out the door). I may even hook my Raspberry up to it. With so many perfectionists eyes I will be racking and stacking media storage.
Be kind, I don't rewind.
Best compression while maintaining quality
Re: Best compression while maintaining quality
I thought I might as well jump in with a bunch of not-very-useful comments. I transcode using a... rather elaborately horrible organically-grown script that I pushed to https://github.com/nickalcock/copy-dvd-title literally seconds ago, so it is currently totally undocumented. It *does* let me (subject to various annoying limitations) pull in DVDs or BluRays via HandBrake or makemkv or even ripped out of vlc, or directories full of pre-ripped stuff from any source, and optionally throw the lot through vapoursynth prior to encoding with HandBrake, feeding in input files that look kind of like this:
--date=1999
--genre=Cult
--audio-codec=opus
--synth=deivtc.vpy
--subtitles=1
--
0 Fight Club :: --synth=denoise.vpy --audio=1,6,7,8,9 --subtitles=1,1,15,22,29,31 --force-subtitles=1
10 Flogging Fight Club :: --no-synth
11 Behind the Scenes: Production: Alternate Main Titles: Textless :: --audio=1,2
40 Behind the Scenes: On location
# ... etc, lines omitted ...
(The options override command-line options to copy-dvd-title, which are of course undocumented but do appear in the script: they are pulled in for each title from dvd-title.conf in all directories above the current directory, then from the top of the specified catalogue file, then from after the :: in the current title. The relationship between all these options is a flaming nightmare which is waiting for me to document it. Not all combinations are valid, not even combinations which would seem to make sense.)
The important point is that this madness means I can save a lot of space by using much better denoisers, deinterlacers, deringers etc than any available from HandBrake, with the huge caveat that this needs a lot of disk space (dozens of gigabytes minimum) for the lossless H.264 intermediates, and that vapoursynthing BluRays is... not at all fast! My preferred denoiser right now is the G41Fun TemporalDegrain2, which sacrifices hardly any detail or sharpness while slashing noisy BluRay file sizes by about a factor of five... but there is the giant caveat that this repo disappeared off the Internet last year (along with its author) so it's kind of hard to find anywhere. I'm tempted to push my fixed-up copy back to my github, since the license permits it... but honestly a lot of other denoisers, including QTGMC, work perfectly well.
So... if you have the machine time spare, do try VapourSynth or AviSynth. They can often save massive amounts of space without sacrificing quality, if you try hard enough.
--date=1999
--genre=Cult
--audio-codec=opus
--synth=deivtc.vpy
--subtitles=1
--
0 Fight Club :: --synth=denoise.vpy --audio=1,6,7,8,9 --subtitles=1,1,15,22,29,31 --force-subtitles=1
10 Flogging Fight Club :: --no-synth
11 Behind the Scenes: Production: Alternate Main Titles: Textless :: --audio=1,2
40 Behind the Scenes: On location
# ... etc, lines omitted ...
(The options override command-line options to copy-dvd-title, which are of course undocumented but do appear in the script: they are pulled in for each title from dvd-title.conf in all directories above the current directory, then from the top of the specified catalogue file, then from after the :: in the current title. The relationship between all these options is a flaming nightmare which is waiting for me to document it. Not all combinations are valid, not even combinations which would seem to make sense.)
The important point is that this madness means I can save a lot of space by using much better denoisers, deinterlacers, deringers etc than any available from HandBrake, with the huge caveat that this needs a lot of disk space (dozens of gigabytes minimum) for the lossless H.264 intermediates, and that vapoursynthing BluRays is... not at all fast! My preferred denoiser right now is the G41Fun TemporalDegrain2, which sacrifices hardly any detail or sharpness while slashing noisy BluRay file sizes by about a factor of five... but there is the giant caveat that this repo disappeared off the Internet last year (along with its author) so it's kind of hard to find anywhere. I'm tempted to push my fixed-up copy back to my github, since the license permits it... but honestly a lot of other denoisers, including QTGMC, work perfectly well.
So... if you have the machine time spare, do try VapourSynth or AviSynth. They can often save massive amounts of space without sacrificing quality, if you try hard enough.
Re: Best compression while maintaining quality
How does your method with vapoursynth, etc. compare to just doing a h.265 transcode? For example, I recently turned a 26.4GB blu-ray rip MKV into a 5.3GB h.265. To my eyes the h.265 looks almost indistinguishable from the source for casual movie watching from the couch.
Re: Best compression while maintaining quality
I've never bothered with HEVC because the rasbpi I play it back on can't do hardware HEVC decoding. (Also, the one time I tried to encode to it, it encoded at about .05 frames/second: thanks, but no thanks. Even QTGMC + TemporalDegrain2 manages 8 fps on my 10-core Broadwell.)
Also... I'm fairly sure that for the sort of thing I bother denoising, things like Breaking Bad or Battlestar Galactica, which usually have a great deal of noise artificially added for the look of the thing, both H.264 and HEVC end up spending a huge amount of space encoding the nearly-incompressible noise rather than anything else. I mean I've just transcoded a bluray of the similarly noisy TV series Chernobyl: 15GiB/episode input, 1.3GiB/episode output, and half of the output is the DTS stream! The perceived quality, to my eyes, is *higher* afterwards, because all that bloody headache-inducing noise is gone. I suppose if you like massive amounts of film grain or are interested in perfect replication of the source your opinion might differ, but I'm more interested in not having a blinding headache and also in being able to get the thing over the wifi link to my media centre without running out of bandwidth (I just tried a straight transcode of one episode, without denoising: 9GiB. So almost eight gigabytes wasted to encode pure noise.)
Also... I'm fairly sure that for the sort of thing I bother denoising, things like Breaking Bad or Battlestar Galactica, which usually have a great deal of noise artificially added for the look of the thing, both H.264 and HEVC end up spending a huge amount of space encoding the nearly-incompressible noise rather than anything else. I mean I've just transcoded a bluray of the similarly noisy TV series Chernobyl: 15GiB/episode input, 1.3GiB/episode output, and half of the output is the DTS stream! The perceived quality, to my eyes, is *higher* afterwards, because all that bloody headache-inducing noise is gone. I suppose if you like massive amounts of film grain or are interested in perfect replication of the source your opinion might differ, but I'm more interested in not having a blinding headache and also in being able to get the thing over the wifi link to my media centre without running out of bandwidth
(I just tried a straight transcode of one episode, without denoising: 9GiB. So almost eight gigabytes wasted to encode pure noise.)Re: Best compression while maintaining quality
I happen to have that same TV series, so I tried some encodings of the first episode. My MKV rip is 12.25 GB. My h.264 encode is 3.64 GB. My 10-bit h.265 encode is 2.89 GB. Not as small as the files you've created, but still significantly smaller than the 1:1 rips. I don't use a denoiser, so all the film grain, both real and artificial, is still there.
For h.264, I use GPU-based encoding on a Radeon Pro 580 in my iMac. It takes my Kaby-Lake 4-core i5 system just under 9 minutes to encode. For 10-bit h.265, I'm stuck with CPU-only encoding courtesy of ffmpeg & as such, it took 2 hours, 27 minutes at an average rate of 9.6 fps. In both cases I'm using an average-bit-rate scheme, with a target of 8000 Kbps for h.264 and 6000 Kbps for 10-bit h.265. Ultimately, the resulting encoded file size difference comes down to the target bit rate used during the encoding. I have found that those targets are generally an excellent trade off (for me anyway) for file size vs quality for most situations. Since h.264 encoding is so much faster, it is my default choice. I generally only mess with 10-bit h.265 when the h.264 encode has some noticeable artifacts; usually color banding.
For simpler video, like animation, those bit rate targets can be lowered significantly with no apparent loss of quality to my eye.
As you might've guessed, reducing disk space usage is not my primary concern. In fact, I generally keep all my MKV rips in addition to creating transcodes. The slow part of digitizing one's movie collection is the process of getting the bits off the disc. The convenient formats of today, like h.264 or h.265, may not be the convenient formats of tomorrow. I'm betting that I'll be able to transcode MKV rips into the format of tomorrow, however.
Ultimately, the discussion is academic. Everyone has their own requirements to satisfy. I'm always curious, however, about the ways people go about it.
For h.264, I use GPU-based encoding on a Radeon Pro 580 in my iMac. It takes my Kaby-Lake 4-core i5 system just under 9 minutes to encode. For 10-bit h.265, I'm stuck with CPU-only encoding courtesy of ffmpeg & as such, it took 2 hours, 27 minutes at an average rate of 9.6 fps. In both cases I'm using an average-bit-rate scheme, with a target of 8000 Kbps for h.264 and 6000 Kbps for 10-bit h.265. Ultimately, the resulting encoded file size difference comes down to the target bit rate used during the encoding. I have found that those targets are generally an excellent trade off (for me anyway) for file size vs quality for most situations. Since h.264 encoding is so much faster, it is my default choice. I generally only mess with 10-bit h.265 when the h.264 encode has some noticeable artifacts; usually color banding.
For simpler video, like animation, those bit rate targets can be lowered significantly with no apparent loss of quality to my eye.
As you might've guessed, reducing disk space usage is not my primary concern. In fact, I generally keep all my MKV rips in addition to creating transcodes. The slow part of digitizing one's movie collection is the process of getting the bits off the disc. The convenient formats of today, like h.264 or h.265, may not be the convenient formats of tomorrow. I'm betting that I'll be able to transcode MKV rips into the format of tomorrow, however.
Ultimately, the discussion is academic. Everyone has their own requirements to satisfy. I'm always curious, however, about the ways people go about it.