FLAC 24bit to 16bit removing zeros + audio delays

[Mango]

Hi,

2 questions:

1.
Often TrueHD (and sometimes DTS-HDMA) 24bit tracks are in fact 16bit tracks artificially bumped to 24bit, the added 8bit containing only zeros.

When converting these tracks, after the conversion eac3to automatically checks the file to see if it is a real 24bit file and if it is a 16bit file bumped up to 24bit it will do a 2nd pass to remove the zeros. This results in a 2x smaller FLAC file.

Furthermore, if this isn't done the resulting FLAC file will be ~2x bigger than the original TrueHD file (TrueHD compresses these 0s very well, but FLAC doesn't). This adds several Gb to the MKV file size.

Question: Does Makemkv also do this properly? Or is it still better to convert 24bit tracks with eac3to (and then remux them with mkvmerge) if we don't want to waste gigabytes for empty zeros?

2.
Some tracks have audio delays. Eac3to also corrects these delays to resync the audio properly without needing to add the delay to the mkv file audio track flag.

Does Makemkv also properly fix the delays in the resulting FLAC?

Thanks.

[mike admin]

1.
Often TrueHD (and sometimes DTS-HDMA) 24bit tracks are in fact 16bit tracks artificially bumped to 24bit, the added 8bit containing only zeros. Question: Does Makemkv also do this properly?

No, it doesn't. Do you know how exactly eac3to does the check - just scans a portion of file? Are there any false positives (a long quiet segment at the beginning of a file)?

Some tracks have audio delays. Eac3to also corrects these delays to resync the audio properly without needing to add the delay to the mkv file audio track flag.
Does Makemkv also properly fix the delays in the resulting FLAC?

No, MakeMKV doesn't change the stream file but adds a proper audio delay at container level (in MKV file). Please note that "simple" audio delay is rare these days - usually a title on a blu-ray disc consist of dozen segments and there is somewhat complex logic to join them properly. You may want to enable "show AV sync issues" in advanced preferences.

[Mango]

No, it doesn't. Do you know how exactly eac3to does the check - just scans a portion of file? Are there any false positives (a long quiet segment at the beginning of a file)?

I don't know how the check is done "exactly", but I do know that it checks the complete file.

This is what madshi (eac3to author) says:

"The bitdepth is simply not known before the processing. The information about the bitdepth is not contained anywhere in the TrueHD track. eac3to has no other choice than to fully decode the full track and check the bitdepth of every single audio sample (of which there are usually 48000 for every second of the runtime). If all 24bit audio samples have 8bits zeroed out, eac3to will state that it's a 16bit track only and will remove the 24bit file."

I have seen these 3 results with eac3to:

1. 24 bit audio is real 24bit -> it keeps it as it is.
2. 24 bit audio is 16 bit -> it does a 2nd pass to remove the zeroed 8bits
3. sometimes it says "average bit rate is 19 bit or 20bit" -> it keeps the 24 bit file as it is.

Perhaps you could get in touch with madshi for more information : http://forum.doom9.org/showthread.php?t=125966

(24bit files that are in fact true 16 bit result in about ~2Gb lost space for the 0s, depending on the audio and duration of course)

No, MakeMKV doesn't change the stream file but adds a proper audio delay at container level (in MKV file). Please note that "simple" audio delay is rare these days - usually a title on a blu-ray disc consist of dozen segments and there is somewhat complex logic to join them properly. You may want to enable "show AV sync issues" in advanced preferences.

Yes, you are right. Audio delays happen less often today. But they still do happen with some blu-rays from time to time.

It would be great if makemkv could fix delays as eac3to does, so that we get a flac file that perfectly matches the video (to make the mkv future proof against audio sync issues, you never know if you will reprocess the file in the future with some tool...).

-------------------------------------

Two more points:

3.
All versions of dtsdecoder.dll have some issues with some DTS-HDMA tracks (resulting in non-lossless output, sound corruption, etc...) .

dtsdecoder.dll 1.1.0.0 works fine for all tracks (including 6.0/6.1/7.1), but not 7.1 "strange setup" tracks.
dtsdecoder.dll 1.1.0.8 decodes 7.1 "strange setup" correctly, but doesn't decode 6.0/6.1 correctly.

So, when eac3to detects a 7.1 "strange setup" track, dtsdecoder.dll 1.1.0.8 should be used (and only for this, as it has issues with 6.0/6.1). Version 1.1.0.0 is fine for everything else.

Question: Could you make makemkv automatically detect 7.1 "strange setup" tracks, and use the correct dtsdecoder.dll accordingly?

4.
Sometimes (I think most often with 6.1 tracks, but not everytime), eac3to will detect a channel mapping problem and it will remap the channels. If it doesn't do this the channel position will be wrong.

Question: Does makemkv also take care of this issue?

[Romansh]

1.
Often TrueHD (and sometimes DTS-HDMA) 24bit tracks are in fact 16bit tracks artificially bumped to 24bit, the added 8bit containing only zeros. Question: Does Makemkv also do this properly?

No, it doesn't. Do you know how exactly eac3to does the check - just scans a portion of file? Are there any false positives (a long quiet segment at the beginning of a file)?

IIRC it analyzes the whole track (presumably somewhere between the decoder and the FLAC encoder).

[Mango]

I have already replied earlier with details, but my post is still awaiting moderator approval as it included a link (Had I known I wouldn't have included a link).

This post also talked about 2 other issues (DTS-HDMA decoding needs 2 different versions of dtsdecoder.dll to be done properly and wrong channel mapping with some tracks).

edit: my earlier post was finally posted, above.

IIRC it analyzes the whole track (presumably somewhere between the decoder and the FLAC encoder).

It first saves the track to FLAC as normal. Then, it analyses the whole track. After the analysis, it re-encodes it to FLAC a 2nd time if the track is 16 bit.

[mike admin]

All versions of dtsdecoder.dll have some issues with some DTS-HDMA tracks (resulting in non-lossless output, sound corruption, etc...) .
dtsdecoder.dll 1.1.0.0 works fine for all tracks (including 6.0/6.1/7.1), but not 7.1 "strange setup" tracks.
dtsdecoder.dll 1.1.0.8 decodes 7.1 "strange setup" correctly, but doesn't decode 6.0/6.1 correctly.

This statement is not correct for MakeMKV. I have to dig into details what dtsdecoderdll.dll really is. It is a reference DTS decoder library (written by DTS) with an API layer on top of it (written by Chinese developers). So, all versions of dtsdecoderdll, from 1.0.0.0 to 1.0.0.8 have exactly the same reference decoder library linked in - "VERSION=325,REVISION=27" . The only difference is a Chinese crap on top. Unlike eac3to and others, MakeMKV uses the reference decoder directly as stated on /dtshd/ . All other software, including eac3to, use the Chinese API. Any version of dtsdecoderdll.dll will work equally fine with MakeMKV, even early 1.0.0.0 variant. MakeMKV decodes proper channel layouts for all possible configurations (that one can create with DTS HD master audio suite). I encourage you to check MakeMKV log file - it lists the DTS decoder version, DTS channel layout and output channel mask.

[mike admin]

IIRC it analyzes the whole track (presumably somewhere between the decoder and the FLAC encoder).

It first saves the track to FLAC as normal. Then, it analyses the whole track. After the analysis, it re-encodes it to FLAC a 2nd time if the track is 16 bit.

The full track analysis is a no-go for MakeMKV, as it converts in a single pass. However I do see the value of this feature. I'm thinking of an approach that should work for all practical purposes - scan about 30 random places in a file, about 1 second each, and if all of them are 16-bit then declare the stream as 16-bit. This scan should only take few seconds. The parameters for the scan (number of points, duration, and whether to scan at all) will be set in profile. In the very unlikely event that all file is 16-bit and 1 second is 24-bit, you'll get a warning and somewhat broken audio. Then you'll have to redo the file with different profile.

[Mango]

This statement is not correct for MakeMKV. I have to dig into details what dtsdecoderdll.dll really is. It is a reference DTS decoder library (written by DTS) with an API layer on top of it (written by Chinese developers). So, all versions of dtsdecoderdll, from 1.0.0.0 to 1.0.0.8 have exactly the same reference decoder library linked in - "VERSION=325,REVISION=27" . The only difference is a Chinese crap on top. Unlike eac3to and others, MakeMKV uses the reference decoder directly as stated on /dtshd/ . All other software, including eac3to, use the Chinese API. Any version of dtsdecoderdll.dll will work equally fine with MakeMKV, even early 1.0.0.0 variant. MakeMKV decodes proper channel layouts for all possible configurations (that one can create with DTS HD master audio suite). I encourage you to check MakeMKV log file - it lists the DTS decoder version, DTS channel layout and output channel mask.

Excellent news, thanks.

[Mango]

The full track analysis is a no-go for MakeMKV, as it converts in a single pass. However I do see the value of this feature. I'm thinking of an approach that should work for all practical purposes - scan about 30 random places in a file, about 1 second each, and if all of them are 16-bit then declare the stream as 16-bit. This scan should only take few seconds. The parameters for the scan (number of points, duration, and whether to scan at all) will be set in profile. In the very unlikely event that all file is 16-bit and 1 second is 24-bit, you'll get a warning and somewhat broken audio. Then you'll have to redo the file with different profile.

Yes, this would be great.

So, as I understand, you would then also do the full analysis while the audio is decoded, and give the information (confirmation that the audio is indeed 16bit or 24bit) in the output log when the conversion is finished? Very nice.

Thanks.

[dannyboy48888]

Using a version from dtsdecoder.dll from TMT6 under Arch linux of all things and have had only one audio issue (See end). converting with FFMPEG to opus and aac shows the source correctly (24bit 5.1/7.1/etc...) The only movies that act stupid are the Star Wars set but they use a odd 6.1 mixing. for those I extract the core and carry on normally. As far as the Zero padding if it's all zeros across the board flac compresses it well and does not add any size (~1%) compared to 16-bit sample. If it's truly 24 bits than the size will reflect as such.

[Mango]

Using a version from dtsdecoder.dll from TMT6 under Arch linux of all things and have had only one audio issue (See end). converting with FFMPEG to opus and aac shows the source correctly (24bit 5.1/7.1/etc...) The only movies that act stupid are the Star Wars set but they use a odd 6.1 mixing. for those I extract the core and carry on normally.

So, this confirms the problem I referred to above. If what you are saying is true Makemkv doesn't fix it as mike said.

If you use dtsdecoder.dll 1.1.0.0 it will extract the odd 6.1 mixing correctly (Caution: Star Wars are not the only movies that use this channel setup).

dtsdecoder.dll 1.1.0.0 decodes everything fine (including 6.0, 6.1 and normal 7.1) but 7.1 Strange setup (which is decoded correctly with dtsdecoder.dll 1.1.0.8 ).

But, if what mike says is true, then I guess the 1.1.0.8 dll has to be used normally (which leaves only eac3to if you want to extract the track properly). This needs to be tested further, but I think that Makemkv can't do it properly if it is accessing the DTS decoder (which is the same in all of the dll versions) contained within the dll directly (this assumes that the other part of the code in the dll is what's remapping the channels correctly, depending on the dll version and channel mixing).

Eac3to is the only app I know off that will show you the correct channel setup used by a track.

As far as the Zero padding if it's all zeros across the board flac compresses it well and does not add any size (~1%) compared to 16-bit sample. If it's truly 24 bits than the size will reflect as such.

TrueHD does, but FLAC does not. Compared to TrueHD, the (true 16bit) 24bit FLAC file will be ~2x bigger. (and the DTS-HDMA will be even bigger than the FLAC file).

I have experimented with the official reference DTS-HDMA and TRUEHD encoders, and converted more than 300 blurays to FLAC.

There are also 1 or 2 audio tracks that always come out smaller with TrueHD than FLAC. Example:

Transformers 3 (TrueHD is 16 bit resampled to 24bit due to the TrueHD format specs):

- TrueHD 24bit is ~200Mb smaller than the 16bit FLAC (16bit detected by eac3to, so the 24bit is auto down converted to 16bit).
- Keeping the FLAC at 24bit (with all the zeros) adds ~2Gb to the file (compared to the 24bit TrueHD and 16bit FLAC).
- Converting to 24bit DTS-HDMA is even bigger that the FLAC.

For most other movies, the FLAC (which if true 16bit must always be down converted from 24bit to 16bit) will always be between ~200 and 800 Mb smaller than the TrueHD (DTS-HDMA is always a lot larger than both). This is also the case for true 24bit.

[dannyboy48888]

Well did some more homework. Think the problem is with FFMEG, as anything that uses it does not decode the 6.1 channels right (FFPLAY, VLC,etc...) while anything based on libav does (mpv, MPC-HC,etc) and you are right about the flac sizes straight from MakeMKV. I was talking about decompress-recompress with flac. anything that has padded 0's is ALOT smaller while if there is actual data there it is within ~1% of makeMKV size. Bit off topic...How do I get email alerts when there is a new reply? Thanks.

[Mango]

Well did some more homework. Think the problem is with FFMEG, as anything that uses it does not decode the 6.1 channels right (FFPLAY, VLC,etc...) while anything based on libav does (mpv, MPC-HC,etc)

Ah yes, this is another problem which doesn't have anything to do with the dtsdecoder.dll decoding.

The 6.1 channels decoding was fixed in FFMPEG since version 1.2. Versions prior to 1.2 were playing back corrupted audio because they didn't have the 6.1 channel mapping. 6.1 channel mapping was introduced in FLAC 1.3 (and FFMPEG got it since version 1.2).

This means that the software players you are having problems with are using an outdated FFMPEG version. If you use anything that relies on FFMPEG 1.2 or newer it will be fine.

This also means that if your 6.1 Star Wars audio plays fine (with proper channel positioning and without corruption), then what mike said was right: Makemkv doesn't have the issues other tools are having due to the way it uses the dts decoder within the dtsdecoder.dll. This is good news.

How do I get email alerts when there is a new reply? Thanks.

Select "Notify me when a reply is posted", just above the "Preview" "Submit" buttons. I think you can also set this as default in your user profile.

[mike admin]

Well did some more homework. Think the problem is with FFMEG, as anything that uses it does not decode the 6.1 channels right (FFPLAY, VLC,etc...) while anything based on libav does (mpv, MPC-HC,etc)

Can you please post the following info for this 6.1 track:

1. Mediainfo output for produced MKV file
2. MakeMKV_log.txt from converting to MKV with FLAC profile. Just start the conversion, wait 10 seconds, cancel, quit MakeMKV and then copy the log file.


Thanks!

[dannyboy48888]

Okay here we go...
Disc: Star Wars Episode 02-Attack Of The Clones-BD
Rip DTS-MA copy: complete
Rip DTS-MA to FLAC: Fail, no error messages, just program exit

@Mango , compiled ffmpeg and libav from GIT and both map the DTS-MA correctly now so thanks. I Will marks this thread as follow.

@Mike_Admin, Sorry about the vagueness...From the get go all the Star Wars would fail to rip to FLAC, I assumed it was because of bogus channel mapping as the DTS-MA acted stupid in my players and just extracted the 5.1 stream. Now I don't know as it decodes properly but still fails to convert to FLAC in makeMKV. there is no log file. Here is the MediaInfo for the file;

General
Unique ID : 235008010610123811985828237045503235387 (0xB0CCE57FBED6A4521F3C26BF66AE153B)
Complete name : /root/Desktop/02-Attack_of_the_Clones/Attack_of_the_Clones_t01.mkv
Format : Matroska
Format version : Version 2
File size : 33.0 GiB
Duration : 2h 22mn
Overall bit rate mode : Variable
Overall bit rate : 33.2 Mbps
Movie name : Attack of the Clones
Encoded date : UTC 2013-11-19 11:58:18
Writing application : MakeMKV v1.8.6 linux(x64-release)
Writing library : libmakemkv v1.8.6 (1.2.0/1.1.0) x86_64-unknown-linux-gnu

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 2h 22mn
Bit rate mode : Variable
Maximum bit rate : 35.7 Mbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Language : English
Default : No
Forced : No

Audio
ID : 2
Format : DTS
Format/Info : Digital Theater Systems
Format profile : MA / Core
Mode : 16
Format settings, Endianness : Big
Codec ID : A_DTS
Duration : 2h 22mn
Bit rate mode : Variable
Bit rate : Unknown / 1 509 Kbps
Channel count : 7 channels / 6 channels
Channel positions : Front: L C R, Side: L R, Back: C, LFE / Front: L C R, Side: L R, LFE
Sampling rate : 48.0 KHz
Bit depth : 24 bits
Compression mode : Lossless / Lossy
Title : Lossless
Language : English
Default : Yes
Forced : No

Text
ID : 3
Format : PGS
Codec ID : S_HDMV/PGS
Codec ID/Info : The same subtitle format used on BDs/HD-DVDs
Language : English
Default : No
Forced : No

Menu
00:00:00.000 : en:Chapter 01
00:00:23.440 : en:Chapter 02
00:01:50.527 : en:Chapter 03
00:04:22.011 : en:Chapter 04
00:07:00.044 : en:Chapter 05
00:10:28.919 : en:Chapter 06
00:14:19.900 : en:Chapter 07
00:21:35.294 : en:Chapter 08
00:24:48.236 : en:Chapter 09
00:29:46.075 : en:Chapter 10
00:31:33.308 : en:Chapter 11
00:33:48.443 : en:Chapter 12
00:34:50.546 : en:Chapter 13
00:36:05.079 : en:Chapter 14
00:38:25.845 : en:Chapter 15
00:39:25.279 : en:Chapter 16
00:40:55.995 : en:Chapter 17
00:42:12.780 : en:Chapter 18
00:44:01.972 : en:Chapter 19
00:45:50.414 : en:Chapter 20
00:47:50.075 : en:Chapter 21
00:50:20.934 : en:Chapter 22
00:53:01.344 : en:Chapter 23
00:56:44.776 : en:Chapter 24
00:58:56.699 : en:Chapter 25
01:00:46.017 : en:Chapter 26
01:03:41.484 : en:Chapter 27
01:06:24.313 : en:Chapter 28
01:11:18.732 : en:Chapter 29
01:14:01.353 : en:Chapter 30
01:15:23.519 : en:Chapter 31
01:17:48.872 : en:Chapter 32
01:21:40.353 : en:Chapter 33
01:23:46.855 : en:Chapter 34
01:28:10.076 : en:Chapter 35
01:30:52.905 : en:Chapter 36
01:33:38.487 : en:Chapter 37
01:35:27.847 : en:Chapter 38
01:42:44.783 : en:Chapter 39
01:45:14.016 : en:Chapter 40
01:49:53.628 : en:Chapter 41
01:56:22.475 : en:Chapter 42
01:57:47.602 : en:Chapter 43
02:00:29.180 : en:Chapter 44
02:04:48.981 : en:Chapter 45
02:07:41.945 : en:Chapter 46
02:11:23.667 : en:Chapter 47
02:12:54.841 : en:Chapter 48
02:14:49.247 : en:Chapter 49
02:15:54.938 : en:Chapter 50