DTS:X decoding bug in MakeMKV 1.9.7 / libdca

[Yoshi]

Hello mike,

after I was finally able to register here (odd IP had been blocked error before), I wanna put your attention to a known bug in one of the last libdca libraries, MakeMKV uses now by default to decode DTS files.

I published my discovery here, where eac3to, which uses the same library now, is discussed:

http://forum.doom9.org/showpost.php?p=1 ... ount=13625

In summary, the decoder doesn't decode the DTS-HD MA track which apparently contains additional DTS:X - information properly, but inverts the data during clipping, leading to very awkward clicks.

Please further take into consideration that the clipping seems the be only avoidable when reducing the volume during decoding by about 3 dB.

eac3to starts a 2nd pass in order to do that, I wonder what MakeMKV does in such a case.

The current version of MakeMKV unfortunately delivers the same faulty result as eac3to did before (version 3.30).

In general, it would be advisable if the user would have the option to make MakeMKV still use the dtsdecoder.dll instead for all DTS sources, just to have some kind of reference and ability to cross-check.

Cheers,
Yoshi

[mike admin]

In general, it would be advisable if the user would have the option to make MakeMKV still use the dtsdecoder.dll instead for all DTS sources, just to have some kind of reference and ability to cross-check.

Go to Preferences, select a valid path to dtsdecoder.dll and then perpend '!' to it, like '!C:\path_to\dtsdecoder.dll' . This will force MakeMKV to use dtsdecoderdll.dll for all streams, as it did before.

[Yoshi]

Many thanks for your reply and hint. I wasn't aware of that option, but even better.

However, since the result of the ArcSoft decoder isn't lossless under all circumstances either, at least in conjunction with eac3to, any plans to change the libdca library?

[ndjamena]

Clipping MUST be dealt with in two passes, if you can't do it in two passes the ArcSoft method is the only other way of dealing with it.

MakeMKV can't encode all the flac to quiet 32 bit just in case it encounters clipping, and I think you'll find MakeMKV just truncates the clipping found in AC3 and any other source it encounters, why would you think DTS-MA would be any different?

You're making too much of this, running through a second pass after encountering clipping is a special feature of eac3to, no other program or device bothers doing that, it's impractical in most cases.

It is impossible to get a lossless encode from your sample, the bugs in libdca have been fixed and MakeMKV should be updated with those fixes in the next version.

[mike admin]

It is impossible to get a lossless encode from your sample, the bugs in libdca have been fixed and MakeMKV should be updated with those fixes in the next version.

Obviously, the next version will come with latest libdca.

[Yoshi]

Clipping MUST be dealt with in two passes, if you can't do it in two passes the ArcSoft method is the only other way of dealing with it.

And what would be wrong again to have two passes at least optionally for the geeks in MakeMKV as well?

MakeMKV can't encode all the flac to quiet 32 bit just in case it encounters clipping

Why not? What's wrong with quiet files? At least I have a volume knob I'm happy to use on my AVR.

and I think you'll find MakeMKV just truncates the clipping found in AC3 and any other source it encounters, why would you think DTS-MA would be any different?

In practise, for me there is an essential difference between handling lossy codecs and (at least nominally) lossless ones: lossy ones like AC3 or DTS, one doesn't want to reencode anyway and keep it "as is". What an AVR actually makes out of it, clip it as well or renders it more quietly, is secondary cause at least, I won't lose any (additional) information. When it comes to lossless codecs however, I tend to convert them in order to save space without losing additional information please. However, due to the different handling, it might not be lossless as intended to be. Hence I would prefer to have the additional effort to get out the most out of the so-called lossless codecs as possible. At the end, keeping them "as is" might be more advisable as well.

You're making too much of this, running through a second pass after encountering clipping is a special feature of eac3to, no other program or device bothers doing that, it's impractical in most cases.

Oh, am I? I thought we were those kind of professionals where the best is just good enough. Talking about practical use, dealing with lossless codecs in the first place in general, we all make too much about it cause in a proper blind test, virtually noone will be able to tell the difference anyway. After all, the core bitrates of AC3 and DTS used on Blu-rays is more than enough to be transparent. Since virtually all the codecs surpassed the point of transparency, in essence, it's the mix and mastering which determines the quality. Which is why, some LaserDisc soundtracks rock while the BLu-ray lossless sucks (best example: Back to the future).

So if we go for lossless, rationally knowing that there the nonsense begins, I want to do it right or let it go completely.

[ndjamena]

You'll find it interesting to know that I encountered a FLAC file made by MakeMKV from a non-DTS:X DTS-MA stream that EAC3To decided had clipping and ran through a second pass.

Apparently when MakeMKV encounters clipping in MA it ignores it and just writes it into the destination codec, or at least it did. Who knows what it does now that there's a new version of libdca.

But that's not a solution to the problem, at some point that stream has to be forced through a 24 bit pipeline and that clipping will have to be dealt with somehow (unless you have a receiver that decodes it and uses 32 bit internally of course.)

The point I'm trying to make is that there IS no practical way of dealing with the issue. The audio overreached its allotted number of bits, the equipment being used to process it simply aren't designed to handle that and there's nothing MakeMKV or Eac3to can do about it except either sacrifice those few extended bits or sacrifice all the lower bits in the stream.

MakeMKV is designed to rip titles from Blu Rays, it's NOT going to put them through a second pass, thankfully it seems to save the info (in flac at least) so if you're willing you can let EAC3To take a look at it after the fact.

I've been trying to get Mike to do something about 16 bit audio in 24 bit audio for years now and nothing has been done about it, I'd be kind of pissed if he went out of his way to fix a few random bits in a handful of DTS-MA tracks that have done the wrong thing. And what you're suggesting is WAY out of his way. Mike has already suggested a variation of two pass encoding for 16 bit in 24 bit, but that variation was simply to scan sections of the disc to guestimate if it is 16 bit, and if it turns out its guess is wrong, inform the user that they need to rerip the disc. I suppose it could do the same thing with clipping.

In the end the only way to get actual lossless audio from these samples is to encode it at a higher bit depth, and the HDMI standard doesn't actually allow anything above 24 bit. Other than that you could sacrifice 2 bits worth of data in every sample in order to save the data from a few random peaks. There's no good way out of that. Unless you plan on loading the file into Audacity and remixing it before converting it back into 24 bit, saving it as 32 bit gets you nowhere, and if you really believe that sacrificing all the lower precision bits to save a few overreaching higher bits is worth it then I guess that's up to you.

YOU ARE MAKING TOO MUCH OF IT. The problem is that the tracks exist when they shouldn't and there's no good solution to that.

Why is this important? What exactly do you plan on doing with them that everyone else has to hear about it?

[Yoshi]

First of all, your in-depth technical knowledge is absolutely impressive and has my full respect. So many thanks for your elaborate reply on this.

You'll find it interesting to know that I encountered a FLAC file made by MakeMKV from a non-DTS:X DTS-MA stream that EAC3To decided had clipping and ran through a second pass.

Yes, I do indeed, although it doesn't exactly comfort me.

But that's not a solution to the problem [...]

Which, according to your attitude I somewhat read between the lines, actually doesn't exist.

[...] at some point that stream has to be forced through a 24 bit pipeline [...]

I know that's not the point but as a side note, taking into account that properly dithered 16-bit PCM audio will be enough forever for human ears, both dynamic- and SNR-wise, it's somewhat funny that 24-bit is considered a pipeline, something has to be forced in. But I get your argument.

[...] and that clipping will have to be dealt with somehow (unless you have a receiver that decodes it and uses 32 bit internally of course.)

Do you happen to have any knowledge about how regular AVRs handle this in practice?

The point I'm trying to make is that there IS no practical way of dealing with the issue. The audio overreached its allotted number of bits, the equipment being used to process it simply aren't designed to handle that and there's nothing MakeMKV or Eac3to can do about it except either sacrifice those few extended bits or sacrifice all the lower bits in the stream.

So if I get you correctly, we talk about input data which shouldn't exist this way or been mastered this way (with clipping) in the first place and all efforts to "cure" the aftermath is somewhat hopeless or not really worth it since already (slightly) screwed up, is that your point?

However, I'd claim that technically, it would be better to prevent clipping since it introduces distortion than trying to save a few bits of resolution which information will be buried deep down below the background noise anyway.

MakeMKV is designed to rip titles from Blu Rays, it's NOT going to put them through a second pass, thankfully it seems to save the info (in flac at least) so if you're willing you can let EAC3To take a look at it after the fact.

Interesting fact, thanks for that.

I've been trying to get Mike to do something about 16 bit audio in 24 bit audio for years now [...]

Are you talking about the 16-bit-audio coming in the 24-bit-audio package with superfluous zero LSBs?

Other than that you could sacrifice 2 bits worth of data in every sample in order to save the data from a few random peaks.

As stated above, I would in fact prefer having avoided clipping entirely than let it clip just to save a few bits probably consisting of noise only anyway. If you argue that this slight clipping won't be heard by anyone anyway, I can argue that the lost 1-2 bits won't be missed by anyone, either.

[...]and if you really believe that sacrificing all the lower precision bits to save a few overreaching higher bits is worth it then I guess that's up to you.

Well, from a technical point of view, I would consider a clipping-free file, properly dithered down to 16-bit to be "cleaner" than a 24-bit one with even the slightest clipping (and thus distortion) in it.

But I'm confused when it comes to this issue in general. What is the technical reason for not having just plain simple 24-bit or 16-bit-PCM in the first place? I'm not familiar with the technical details yet behind Dolby TrueHD and DTS-HD MA but why not conserve pure PCM? 16 bit lossless and convertible to 16 bit FLAC would be already totally okay for me (well in most cases as far as I'm aware, that's possible). Why this 32-bit-float fuss?

YOU ARE MAKING TOO MUCH OF IT. The problem is that the tracks exist when they shouldn't and there's no good solution to that.

So I guess we can consider clipped tracks to arise out of pure stupidity by the mastering engineers who slept during class as almost all nowadays when they taught that clipping is to be avoided under all circumstances.

Why is this important? What exactly do you plan on doing with them that everyone else has to hear about it

Maybe I misunderstand you but I don't get your somewhat reproachful statement. Why shouldn't it be important?

Is all that essential for everyday life? Of course not. Will I hear the difference between a movie soundtrack which slightly clips for a few milliseconds and a proper one? Maybe not. But if digging into the very details isn't the maxim, I wonder why you apparently "waste" significant valuable lifetime or yours dealing with that stuff at all. At the end you appear to be highly knowledgeable and skilled so you're the last I'd expect to doubt the magnificence of - let's face it - purely academic or "first world" problems.

As I already stated, from a totally rational point of view, even lossless codecs don't make too much sense except for the avoidance of any cascading effects resulting from re-encoding when in need since I probably never heard the difference and probably never will for the rest of my life.

Your statement sounds reproachful as if I would disturb my environment and had nothing better to do than bothering it with details.

After all, did Andrew Wiles need a reason to finally prove Fermat's last theorem other than his pure curiosity and diligence?

Anyway, a happy new year.