PGS Subtitles MakeMKV & Handbrake

[samtoy]

I also just stumbled over this MakeMKV-related Avatar forced subs bug. Hope this gets fixed sometime. Cheers.

[Romansh]

I don't use Handbrake but there appears to be something off with MakeMKV's forced sub recognition. I always rip main movie only with DVDFab to HDD keeping all English and German streams (I always check forced only too). Then I use MakeMKV and remux again with MKVToolnix (since WDTV does not support uncompressed PGS subs). My recent rips turned out like this:

• MakeMKV extracted 2 forced sub streams which are identical (only four items translating signs). Each of which is marked as forced in BDSUP2SUB. But when I add the MKV to MKVToolnix only one stream is flagged as forced?

It's not uncommon for forced subtitles to be in a dedicated track, with all subtitles flagged as forced. Any chance it's the same source track, extracted twice (once for the regular checkbox, once for the forced checkbox)?

In that case the track extracted with the forced checkbox would be set as forced at the MKV level, but not the one extracted with the regular checkbox.

Also, if I am not mistaken, only one track of a given language can be set as forced in the MKV container. This could also explain why MakeMKV did not set two tracks of the same language as forced.

• MakeMKV extracted 2 forced sub streams which are not identical. The second stream consist of one only 1 (!) item which actually is present as item 6 (of 13) in the first forced stream.

This is a bit odd, but you never know (BD and DVD discs can be authored in many different ways). Which disc is that, if I may ask?

• MakeMKV extracts a stream with very few items, none of which are flagged as forced.

That's not uncommon either: forced subtitles being in a dedicated track, with none of the subtitles (instead of all, see above) being flagged as forced.

[Chetwood]

It's not uncommon for forced subtitles to be in a dedicated track, with all subtitles flagged as forced. Any chance it's the same source track, extracted twice (once for the regular checkbox, once for the forced checkbox)?

No, regular sub is track 4 and the other two are tracks 6 and 7.

Also, if I am not mistaken, only one track of a given language can be set as forced in the MKV container. This could also explain why MakeMKV did not set two tracks of the same language as forced.

MKVToolnix sets both as forced, not that it makes too much sense.

This is a bit odd, but you never know (BD and DVD discs can be authored in many different ways). Which disc is that, if I may ask?

Men In B!ack 2.

That's not uncommon either: forced subtitles being in a dedicated track, with none of the subtitles (instead of all, see above) being flagged as forced.

Except that these aren't from a dedicated track but apparently ripped from the regular track. When I open the regular track in BDSUP2SUB these particular items aren't flagged as forced so I'm wondering how MakeMKV recognized them as such.

[gbooker]

In improving my player for the GoogleTV I had hoped that writing a PGS subtitle display mechanism would prove simple after successfully completing VobSub. I had the unfortunate experience of using a force-only rip of the subtitles from "Avatar" as my first test case. In trying to figure out how one is supposed to determine durations of subtitles, I ran across this thread suggesting that the force-only streams are broken somehow. From what I can tell, MakeMKV is ripping only the forced subtitles in forced-only mode, taking no other action, which is not producing a stream which can correctly display these subtitles. I ripped the disk both with force-only and with all English subtitles and compared them. I'm posting my findings in the hope that this will prove useful in improving MakeMKV in subsequent versions.

First, the mkv file has no durations on the subtitles themselves. It is using [A3] SimpleBlocks to store the subtitle data, which cannot contain duration (assuming I am reading the mkv spec correctly). So, with this usage, stopping the display of a subtitle must be done through the normal mechanisms done in a PGS stream, so I examined both the forced-only and all streams (through mkvextract). I'll only outline the composition packets here and only go through the first 5 forced subtitles for brevity:

Code: Select all

5047 0396296A 00000000 16 0013 0780 0438 10 00F2 40 00 02 01 0002 00 40 0000 0000
Composition F2, one Composition (forced)

5047 0399239A 00000000 16 0013 0780 0438 10 00F3 40 00 03 01 0002 00 40 0000 0000
Composition F3, one Composition (forced)

5047 039C1E24 00000000 16 0013 0780 0438 10 00F4 40 00 04 01 0001 00 00 0168 039D
Composition F4, one Composition (not forced)

5047 039D8CD2 00000000 16 0013 0780 0438 10 00F5 40 00 05 01 0002 00 40 0000 0000
Composition F5, one Composition (forced)

5047 03A09620 00000000 16 0013 0780 0438 10 00F6 40 00 06 01 0003 00 00 0424 039D
Composition F6, one Composition (not forced)

5047 03A221FC 00000000 16 0013 0780 0438 10 00F7 40 00 07 01 0002 00 40 0000 0000
Composition F7, one Composition (forced)

5047 03A54878 00000000 16 0013 0780 0438 10 00F8 40 00 00 01 0002 00 40 0000 0000
Composition F8, one Composition (forced)

5047 03A8343E 00000000 16 000B 0780 0438 10 00F9 00 00 01 00
Composition F9, no Compositions

All of the above are present when ripping with all subtitles, but when ripping with forced-only, F4, F6, and F9 are missing. With these subtitles missing, there is no indicator as to the end of the previous subtitle except for the next subtitle. If this were to be followed explicitly, F8 would be on the screen over 30 minutes as the next subtitle in the forced-only stream is Composition 03A6 at time 0DCA0622.

I believe the proper action when ripping forced-only is to keep track of whether the previous subtitle was forced or not. If the previous subtitle was forced when encountering a non-forced subtitle, then a Composition packet with no compositions should be written along with a window definition and empty palette (which seems to be the normal procedure in this stream). Then, all forced subtitles which are followed by a non-forced subtitle or an empty one, would correctly cease display at the correct times.

Any thoughts on these findings? Is my above analysis correct or have I missed something?

[kfreeb]

Any thoughts on these findings? Is my above analysis correct or have I missed something?

You nailed it, this is exactly what is wrong with MMKVs, (Make MKV) extraction of PGS forced sub titles and what the developers of Handbrake have been saying about this issue on there forums: https://forum.handbrake.fr/viewtopic.ph ... 3&start=25

Many of us here are hoping that the developers of MMKV will be able to fix this problem, (with forced sub titles) in a future release of MMKV. As for now, we have been working around this problem by, (as you discovered) extracting all subs and then durning handbrake encoding, selecting the forced sub title option, (though admittedly not a perfect solution) seems to work OK for now, (forced durations are not accurate but close enough for the most part.)

[gbooker]

As for now, we have been working around this problem by, (as you discovered) extracting all subs and then durning handbrake encoding, selecting the forced sub title option, (though admittedly not a perfect solution) seems to work OK for now, (forced durations are not accurate but close enough for the most part.)

Unfortunately, this is not a solution for me. I'm not handbraking my rips; I'm streaming the raw BD stream to my GoogleTV directly. I just needed to overlay the subtitles over the video (since the GTV doesn't play subtitles yet), so I wrote the code that does so. I just need a means to detect when a forced subtitle needs to cease display.

Does the method I suggested above, of inserting the a 0 composition packet, produce the correct timings for forced subtitles? It seems to be correct in my examinations. BTW, your above link is broken.

[kfreeb]

The person that really understands what your asking is, "Romansh" on this forum, or on the Handbreak forum I would talk to Rodeo, he's one of the developers on the Handbreak forums and has been helping me out with what is going on with PGS subtitles and how MMKV doesn't yet decode them corretly, (yet.)

PS: I think the link works now, give it a try!

https://forum.handbrake.fr/viewtopic.ph ... 3&start=25

I'm using an Apple TV 2, so I encode for my media server in in Mp4/m4v format. The forced subs, (foreign languages) are burned into the video with handbrake, which for my needs works, but it would seem that your needs differ from mine and is outside the scope of my understanding.

Again, Romansh on this forum, may be of help, but most people here it would seem, don't stream BD content as you do. Sorry I couldn't be of more help.

[Romansh]

The person that really understands what your asking is, "Romansh" on this forum, or on the Handbreak forum I would talk to Rodeo, he's one of the developers on the Handbreak forums and has been helping me out with what is going on with PGS subtitles and how MMKV doesn't yet decode them corretly, (yet.)

Not sure how much I can help. I'm not particularly familiar with the MKV specification.

From what I can tell, MakeMKV is ripping only the forced subtitles in forced-only mode, taking no other action, which is not producing a stream which can correctly display these subtitles.

For some reason, this happens for some discs and not others. I haven't encountered this issue with any of my BDs and MakeMKV (i.e. it keeps relevant subtitles without composition segments when doing forced-only extraction on the discs I have), but several people have reported the issue for Avatar.

First, the mkv file has no durations on the subtitles themselves. It is using [A3] SimpleBlocks to store the subtitle data, which cannot contain duration (assuming I am reading the mkv spec correctly).

Interesting. I assumed MakeMKV did set the subtitle durations at the MKV level (which, IIRC, mike_admin confirmed at some point), but I never checked it. Maybe I should.

[samtoy]

Hi all, what I really would like to know:

Is there another Blu-ray disc where this particular subtitle-problem occurs (except Avatar)?

[BrownWaffles]

Hi all, what I really would like to know:

Is there another Blu-ray disc where this particular subtitle-problem occurs (except Avatar)?

Yes, I am having this problem with John Carter on Blu Ray. The subtitle is there in the MKV, but it not recognized
and shows up as und instead of eng. MakeMKV was used.

[samtoy]

Until this problem is fixed on MakeMKV's side, is there a tool which can be used to *properly* get Avatar's forced subs out of the main subtitle stream into an extra stream?

Thanks in advance.

[Romansh]

Until this problem is fixed on MakeMKV's side, is there a tool which can be used to *properly* get Avatar's forced subs out of the main subtitle stream into an extra stream?

mkvextract, BDSup2Sub & mkvmerge GUI.

Or, if you're going to re-encode it anyway, the HandBrake nightlies can do that for you (include the full track in your MakeMKV rip, and use "Forced Only" in HandBrake - or better yet, encode directly from a "full disc backup").

[samtoy]

Thanks for the suggestion but it's not working using BDSup2Sub (.exe version).

I have demuxed the english PGS subtitles (from my Original BD) using TSMuxeR to a normal (BD) SUP file. When I open that file in BDSup2Sub.exe it already shows "246 warnings" when loading. When trying to export only the forced subs to BD SUP it shows an "Pallete ID out of bounds" error and then stops.

Maybe Avatar's subtitles are somehow not standard conform??

[Romansh]

I have demuxed the english PGS subtitles (from my Original BD) using TSMuxeR to a normal (BD) SUP file.

Then try extracting with eac3to instead, or with mkvextract on MakeMKV's output.

[Chetwood]

When I open that file in BDSup2Sub.exe it already shows "246 warnings" when loading.

Are you testing BDSUP2SUB++ Sassbot is working on? If you get the errors there too you might wanna ask about it in the official thread over at Doom9. He seems quite eager in fixing bugs/improving its capabilities. Messed up subs could help with that.