Recording from streaming sources
Posted: Tue Aug 29, 2023 7:21 pm
I realize this is off topic from MakeMKV, but I hope that's okay. I figure since so much content is streaming these days, and may never appear on Blu-ray, many people who care about MakeMKV might also be interested in this topic.
If you're willing to settle for 720p with stereo audio, the PlayOn Home app works pretty well. If you trim the recordings PlayOn Home creates using a video editor, the output can be pretty clean, except for things like "Skip Intro" badges appearing for a few seconds at the start of a video. A pleasant surprise is that, with all of the shows I've recorded so far, the end credits of each show are recorded in full, without shrinking into a tiny box, replaced with a full-screen banner about the next episode, as you typically see when you're watching such shows live.
The PlayOn Cloud service lets you record 1080p, but it's still only with stereo audio, and some services like Apple TV aren't available. It's also a pay-per-each-recording service, unlike PlayOn Home, and 1080p recordings cost three "credits", compared to one credit per 720p recording.
PlayOn Home has a flat subscription fee so you can record as much as you like, albeit at real-time recording speed. Both services act as virtual streaming hardware, pulling down content as any streaming media device would. That's why things like a "Skip Intro" badge may appear, and that's why it takes a full hour to record an hour-long show.
With a bit of patience and effort you can even preserve separate subtitle tracks, but that takes a bit of work with MKVCleaver, MKVToolNix, a subtitle editor, and a lot of patience to get the timing right if you trim the recordings.
I also have a way to capture 1080p content with Dolby Digital (AC-3) surround, but that comes with its own set of problems.
I'm using a Hauppauge HD PVR 2. Coupled with an HDMI video splitter which, as a side effect for this particular model of splitter, just happens to remove HDCP copy protection, I can record shows from my Apple TV hardware. The problem with recording AC-3 audio is that the PVR's recordings fail unless AC-3 is streaming at the very moment you start a recording. If you hit record before you start playing a show, the few moments of 2-channel silence that are sent out before the show starts ruin the recording.
Even if it weren't for that AC-3 problem, it's not too unusual for the first few seconds of playback to be a bit wonky until buffering settles in anyway.
At the end of each show played back from my Apple TV, as I mentioned before, the credits don't play out in full, just in case you want to capture them. The PVR also doesn't record subtitles, unless, of course, you want burned-in subtitles, and turn on subtitles on your streaming device.
The recordings from the Hauppauge are on the large size since it's hard to do real-time H.264 encoding efficiently.
As a proof-of-concept, I was able to create a recording with the best of both worlds, but it's way too much effort to be practical for normal use, unless I can find a way to automate the cumbersome process. The idea is to use the 720p stereo recording as a source for the first few seconds of each show, the end credits, and the subtitles if you want them, while retaining the bulk of a given show from the 1080p AC-3 recording.
The main obstacle to splicing these pieces together is that (at least for the Solveig software I'm using) they won't join unless both the audio and video are in identical formats.
Handbrake will upscale the 720p video to 1080p without any problem, also converting the video into H.265, the format I decided to use for the end product.
What Handbrake won't do is convert stereo audio into fake 5-channel audio. Without that conversion the start, middle, and end clips can't be joined.
I extracted the stereo audio with MKVCleaver, then used Audacity to convert from stereo to fake 5.1 channel audio (not a simple, obvious process). My audio still wasn't compatible for splicing until I also realized that the PlayOn audio was 44.1 KHz, while the AC-3 audio from the Apple TV was 48 KHz, so the PlayOn audio also needed a sampling rate conversion.
Further, I converted the PVR 1080p video into H.265. Not only did this making it compatible with the upscaled PlayOn video, but it made the recording substantially smaller, without, as far as I could tell, any noticeable degradation in quality for undergoing an extra round of lossy compression.
The end result of all of this work, once spliced together, was a very satisfying, high-quality recording, with the exception of two slight audio glitches at each splice points.
These glitches occurred because the original AC-3 audio played at a slight delay in its sync with the video content, as compared to the timing of the stereo audio. Even though the my video splicing was frame-perfect, the audio therefore was not. Since I was just creating a proof-of-concept here I didn't bother to struggle at perfecting the audio sync to the millisecond.
Even with perfect audio sync, I'd ideally want a way to crossfade the fake 5.1 audio into the real 5.1 audio over, say, 0.5-1.0 seconds.
There's no way I'll go through this much work for one episode after another after another. And perhaps 720p stereo is more than good enough for backup when a show is either discontinued, or you don't have access to streaming.
I am, however, tempted to see how much of this process I could automate. A lot of the tools I use are scriptable, although my video editor AFAIK is not. I'd also need to find or devise a way to automate finding the time sync between different video and audio sources.
Since I end up putting both video streams through an extra Handbrake re-encoding step anyway, perhaps I can find a video editor that's willing to splice mismatched video/audio sources together. The Solveig SMMVSplitter's main strength is that it only re-encodes video and audio at splice points, sparing the rest of the content from generational loss caused by unnecessary re-encoding. That feature, however, becomes less important if, through a different process, I only incur one extra re-encoding of the audio and video anyway.
If you're willing to settle for 720p with stereo audio, the PlayOn Home app works pretty well. If you trim the recordings PlayOn Home creates using a video editor, the output can be pretty clean, except for things like "Skip Intro" badges appearing for a few seconds at the start of a video. A pleasant surprise is that, with all of the shows I've recorded so far, the end credits of each show are recorded in full, without shrinking into a tiny box, replaced with a full-screen banner about the next episode, as you typically see when you're watching such shows live.
The PlayOn Cloud service lets you record 1080p, but it's still only with stereo audio, and some services like Apple TV aren't available. It's also a pay-per-each-recording service, unlike PlayOn Home, and 1080p recordings cost three "credits", compared to one credit per 720p recording.
PlayOn Home has a flat subscription fee so you can record as much as you like, albeit at real-time recording speed. Both services act as virtual streaming hardware, pulling down content as any streaming media device would. That's why things like a "Skip Intro" badge may appear, and that's why it takes a full hour to record an hour-long show.
With a bit of patience and effort you can even preserve separate subtitle tracks, but that takes a bit of work with MKVCleaver, MKVToolNix, a subtitle editor, and a lot of patience to get the timing right if you trim the recordings.
I also have a way to capture 1080p content with Dolby Digital (AC-3) surround, but that comes with its own set of problems.
I'm using a Hauppauge HD PVR 2. Coupled with an HDMI video splitter which, as a side effect for this particular model of splitter, just happens to remove HDCP copy protection, I can record shows from my Apple TV hardware. The problem with recording AC-3 audio is that the PVR's recordings fail unless AC-3 is streaming at the very moment you start a recording. If you hit record before you start playing a show, the few moments of 2-channel silence that are sent out before the show starts ruin the recording.
Even if it weren't for that AC-3 problem, it's not too unusual for the first few seconds of playback to be a bit wonky until buffering settles in anyway.
At the end of each show played back from my Apple TV, as I mentioned before, the credits don't play out in full, just in case you want to capture them. The PVR also doesn't record subtitles, unless, of course, you want burned-in subtitles, and turn on subtitles on your streaming device.
The recordings from the Hauppauge are on the large size since it's hard to do real-time H.264 encoding efficiently.
As a proof-of-concept, I was able to create a recording with the best of both worlds, but it's way too much effort to be practical for normal use, unless I can find a way to automate the cumbersome process. The idea is to use the 720p stereo recording as a source for the first few seconds of each show, the end credits, and the subtitles if you want them, while retaining the bulk of a given show from the 1080p AC-3 recording.
The main obstacle to splicing these pieces together is that (at least for the Solveig software I'm using) they won't join unless both the audio and video are in identical formats.
Handbrake will upscale the 720p video to 1080p without any problem, also converting the video into H.265, the format I decided to use for the end product.
What Handbrake won't do is convert stereo audio into fake 5-channel audio. Without that conversion the start, middle, and end clips can't be joined.
I extracted the stereo audio with MKVCleaver, then used Audacity to convert from stereo to fake 5.1 channel audio (not a simple, obvious process). My audio still wasn't compatible for splicing until I also realized that the PlayOn audio was 44.1 KHz, while the AC-3 audio from the Apple TV was 48 KHz, so the PlayOn audio also needed a sampling rate conversion.
Further, I converted the PVR 1080p video into H.265. Not only did this making it compatible with the upscaled PlayOn video, but it made the recording substantially smaller, without, as far as I could tell, any noticeable degradation in quality for undergoing an extra round of lossy compression.
The end result of all of this work, once spliced together, was a very satisfying, high-quality recording, with the exception of two slight audio glitches at each splice points.
These glitches occurred because the original AC-3 audio played at a slight delay in its sync with the video content, as compared to the timing of the stereo audio. Even though the my video splicing was frame-perfect, the audio therefore was not. Since I was just creating a proof-of-concept here I didn't bother to struggle at perfecting the audio sync to the millisecond.
Even with perfect audio sync, I'd ideally want a way to crossfade the fake 5.1 audio into the real 5.1 audio over, say, 0.5-1.0 seconds.
There's no way I'll go through this much work for one episode after another after another. And perhaps 720p stereo is more than good enough for backup when a show is either discontinued, or you don't have access to streaming.
I am, however, tempted to see how much of this process I could automate. A lot of the tools I use are scriptable, although my video editor AFAIK is not. I'd also need to find or devise a way to automate finding the time sync between different video and audio sources.
Since I end up putting both video streams through an extra Handbrake re-encoding step anyway, perhaps I can find a video editor that's willing to splice mismatched video/audio sources together. The Solveig SMMVSplitter's main strength is that it only re-encodes video and audio at splice points, sparing the rest of the content from generational loss caused by unnecessary re-encoding. That feature, however, becomes less important if, through a different process, I only incur one extra re-encoding of the audio and video anyway.