DNS/IP Filtering

[Woodbeck]

I gave recently started trying to rip UHD 4k Discs and was getting a "volume key is unknown for this disc" error for every disc I tried.

To determine if this was a DNS/IP filtering issue, I spun up a VPN for the machine and tried again.

This time I got a "downloading latest HK to c:\....." notification and the disc could be ripped.

This tells me that one of more of the URLs or IP addresses is being filtered and blocked somewhere,

I would like to test if the issue is caused by a return of NXDomain from a DNS server or a Blocked IP from an IP blocklist, so I can make necessary adjustments. However, to do this I need to know what URLs (or direct IP addresses) Makemkv is trying to access to download the latest HK.

Can anyone provide me with this list or tell me how I can get it. Makemkv logfile sheds no light on this.

Thanks

[Woodstock]

You might want to look at the other topic on this subject, started a day or so ago... viewtopic.php?f=12&t=30605

[Woodbeck]

@woodstock Yes I saw that but MakeMKV say that fairuse.org is no longer used.

I am assuming that MakeMKV must go through a simple process to determine if new HK data is available and where it is located

[Woodbeck]

Well looking at the network packets, it looks like it is looking for host1.pwhost.ru and Downloads.napp-2.com, though even though it can contact these it still doesnt work without a VPN, so I guess there is something else that is being blocked that I am not seeing.

[awdspyder]

I'm assuming you're in the US or Europe? This has been my experience as well for several months now. Frankly, I don't know why people keep saying that MakeMKV doesn't rely on fairuse.org, however. I literally see this in my firewall logs every time a disc attempts to load without a VPN on MakeMKV v1.17.3:

url="https://hkdata.fairuse.org/" referer="" error="Host not found"

This corresponds to this error:

"Automatic HK downloading is disabled or failed."

This means there's either some deprecated code that is still phoning home to hkdata.fairuse.org and things are failing for some other reason, or MakeMKV still relies on name resolution for hkdata.fairuse.org. The latter seems most likely considering a VPN resolves the issue every time.

As has been discussed here before, the hkdata subdomain has not resolved properly in the US for some time.

[Woodbeck]

@awdspyder

No actually I am in the UK. But good to know, it isnt just me.

[awdspyder]

I've seen the same issue when using VPN exit points in the UK and Europe as well. The only resolution is to use VPN exit nodes in Asia, such as from Japan. Seems to resolve fine from there.

I've asked this question multiple times without response: If we're told that https://hkdata.fairuse.org is not used by the application any longer, why is the application still using https://hkdata.fairuse.org?

Of course I get that the HK site is likely on a Russian server somewhere and there are, um, "events" currently transpiring. It was, in fact, a little over a year ago that this issue started. It then seemed to be resolved for a while, but has now been broke for a couple of months.

[theroninhunter]

Is this still ongoing?

[Billycar11]

Is this still ongoing?

Was never an issue for me keys downloads have been fine for me

[theroninhunter]

I think there are 2 problems,

1. hkdata.fairuse.org is no longer active and used, DNS name doesn't resolve.

2. My firewall geoblocks inbound/outbound traffic to Russia, hkdata.fairuse.org was an exclusion to a specific device.

Looks like my machine was trying to access this ip 185.10.186.99, I'll play around tomorrow and see if unblocking that address helps......

[TVBanks98]

I discovered a number of issues and was able to finally resolve my connection.

1) hkdata.fairuse.org resolves to an IP address in Russia and a lot of networks block traffic from certain countries. If your network blocks traffic to and from Russia, add an IP-specific exception for hkdata.fairuse.org (currently 185.84.108.20). You can use nslookup hkdata.fairuse.org to find its current IP address.

2) hkdata.fairuse.org appears to be a Dynamic DNS host name. It's IP recently changed, so I think that started the ball rolling for everyone. Again, do an nslookup to verify the current address.

3) Some networks use content filtering by category. hkdata.fairuse.org is likely categorized as a file download site or something else that may be blocked categorically on your network. You can put in an exception for hkdata.fairuse.org to bypass content filtering on that DNS name.

4) Your antivirus is probably blocking the DNS name or IP address on reputation. Mine was. This was the final piece for me. It was listed as a Dynamic DNS host name and therefore had a bad reputation. I added an exception for hkdata.fairuse.org, but found that a second host name was being blocked as well, likely due to RDNS lookup. You'll also need to put in a reputation-based exception for hkdata.crabdance.com. Once I did that I was up and running again.

Hope this helps

[Woodbeck]

In addition to @TVBanks98, there are some further issues to resolve.

I notice that Makemkv seems to do DNS queries to specific servers using DNS over HTTPS (TLS port 443), rather than use the DNS settings of the host pc. There does not seem to be a way of telling makemkv to use the pc's settings.

My router is configured (like other secure networks) to perform all external DNS queries (which it does using DNS over TLS), so it can perform DNS filtering. Consequently, it blocks all DoH and DoT queries trying to access external DNS servers. This is for network security reasons, so the router can decide if a site being resolved by the DNS query is safe (applies DNS Filtering) - allowing internal clients to bypass this is unsafe and is therefore blocked.

Additionally, any normal DNS queries (using port 53) to any external DNS servers, are redirected for the router to resolve using DoT. Hence all external unencrypted DNS queries are resolved using private and encrypted DoT.

The DNS DoH servers makemkv appears to try to use are: 1.1.1.1, 8.8.8.8 and 9.9.9.10.

I wish there was a way of changing the way makemkv does DNS resolution to use the pc's DNS configuration.

A way around this is to run a VPN when running makemkv and configure the router to allow the VPN traffic, which would also get around any issues associated with your ISP performing filtering.

[TrueCineaste]

An excellent post by TVBanks98 along with helpful resolution tips. I am still running into some roadblocks, however, so perhaps there's at least one step I'm missing? I tired the following:

Step 1) On the Start menu of my Dell XPS 15 (Windows 10), I clicked ‘Windows Firewall with Advanced Security’.
Step 2) I clicked the ‘Advanced settings’ option in the sidebar.
Step 3) On the left side, I clicked the option ‘Inbound Rules’.
Step 4) On the right, under the section ‘Actions’, I clicked on the option ‘New Rule’. Windows Firewall shows me the New Inbound Rule Wizard.
Step 5) A new window opens and I Select the ‘custom’ option and click Next.
Step 6) In the left-hand side again, I go to the option ‘Scope’.
Step 7) I added the current IP address for hkdata.fairuse.org and clicked on the ‘Ok’ button.

I am still receiving the same set of messages when I try to rip a 4K UHD disc. I don't really have an antivirus currently running that would block the DNS name or IP address. What could I be neglecting to set?

I discovered a number of issues and was able to finally resolve my connection.

1) hkdata.fairuse.org resolves to an IP address in Russia and a lot of networks block traffic from certain countries. If your network blocks traffic to and from Russia, add an IP-specific exception for hkdata.fairuse.org (currently 185.84.108.20). You can use nslookup hkdata.fairuse.org to find its current IP address.

2) hkdata.fairuse.org appears to be a Dynamic DNS host name. It's IP recently changed, so I think that started the ball rolling for everyone. Again, do an nslookup to verify the current address.

3) Some networks use content filtering by category. hkdata.fairuse.org is likely categorized as a file download site or something else that may be blocked categorically on your network. You can put in an exception for hkdata.fairuse.org to bypass content filtering on that DNS name.

4) Your antivirus is probably blocking the DNS name or IP address on reputation. Mine was. This was the final piece for me. It was listed as a Dynamic DNS host name and therefore had a bad reputation. I added an exception for hkdata.fairuse.org, but found that a second host name was being blocked as well, likely due to RDNS lookup. You'll also need to put in a reputation-based exception for hkdata.crabdance.com. Once I did that I was up and running again.

Hope this helps

[TVBanks98]

If you're running modern antivirus software, it likely has a reputation-based protection that may be blocking the site/IP.

If you're running Windows Defender, these articles will show you where to go to manage that protection.

https://support.microsoft.com/en-us/win ... f0607f7a6e

https://learn.microsoft.com/en-us/micro ... -worldwide