Super slooooooooooowwwww site

[privacy_user_mkv]

I also noticed the site being unresponsive. I actually work in tech on large websites. Maybe I could help!

[Woodstock]

Possibly, but...

Part of the problem seems to be Cloudflare, and a big part seems to be that thousands of individual IPs sometimes seem to attack the forum. For example, currently, there are 3600 "guests" (unlogged in users) accessing the page. In terms of people, we're talking maybe 50 to 100, the others are machines doing various things. This doesn't include the Google BOT indexing the forum... it's relatively benign.

Cloudflare has some major screw-ups attributed to it from time to time, but the site itself is on a shared hosting server that has multiple clients to serve, so there is a limit to what you can get for an individual client.

[kaysee]

The Cloudflare error pages that were displayed over most of yesterday indicated that the connection through Cloudflare was fine, but the forum server was not responding. That suggests that Cloudlare was responding normally, and probably fighting DDOS attacks as usual, but the forum server was having problems.

[AstralWanderer]

Been encountering a cluster of Cloudflare 522 and 525 errors recently, with their error page pointing the finger at forum.makemkv.com.

Given the large number of reported visitors, I had a look at makemkv's robots.txt file - and, aside from comments, it's empty. No restrictions or limitations on search engine spiders or AI scrapers at all. Now there are presumably restrictions set via Cloudflare, but having some in robots.txt would reduce initial connection attempts (at least from well behaved crawlers).

So cutting and pasting another robots.txt, say from the example listed on Using robots.txt in order to optimize your site performance and reduce your website load (apologies if this seems somewhat spammy and WordPress focused, but with others focused on SEO/marketing, this seemed the least worst) could help matters significantly.

[AstralWanderer]

Quick update - forum performance is back to normal - so whatever the problem was, seems to have been fixed. Thanks.

[zittrig]

Yesterday in the evening the forum was like a turn signal: Works, works not, works, works not, works .....

[Coopervid]

Same thing again now.

[flojo]

robots.txt won't help, in fact it never did. At least 1 Google bot always scraps without identifying as a bot, but they might not serve the results based on robots.txt (depending on advertising). Ever wonder how Google knows a site is relevant, but still summarizes the site something like "This website has chosen not to show..."? So the Google bot reads robots.txt, but Google could only know it's a relevant site because Google scraped it.

With everyone and their mother perpetually and for the rest of time re-re-re-re-training AI with they're own licensed agent software, I can only imagine how frustrating it is to admin a site. Of course we all get to feel all websites slowing down because of it, now and forever.

[zittrig]

Hooray, it's working again... for now ....

[DennisOlof]

What it going on. I use a lot of different sites and forums but I have never experienced a forum that is this slow or have so much problems. I am starting to think there is something that is sucking resources etc. Is anyone investigating this, I think something needs to be done to get this forum working better.

I like the forum, it is sad that from time to time it is so sloooooow, timeout etc.

[Billycar11]

What it going on. I use a lot of different sites and forums but I have never experienced a forum that is this slow or have so much problems. I am starting to think there is something that is sucking resources etc. Is anyone investigating this, I think something needs to be done to get this forum working better.

I like the forum, it is sad that from time to time it is so sloooooow, timeout etc.

It is very sad wish whatever is causing this wold stop.
I believe it's ai crawlers just hammering it but not sure could be other stuff too

[DennisOlof]

@Billycar11 yes there is something going on.

Sure, forums could be slow from time to time, hardware issues etc but this forum is beyond that. Someone need to look into this, or get help from someone who can investigate this. When the forum is normal it is fine but this dead crawl and timeout is crazy, it is not normal and it should be possible to find out what is the cause and fixing it.

If you have to put up limits, block things etc then do it. The forum is hardly usable when it is that crazy slow.