cross-posted from: https://discuss.online/post/32165111

I realize my options are limited, but what about any robots.txt style steps? Thanks for any suggestions.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 10 months ago (1 child)

A big issue is that this works for bots that announce themselves as such, but there's lots that pretend to be regular users, with fake user agents and ips selected from a random pool with each ip only sending like 1-3 request/day, but overall many thousands of requests. In my experience a lot of them are from huawei and tencent cloud/ASN

  • source
  • parent
  • hideshow 1 child comment