222
Why are anime catgirls blocking my access to the Linux kernel? (lock.cmpxchg8b.com)
submitted 1 year ago by to c/selfhosted@lemmy.world
117 comments fedilink hide all child comments
 

Some thoughts on how useful Anubis really is. Combined with comments I read elsewhere about scrapers starting to solve the challenges, I'm afraid Anubis will be outdated soon and we need something else.

you are viewing a single comment's thread
view the rest of the comments
[–] 97 points 1 year ago* (36 children)

And it was/is for sure the lesser evil compared to what most others did: put the site behind Cloudflare.

I feel people that complain about Anubis have never had their server overheat and shut down on an almost daily basis because of AI scrapers 🤦

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 36 child comments
    •
  • [–] [S] 18 points 1 year ago* (5 children)

    Yeah, I'm just wondering what's going to follow. I just hope everything isn't going to need to go behind an authwall.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 39 points 1 year ago (4 children)

    The developer is working on upgrades and better tools. https://xeiaso.net/blog/2025/avoiding-becoming-peg-dependency/

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 22 points 1 year ago (2 children)

    I'll say the developer is also very responsive. They're (ambiguous 'they', not sure of pronouns) active in a libraries-fighting-bots slack channel I'm on. Libraries have been hit hard by the bots: we have hoards of tasty archives and we don't have money to throw resources at the problem.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 5 points 1 year ago (6 children)

    Is there a reason other than avoiding infrastructure centralization not to put a web server behind cloudflare?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 25 points 1 year ago

    Yes, because Cloudflare routinely blocks entire IP ranges and puts people into endless captcha loops. And it snoops on all traffic and collects a lot of metadata about all your site visitors. And if you let them terminate TLS they will even analyse the passwords that people use to log into the services you run. It's basically a huge survelliance dragnet and probably a front for the NSA.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 10 points 1 year ago (4 children)

    Cloudflare would need https keys so they could read all the content you worked so hard to encrypt. If I wanted to do bad shit I would apply at Cloudflare.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 6 points 1 year ago* (3 children)

    Maybe I'm misunderstanding what "behind cloudflare" means in this context, but I have a couple of my sites proxied through cloudflare, and they definitely don't have my keys.

    I wouldn't think using a cloudflare captcha would require such a thing either.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 16 points 1 year ago (1 child)

    That's because they just terminate TLS at their end. Your DNS record is "poisoned" by the orange cloud and their infrastructure answers for you. They happen to have a trusted root CA so they just present one of their own certificates with a SAN that matches your domain and your browser trusts it. Bingo, TLS termination at CF servers. They have it in cleartext then and just re-encrypt it with your origin server if you enforce TLS, but at that point it's meaningless.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 5 points 1 year ago* (last edited 1 year ago)

    Hmm, I should look up how that works.

    Edit: https://developers.cloudflare.com/ssl/origin-configuration/ssl-modes/#custom-ssltls

    They don't need your keys because they have their own CA. No way I'd use them.

    Edit 2: And with their own DNS they could easily route any address through their own servers if they wanted to, without anyone noticing. They are entirely too powerful. Is there some way to prevent this?

  • permalink
  • fedilink
  • source
  • parent
  • [–] 0 points 1 year ago (1 child)

    Out of curiosity, what’s the issue with Cloudflair? Aside from the constant worry they may strong arm you into their enterprise pricing if you’re site is too popular lol. I understand support open source, but why not let companies handle the expensive bits as long as they’re willing?

    I guess I can answer my own question. If the point of the Fediverse is to remove a single point of failure, then I suppose Cloidflare could become a single point to take down the network. Still, we could always pivot away from those types of services later, right?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] -1 points 1 year ago

    Cloudflare has IP banned me before for no reason (no proxy, no VPN, residential ISP with no bot traffic). They've switched their captcha system a few times, and some years it's easy, some years it's impossible.

  • permalink
  • fedilink
  • source
  • parent
  • [–] -4 points 1 year ago (6 children)

    I still think captchas are a better solution.

    In order to surpass them they have to run AI inference which is also comes with compute costs. But for legitimate users you don't run unauthorized intensive tasks on their hardware.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 10 points 1 year ago (5 children)

    They are much worse for accessibility, and also take longer to solve and are more distruptive for the majority of users.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] -5 points 1 year ago* (4 children)

    Anubis is worse for privacy. As you have to have JavaScript enabled. And worse for the environment as the cryptographic challenges with PoW are just a waste.

    Also reCaptcha types are not really that disturbing most of the time.

    As I said, the polite thing you just be giving users the options. Anubis PoW running directly just for entering a website is one of the most rudest piece of software I've seen lately. They should be more polite, and just give an option to the user, maybe the user could chose to solve a captcha or run Anubis PoW, or even just having Anubis but after a button the user could click.

    I don't think is good practice to run that type of software just for entering a website. If that tendency were to grow browsers would need to adapt and straight up block that behavior. Like only allow access to some client resources after an user action.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 15 points 1 year ago* (3 children)

    Are you seriously complaining about an (entirely false) negative privacy aspect of Anubis and then suggest reCaptcha from Google is better?

    Look, no one thinks Anubis is great, but often it is that or the website becoming entirely inaccessible because it is DDOSed to death by the AI scrapers.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] -4 points 1 year ago (2 children)

    First, I said reCaptcha types, meaning captchas of the style of reCaptcha. That could be implemented outside a google environment. Secondly, I never said that types were better for privacy. I just said Anubis is bad for privacy. Traditional captchas that work without JavaScript would be the privacy friendly way.

    Third, it's not a false proposition. Disabling JavaScript can protect your privacy a great deal. A lot of tracking is done through JavaScript.

    Last, that's just the Anubis PR slogan. Not the truth, as I said ddos mitigation could be implemented in other ways. More polite and/or environmental friendly.

    Are you astrosurfing for anubis? Because I really cannot understand why something as simple as a landing page with a button "run PoW challenge" would be that bad

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 6 points 1 year ago (1 child)

    Anubis is not bad for privacy, but rather the opposite. Server admins explicitly chose it over commonly available alternatives to preserve the privacy of their visitors.

    If you don't like random Javascript execution, just install an allow-list extension in your browser 🤷

    And no, it is not a PR slogan, it is the live experience of thousands of server admins (me included) that have been fighting with this for month now and are very grateful that Anubis has provided some (likely only temporary) relief from that.

    And I don't get what the point of an extra button would be when the result is exactly the same 🤷

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 5 points 1 year ago

    Latest version of Anubis has a JavaScript-free verification system. It isn't as accurate, so I allow js-free visits only if the site isn't being hammered. Which, tbf, prior to Anubis no one was getting in, JS or no JS.

  • permalink
  • fedilink
  • source
  • parent
  • [+] -13 points 1 year ago (13 children)

    Unless you have a dirty heatsink, no amount of hammering would make the server overheat

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 13 child comments
    •
  • [–] 11 points 1 year ago (12 children)

    Are you explaining my own server to me? 🙄

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 12 child comments
    •
  • [–] -2 points 1 year ago (11 children)

    What CPU do you have made after 2004 that doesn't have automatic temperature control ?
    I don't think there is any, unless you somehow managed to disable it ?
    Even a raspberry pi without a heatsink won't overheat to shutdown

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 11 child comments
    •
  • [–] 10 points 1 year ago (10 children)

    You are right, it is actually worse, it usually just overloads the CPU so badly that it starts to throttle and then I can't even access the server via SSH anymore. But sometimes it also crashes the server so that it reboots, and yes that can happen on modern CPUs as well.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 10 child comments
    •
  • [–] -4 points 1 year ago (9 children)

    You need to set you http serving process to a priority below the administrative processes (in the place where you are starting it, so assuming linux server that would be your init script or systemd service unit).

    Actual crash causing reboot ? Do you have faulty ram maybe ? That's really not ever supposed to happen from anything happenning in userland. That's not AI, your stuff might be straight up broken.

    Only thing that isn't broken that could reboot a server is a watchdog timer.

    You server shouldn't crash, reboot or become unreachable from the admin interface even at 100% load and it shouldn't overheat either, temperatures should never exceed 80C no matter what you do, it's supposed to be impossible with thermal management, which all processors have had for decades.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [–] 2 points 1 year ago (8 children)

    Great that this is all theoretical 🤷 My server hardware might not be the newest but it is definitly not broken.

    And besides, what good is that you can still barely access the server through ssh, when the cpu is constantly maxed out and site visitors only get a timeout when trying to access the services?

    I don't even get what you are trying to argue here. That the AI scraper DDOS isn't so bad because in theory it shouldn't crash the server? Are you even reading what you are writing yourself? 🤡

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 1 point 1 year ago* (1 child)

    Why the hell don't you limit the CPU usage of that service?

    For any service that could hog resources so bad that they can block the entire system the normal thing to do is to limit their max resource usage. This is trivial to do using containers. I do it constantly for leaky software.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] -4 points 1 year ago (5 children)

    Even if your server is a cell phone from 2015, if it's operating correctly and the cpu is maxed out, that means it's fully utilized and services hundreds of megabits of information.

    You've decided to let the entire world read from your server, that indiscriminatory policy is letting people you don't want getting your data, get your data and use your resources.

    You want to correct that by making everyone that comes in solve a puzzle, therefore in some way degrading their access, it's not surprising that they're going to complain. The other day I had to wait over 30 second at an anubis puzzle page, when I know that the AI scrappers have no problem getting through, something on my computer, probably some anti-crypto mining protection is getting triggered by it and now I can't no-script the web either because of that thing and it can't even stop scrappers anyway !

    So, anubis is going to be left behind, all the real users are, for years, going to be annoyed and have their entire internet degraded by it while the scrappers got that institutionally figured out in days.

    If it's freely available public data then the solution isn't restricting access trying to play a futile arms race with the scrapper and throwing the real users to the dogs, it's to have standardized incremental efficient database dumps so the scrappers stop assuming every website is interoperability-hostile and scrape them. Let facebook and xitter fight the scrappers, let anyone trying to leverage public (and especially user contributed data) fight the scrappers.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] [S] 3 points 1 year ago (1 child)

    Even if one would want to give them everything, they don't care. They just burn through their resources and recursively scrape every single link on your page. Providing standardized database dumps is absolutely not helping against your server being overloaded by scrapers of various companies with deep pockets.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] -1 points 1 year ago

    Like anubis, that's not going to last, the point isn't to hammer the web servers off the net, it's to get the precious data. The more standardized and streamlined that's going to be made and only if there's no preferential treatment to certain players (open ai / google facebook) then the dumb scraper will burn themselves out.

    One nice thing about anubis and nepenthes is that it's going to burn out those dumb scrapers faster and force them to become more sophisticated and stealth. That's should resolve the ddos problem on its own.

    For the truly public data sources, I think coordinated database dumps is the way to go, for hostile carrier, like reddit and facebook, it's going to be scrapper arms race warfare like Cory Doctorow predicted.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 1 year ago (2 children)

    Aha, an apologist for AI scraper DDOS, why didn't you say so directly instead of wasting my time?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] -3 points 1 year ago (1 child)

    The ddos is caused by the gatekeeping, there was no such issue before the 2023 API wars, fork over the goods and nobody gets hurt, it's not complicated, you want to publish information to the public, don't scrunch it up behind diseased trackers and ad infested pages which burn you cpu cycles. Or just put it in a big tarball torrent, the web is turning into a cesspool, how long until our browsers don't even query websites at all but self-hosted crawler and search like searxng, at least then I won't be catching cooties from your javascript cryptomining bots embed into the pages !

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •