Selfhosted
A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.
Rules:
-
Be civil.
-
No spam.
-
Posts are to be related to self-hosting.
-
Don't duplicate the full text of your blog or readme if you're providing a link.
-
Submission headline should match the article title.
-
No trolling.
-
Promotion posts require active participation, with an account that is at least 30 days old. F/LOSS without a paywall has exceptions, with requirements. See the rules link for details. Tags [CBH] or [AIP] are required, see the links in Rule 8 for details.
-
AI-related discussions and AI-involved promotional posts have additional requirements for tagging, as noted in Rule 7 and the AI & Promotional Post Expanded Rules post, and find example disclosures here.
Resources:
- selfh.st Newsletter and index of selfhosted software and apps
- awesome-selfhosted software
- awesome-sysadmin resources
- Self-Hosted Podcast from Jupiter Broadcasting
Any issues on the community? Report it using the report flag.
Questions? DM the mods!
view the rest of the comments
You're given the challenge to solve by the server, yes. But just because the challenge is provided to you, that doesn't mean you can fake your way through it.
You still have to calculate the answer before you can get any farther. You can't bullshit/spoof your way through the math problem to bypass it, because your correct answer is required to proceed.
Unless the server gives you a well-known problem you have the answer to/is easily calculated, or you find a vulnerability in something like Anubis to make it accept a wrong answer, not really. You're stuck at the interstitial page with a math prompt until you solve it.
Unless I'm misunderstanding your position, I'm not sure what the disconnect is. The original question was about spoofing the challenge client side, but you can't really spoof the answer to a complicated math problem unless there's an issue with the server side validation.
@Passerby6497 my stance is that the LLM might recognize that the best way to solve the problem is to run chromium and get the answer from there, then pass it on?
Anubis has worked if that's happening. The point is to make it computationally expensive to access a webpage, because that's a natural rate limiter. It kinda sounds like it needs to be made more computationally expensive, however.
Congrats on doing it the way the website owner wants! You're now into the content, and you had to waste seconds of processing power to do so (effectively being throttled by the owner), so everyone is happy. You can't overload the site, but you can still get there after a short wait.
@Passerby6497 yes I’ve been told as much 😅
https://lemmy.world/comment/18919678
Jokes aside, I understand this was the point. I just wanted to make the point that it is feasible, if not currently economically viable
That solution still introduces lots of friction. At the volume and rate that these bots want to be traversing the internet, they probably don’t want to be fully graphically rendering pages and spawning extra browser processes then doing text recognition to then pass on to the LLM training sets. Maybe I’m wrong there, but I don’t think it’s that simple and actually just shifts solving the math challenge horizontally (i.e., in both cases, the scraper or the network the scraper is running on still has to solve the challenge)
LLMs can't just run chromium unless they're tool aware and have an agent running alongside them to facilitate tool use. I highly suspect that AI web crawlers aren't that sophisticated.