All about open source! Feel free to ask questions, and share news, and interesting stuff!
Community icon from opensource.org, but we are not affiliated with them.
I'm no fan of AI generally, but "AI Vulnerable" as a term just doesn't make much sense to me. Code reviewing should be filtering out bad code whether it originates from an AI or a human.
PR spamming with the usage of AI is another problem which is very serious and harmful for OSS, but that's not due to some unique danger that only AI code has and human contributors don't.
Code reviewing should be filtering out bad code whether it originates from an AI or a human.
But studies are showing it doesn't work.
A human makes a mental model of the entire system, does some testing, and submits code that works, passes tests, and fits their unstanding of what is need.
A present day AI makes an educated guess which existing source code snippets best match the request, does some testing, and submits code that it judges is most likely to pass code review.
And yes, plenty of human coders fall into the second bracket, as well.
But AI is very good at writing code that looks right. Code review is a good and necessary tool, but the data tells us code review isn't solving the problem of bugs introduced by AI generated code.
I don't have an answer, but "just use code review" probably isn't it. In my opinion, "never use AI code assist" also isn't the answer. There's just more to learn about it, and we should proceed with drastically more caution.
A present day AI makes an educated guess which existing source code snippets best match the request, does some testing, and submits code that it judges is most likely to pass code review.
That's still on the human that opened the PR without doing the slightest effort of testing the AI changes though.
I agree there should be a lot of caution overall, I just think that the problem is a bit mischaracterized. The problem is the newfound ability to spam PRs that look legit but are actually crap, but the root here is humans doing this for Github rep or whatever, not AI inherently making codebases vulnerable. There need to be ways to detect such users that repeatedly do zero effort contributions like that and ban them.
Yes, it is their fault, and also, that fault is a widespread problem
That's still on the human that opened the PR without doing the slightest effort of testing the AI changes though.
That makes sense when talking about people's accounts.
A "Claude" account serves PR (as in public relations) purposes, and having to do a stringent human review before submitting a pull request is bad for PR.
Which by no means is me saying submissions from the Claude account need to be banned, but that the "Claude" account's goals are probably to have Claude do all of this "himself" - which is a recipe for disaster.
I think Claude account PRs should absolutely be banned, that's the easiest counter measure to implement too
Here's an example I ran into, since work wants us to use AI to produce work stuff, whatever, they get to deal with the result.
But I had asked it to add some debug code to verify that a process was working by saving the in memory result of that process to a file, so I could ensure the next step was even possible to do based on the output of the first step (because the second step was failing). Get the file output and it looks fine, other than missing some whitespace, but that's ok.
And then while debugging, it says the issue is the data for step 1 isn't being passed to the function the calls if all. Wait, how can this be, the file looks fine? Oh when it added the debug code, it added a new code path that just calls the step 1 code (properly). Which does work for verifying step 1 on its own but not for verifying the actual code path.
The code for this task is full of examples like that, almost as if it is intelligent but it's using the genie model of being helpful where it tries to technically follow directions while subverting expectations anywhere it isn't specified.
Thinking about my overall task, I'm not sure using AI has saved time. It produces code that looks more like final code, but adds a lot of subtle unexpected issues on the way.
It produces code that looks more like final code, but adds a lot of subtle unexpected issues on the way.
That is an excellent summary of the challenge. The code looks high quality sooner in the debug lifecycle, which actually makes debugging a little bit slower, at least with our current tools.
Yeah, it's good enough that it even had me fooled, despite all my "it just correlates words" comments. It was getting to the desired result, so I was starting to think that the framework around the agentic coding AIs was able to give it enough useful context to make the correlations useful, even if it wasn't really thinking.
But it's really just a bunch of duct tape slapped over cracks in a leaky tank they want to put more water in. While it's impressive how far it has come, the fundamental issues will always be there because it's still accurate to call LLMs massive text predictors.
The people who believe LLMs have achieved AGI are either just lying to try to prolong the bubble in the hopes of actually getting it to the singularity before it pops or are revealing their own lack of expertise because they either haven't noticed the fundamental issues or think they are minor things that can be solved because any instance can be patched.
But a) they can only be patched by people who know the correction (so the patches won't happen in the bleeding edge until humans solve the problem they wanted AI to solve), and b) it will require an infinite number of these patches even to just cover all permutations of everything we do know.
why is cpython on github? I thought they had their own forge like GNOME and KDE
The only link listed on wikipedia is on github
They moved to GitHub a few years ago, mostly for the benefits of issue tracking, which previously was not integrated in the forge IIRC.
I don't code so correct me if I'm wrong, but wouldn't the code have to be generally accepted, reviewed, and verified by other members of the project? Ai can fuck right off as far as I'm concerned, but this isn't a situation where a CEO just unilaterally decides vibe coding is the move. Unless I'm mistaken.
Yeah. The real problem with AI generated code in open source projects is people flooding projects with slop merge requests.
What should be done?
Waybe it's save if the maintainer reviews PRs before merging
The problem is they get overwhelmed with these PRs. Godot has been talking about not being able to manage the workload lately, people just task AIs to vibecode fixes to perceived bugs and half of them don’t even do what they were prompted to do.
You can block those users but they just make new accounts
It honestly feels like a DDoS on do it yourself computing, by corporations who want total control over our thoughts.
I can't wait for the money to dry up. It's insane to me just how stupid people have been, trusting LLMs with anything whatsoever. These things cost so much money to run and they seem to fucking hypnotize investors into burning their money. Sooner or later the fact that they're not making money has to catch up with them, right?
Yet somehow there's still a ton of money in web3/crypto. It'll be a long time before the AI money dries up.
Thank for the explanation! The user in the image is claude itself, not a random anonymuous user. I see the problem of the ddos with issues, tickets etc. that is a real problem! But I don't get the rigid denial of generative ai. As long as I review the code it generates, it can save me lots of time. I would hate the actions you described as well but the image depicts nothing fishy. Am I wrong about this?
Slop PRs are submitted by users, not by a Claude bot like this screenshot refers to
It may be good to allow repo owners reputation-based PR filters based on account age, history, rejected PRs etc.
And maybe the janitor should sift through that river of diarrhea for the couple of pennies someone might have swallowed.
But if you go to the repo, and search all pull requests by author, nothing comes up for claude
I think when one uses Claude Code CLI the PR is attributed to the human user but Claude adds itself as an author to the code when it is committed.
Context?
cpython is the reference implementation of the python interpreter. The person who took this screenshot has the Claude user on GitHub blocked so that whenever it contributed to a git repo you see this warning. The Claude user is an AI agent. AI code is garbage.
That was as to the point as humanly possible. Thank you.
I'm not understanding how people know that this is an ai agent. Do people really just have no lives so they dig into it more?
Because most people have at least tried these products and it’s not like the agent is being hidden away. It makes commits to your repo using its name as the author (for now).
Do you assume anyone who knows something you don’t has no life? Strange way to live.
Claude is a common name. I would assume that someone has already had an account named that since the very beginning of github.
ok well, the people who use github and write programs are probably a lot more likely to have experience with claude than you are so they probably didn't make the same assumptions you did!
sadge
I mean it's Python. This is what we get for having been overly reliant on it.
All kidding aside, I am a more than a bit confused by this.
Maybe I don't understand what this warning means, but I don't see anything here: https://github.com/python/cpython/issues?q=is%3Apr+author%3Aclaude
Does this mean something else?
If you block Claude, or any user really, and then visit a repo they've contributed to you will see this message.
Maybe Claude didn't open the PR but contributed commits.
I tried a git log --grep=claude but it doesn't net much, basically just this PR (which in fairness does look vibecoded).
Maybe there's some development branch in the repository that has a commit authored by Claude but if so it's not on main.
And now Anthropic has no guardrails, meaning that if the US government demands a backdooe into Python itself...
warclaude is Coming
What is "AI vulnerable"? What is the problem here? Claude isn't reverse-Midas, it's not like everything they touch turns to shit.
Studies continue to show that AI routinely generates unsafe code and even human code reviews often don’t catch major problems. AI generated code should not be trusted or accepted and projects that accept them should be treated as compromised.
Humans can barely write safe C code, so I definitely don't trust AI to. I'm not even blanket against AI assistance in programming, but there are way too many hidden landmines in C for an LLM to be reliable with.