Amazon’s ecommerce business has summoned a large group of engineers to a meeting on Tuesday for a “deep dive” into a spate of outages, including incidents tied to the use of AI coding tools.

The online retail giant said there had been a “trend of incidents” in recent months, characterized by a “high blast radius” and “Gen-AI assisted changes” among other factors, according to a briefing note for the meeting seen by the FT.

Under “contributing factors” the note included “novel GenAI usage for which best practices and safeguards are not yet fully established.”

top 50 comments

sorted by: hot top controversial new old
[–] 286 points 6 months ago

"Everyone must use AI."

...

"No! Not like that!"

  • source
  • [–] 178 points 6 months ago (23 children)

    Well that's going to make your senior developers quit.

  • source
  • hideshow 23 child comments
  • [–] 120 points 6 months ago* (8 children)

    Exactly. If you're too stupid or lazy to adequately vet what your LLM puts out yourself, it shouldn't be somebody else's job to wade through the sewage you're producing. You either shouldn't be using one or, if you can't do your job without it, you shouldn't have that job.

    —Someone who doesn't use genAI but has spent way too much time digging through LLM slop

  • source
  • parent
  • hideshow 8 child comments
  • [–] 19 points 6 months ago*

    I mean honestly yeah, I'm not going to waste my time with some junior developer who can't explain how the code works and how it interacts with whatever framework I'm working on. I ain't got time for that nonsense, especially when the code I deal with involves safety critical sections of code.

    Honestly if my work ever decided to allow unfettered AI code generation into my code base, I would immediately look for a new job at that point.

  • source
  • parent
  • [–] 56 points 6 months ago (2 children)

    I am not a developer, but:

    I told the owner of the company recently that, and I quote, "I will fucking kill myself if my job becomes reviewing AI output"

  • source
  • parent
  • hideshow 2 child comments
  • [–] 29 points 6 months ago (7 children)

    It's going to make snr devs get fired, surely?

    They either refuse to sign off when boss wants them to and get fired or sign off and get fired when ai code they signed off on causes issues.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 26 points 6 months ago (3 children)

    Bingo.

    Maybe not outright fired, but absolutely open them up to career limits based on what you described.

    All of Amazon's code undergoes code reviews already. Accepting a PR is already spiritually a sign off.

    This is just explicitly a threat, explicitly trying to find someone to hold accountable because you can't hold ai accountable. What are they gonna do, fire the ai? Sign here to be the fall guy. Fuck off.

  • source
  • parent
  • hideshow 3 child comments
  • load more comments (3 replies)
  • [–] 17 points 6 months ago

    Or quit/find new jobs. I suspect that's by design by Business Idiots.

    *Get rid of the most expensive engineers and the cheaper ones can just use AI to make up the difference in output. And we can make the lower engineers the fall guy when convenient and replace them at our leisure *

    The disdain bosses have for average people is astonishing.

  • source
  • parent
  • load more comments (2 replies)
  • [–] 16 points 6 months ago

    it's pretty fucking stark right? these are the devs that stayed after management mandated they USE the shit in the first place, now they want the same devs to become responsible for what the shit does to their codebases.

  • source
  • parent
  • [–] 148 points 6 months ago (7 children)

    Junior and mid-level engineers will now require more senior engineers to sign off any AI-assisted changes, Treadwell added.

    So instead of getting a human to write it and AI peer reviewing it you want the most expensive per hour developers to look at stuff a human didn't write and the other engineers can't explain? Yeah, this is where the efficiency gains disappear.

    I read stuff from one of my Jr's all the time and most of it is made with AI. I don't understand most of it and neither does the Dev. He keeps saying how much he's learned from AI but peer programming with him is the pits. I try to say stuff like, "Oops! Looks like we forgot the packages." And then 10 secs of silence later, "So you can go to line 24 and type..."

  • source
  • hideshow 7 child comments
  • [–] 24 points 6 months ago* (2 children)

    Just to add to this:

    • When a senior dev reviews code from a more junior dev and gives feedback the more junior person (generally) learns from it.
    • When a senior dev reviews code from an AI, the AI does not learn from it.

    So beyond the first order effects you pointed out - the using of more time from more experience and hence expensive people - there is a second order effect due of loss of improvement in the making of code which is both persistent and cumulative with time: every review and feedback of the code from a junior dev reduces forever the future need for that, whilst every review and feedback of the code from an AI has no impact at all in need for it in the future.

    Given enough time, the total time wasted in reviews and feedback for code from junior devs is limited - because they eventually learn enough not to do such mistakes - but the total time wasted in reviews and feedback for code from an AI is unlimited - because it will never improve.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 21 points 6 months ago

    Lol I would be your Jr, except instead of 10 seconds of silence it would be 10 seconds of me frantically clacking on the keyboard "add a block to this for these packages with proper syntax, I forgot to include it" to claude. Then I'd of course be all discombobulated and shit so I wouldn't even bother to open code, I'd just ctrl-c about 100 lines somewhere around the general area of where I think the new code block should go, then ctrl-v the whole thing into the chat box because why not the company is paying out the dick for these tokens so might as well use them.

    And two weeks later half our website crashes which results in you having to go to a meeting where management tells you to keep a closer eye on me. Which is basically what you had been already doing before AI but now you get to babysit me and claude!

  • source
  • parent
  • load more comments (3 replies)
    [–] 81 points 6 months ago (16 children)

    ...

    Do the senior engineers NOT sign off on changes to systems that can take down the production servers? Even if we take out the LLM created code, this sounds like a bigger problem

  • source
  • hideshow 16 child comments
  • [–] 41 points 6 months ago (9 children)

    We may start to see people realize that "have the AI generate slop, humans will catch the mistakes" actually is different from "have humans generate robust code."

  • source
  • parent
  • hideshow 9 child comments
  • [–] 29 points 6 months ago (7 children)

    Not only that, but writing code is so much easier than understanding code you didn't write. Seems like either you need to be able to trust the AI code, or you're probably better of writing it yourself. Maybe there's some simple yet tedious stuff, but it has to be simple enough to understand and verify faster than you could write it. Or maybe run code through AI to check for bugs and check out any bugs it finds…

    I definitely have trusted AI to write miniature pointless little projects - like a little PHP page that loaded music for the current directory and showed a simple JS player in a webpage so I could share Christmas music with my family and friends. No database, no file uploading or anything. It worked decently, although not perfectly, and that's all it needed to do.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 13 points 6 months ago* (last edited 6 months ago)

    This is true not just with code, but with many types of complex outputs. Going through and fixing somebody’s horrible excel model is much worse than building a good one yourself. And if the quality is really bad, it’s also just faster to do it yourself from scratch.

  • source
  • parent
  • load more comments (6 replies)
  • load more comments (1 reply)
  • [–] 29 points 6 months ago (4 children)

    I guarantee there's so much pressure on those engineers to deliver code that they rubber stamp a ton of it with the intention of "fixing it later"

    Source: I've worked in software for 20+ years and know a lot of folks working for and who have worked for Amazon

  • source
  • parent
  • hideshow 4 child comments
  • load more comments (1 reply)
  • load more comments (1 reply)
    [–] 65 points 6 months ago* (7 children)

    Yes, so now when there's a success, it gets attributed to AI. When there's an outage, that's the fault of humans not reviewing correctly. These senior engineers will get fucked in all scenarios.

  • source
  • hideshow 7 child comments
  • [–] 43 points 6 months ago* (6 children)

    Precisely. From Cory Doctorow's latest, very insightful essay on AI, where he talks about the promise of AI replacing 9 out of 10 radiologists:

    "if the AI misses a tumor, this will be the human radiologist's fault, because they are the 'human in the loop.' It's their signature on the diagnosis."

    This is a reverse centaur, and it's a specific kind of reverse-centaur: it's what Dan Davies calls an "accountability sink." The radiologist's job isn't really to oversee the AI's work, it's to take the blame for the AI's mistakes.

  • source
  • parent
  • hideshow 6 child comments
  • load more comments (6 replies)
  • [–] 57 points 6 months ago (7 children)

    What is AI good at? Creating thousands of lines of code that look plausibly correct in seconds.

    What are humans bad at? Reviewing changes containing thousands of lines of plausibly correct code.

    This is a great way to force senior devs to take the blame for things. But, if they actually want to avoid outages rather than just assign blame to them, they'll need to submit small, efficient changes that the submitter understands and can explain clearly. Wouldn't it be simpler just to say "No AI"?

  • source
  • hideshow 7 child comments
  • [–] 20 points 6 months ago* (last edited 6 months ago)

    AI's greatest feature in the eyes of the Epstein class is the ability to shift responsibility. People will do all kinds of fucked up shit if they can shift the blame to someone else, and AI is the perfect bag holder.

    Just ask the school of little girls in Iran which were likely targets picked by AI with out of date information about it being a barracks. Why bother confirming the target with current intel from the ground when no one's going to take the blame anyway?

  • source
  • parent
  • load more comments (6 replies)
    [–] 49 points 6 months ago (7 children)

    "Huge rich company responsible for hosting like half of the fucking internet spent the last year pushing code to global-scale production without so much as a review by a senior engineer."

    That's how I read that headline.

  • source
  • hideshow 7 child comments
  • [–] 38 points 6 months ago (1 child)

    How in the glorious fuck was this not a thing from the start? In a system this big and this critical all code should be reviewed by cognizant individuals. Anyone who thought an LLM would be perfect and not need code reviews has their heads so far up their asses they can see through their pee hole.

  • source
  • hideshow 1 child comment
  • [–] 15 points 6 months ago

    If you do this, you signal the AI isn't ready for production capabilities, which limits your sales groups capability to market it. Which is in reality the actual case and AI sucks and should never be trusted.

  • source
  • parent
  • [–] 37 points 6 months ago (1 child)

    I always saw a code review like a dissertation defense. Why did you choose to implement the requirement in this way? Answers like 'I found a post on Stackoverflow' or 'the AI told me to' would only move the question back one step; why did you choose to accept this answer?
    I was a very unpopular reviewer.

  • source
  • hideshow 1 child comment
  • [–] 24 points 6 months ago (1 child)

    The way AI is being pushed onto workers on a global scale has to be the dumbest thing to ever happen in the work space. Executives are getting hysterical over something they don't even try to understand and even governments shower companies in subsidies if they do anything with AI. Of course the only result so far are mass layoffs and exploding costs for energy and hardware. All the while economies are crumbling everywhere because of course they do when mass unemployment sweeps around the globe. And again, governments everywhere are subsiding this crap with tax payer money. What's even worse than all of that is the insane environmental damage all of this causes. But I'll have to cut myself short here because I'm just getting increasingly upset here.

    I guess what I'm trying to say is: We're funding our own decline in rapid speed. Human stupidity has found a new peak in 2026 and it's not even close. I knew the way AI was advertised was completely overblown years ago but I never anticipated it would get this bad this quickly.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 18 points 6 months ago

    as a sr, I would just keep rejecting them and make AI find "reasons" why.

  • source
  • [–] 16 points 6 months ago* (1 child)

    or hear me out, they can build it themselves so they don’t have to chase hallucinations. as a matter of fact, let’s cut the ai out of the project and leave it to summarizing emails.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 15 points 6 months ago

    Keep taking Ls Amazon!

  • source
  • [–] 15 points 6 months ago (1 child)

    If my job ends up being reviewing AI code spammed at me by vibe coding juniors all day, I’m joining a nunnery.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 14 points 6 months ago (1 child)

    Aren't their names already on the commits? Or is the AI given write access to their code repository?

  • source
  • hideshow 1 child comment
  • [–] 14 points 6 months ago (1 child)

    LOL, so they can blame and fire SOMEONE.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 13 points 6 months ago (1 child)

    They want to move fast and break things but they still want a few meat bags around to blame when things inevitable blow up in their faces.

  • source
  • hideshow 1 child comment
  • load more comments (1 reply)
    [–] 12 points 6 months ago

    Couldn't they, I don't know, just go back to people writing the code, and stop using AI to do something it clearly can't handle? Just an idea.

    I guess they've invested (thrown) so much money at this thing, they're determined to make it work. Also, I know they've gone into insanely deep debt and if it doesn't work they're going to lose an eye watering amount of money, and perhaps the bubble bursting will be the catalyst to bringing down the entire world economy.

    Oh, so yeah, they do have great incentive to make this work, but I don't see it happening. As usual, they fuck up and the rest of us pay the bill. None of the billionaires will suffer any more than loss of face over this. Even if they've broken laws, all they ever get is a small fine and a slap on the back, "Better luck, next time, ol' boy!"

  • source
  • [–] 12 points 6 months ago (2 children)

    AI is an assistant, not a replacement. It amazes me that Amazon, Microsoft, Google, and all these "tech leader" companies are going to make the same tech fuckup multiple times.

  • source
  • hideshow 2 child comments
  • load more comments (2 replies)
    [–] 10 points 6 months ago

    xD

    Guess that all-in-on-AI attitude was not such a bold and brilliant idea after all.

  • source
  • load more comments
    view more: next ›