726
After outages, Amazon to make senior engineers sign off on AI-assisted changes (arstechnica.com)
submitted 7 months ago by to c/technology@lemmy.world
114 comments fedilink hide all child comments
 

Amazon’s ecommerce business has summoned a large group of engineers to a meeting on Tuesday for a “deep dive” into a spate of outages, including incidents tied to the use of AI coding tools.

The online retail giant said there had been a “trend of incidents” in recent months, characterized by a “high blast radius” and “Gen-AI assisted changes” among other factors, according to a briefing note for the meeting seen by the FT.

Under “contributing factors” the note included “novel GenAI usage for which best practices and safeguards are not yet fully established.”

you are viewing a single comment's thread
view the rest of the comments
[–] 24 points 6 months ago* (2 children)

Just to add to this:

  • When a senior dev reviews code from a more junior dev and gives feedback the more junior person (generally) learns from it.
  • When a senior dev reviews code from an AI, the AI does not learn from it.

So beyond the first order effects you pointed out - the using of more time from more experience and hence expensive people - there is a second order effect due of loss of improvement in the making of code which is both persistent and cumulative with time: every review and feedback of the code from a junior dev reduces forever the future need for that, whilst every review and feedback of the code from an AI has no impact at all in need for it in the future.

Given enough time, the total time wasted in reviews and feedback for code from junior devs is limited - because they eventually learn enough not to do such mistakes - but the total time wasted in reviews and feedback for code from an AI is unlimited - because it will never improve.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 11 points 6 months ago (1 child)

    Seniors reviewing code is fine but only when, as someone else mentioned, the code writer is learning from the review. The AI doesn't learn at all and the Jr Dev probably learns very little because they didn't understand the original code. Reviewing AI code often turns into me rewriting most of it.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 5 points 6 months ago*

    Exactly.

    The best way to learn is to have done the work yourself with all the mistakes that come from not knowing certain things, having wrong expectations or forgetting to account for certain situations, and then get feedback on your mistakes, especially if those giving the feedback know enough to understand the reasons behind the mistakes of the other person.

    Another good way to learn is by looking through good quality work from somebody else, though it's much less effective.

    I suspect that getting feedback on work of "somebody" else (the AI) which isn't even especially good, yields very little learning.

    So linking back to my previous post, even though the AI process wastes a lot of time from a more senior person, not only will the AI (which did most of the implementation) not learn at all, but the junior dev that's supposed to oversee and correct the AI will learn very little thus will improve very little. Meanwhile with the process that did not involve an AI, the same senior dev time expenditure will have taught the junior dev a lot more and since that's the person doing most of the work yielded a lot more improvement next time around, reducing future expenditure of senior dev time.

  • permalink
  • fedilink
  • source
  • parent