cross-posted from: https://lemmy.ml/post/54396848

If you want to know whether artificial intelligence is a gigantic economic bubble, don’t just take pundits’ word for it. Major investors are now moving their money to try to protect themselves from the AI bubble popping.

For an industry that we’re told is central to our collective future, the artificial intelligence (AI) industry increasingly seems to be built on a combination of artificially inflated hopes and bad accounting.

The industry’s critics have been raising concerns about its finances for a while: the fact that most of its flagship firms are grossly unprofitable, that they’ve had to jack up their prices to try break even (leading their business customers to pump the brakes on using their products), or that many of the eye-popping investment announcements that are meant to show demand for AI is strong have actually been circular financing — meaning different AI firms cycling money to and from each other.

But none of it has seemed to slow the industry’s boom or the relentless hype machine behind it. Until now, anyway. It used to be that you mostly heard concerns about the industry’s underlying economics in newspapers, blogs, social media posts, and podcasts, and that those concerns went completely unreflected in the markets. But these concerns seem now to have finally bled over into investors’ thinking.

you are viewing a single comment's thread
view the rest of the comments
[–] 138 points 2 days ago (38 children)

Not just wall street, some of the pro-slop people in my work are getting increasingly desperate and unhinged.

They formed an 'AI native team' to develop 'groundbreaking new applications' more rapidly than our dumb old software development teams are doing even with AI tools available.

The logic: Obviously software development skills just make the developers too opinionated and they stand in the way of AI.

The problem: After 6 months, every single vibed up slop project has failed to get through QA, and the developers can't get AI to address the issues QA is reporting (similar stuff as people have been reporting with the slop-clones of Office and Photoshop, UI elements that fail to do anything despite looking like they should, the software doing one thing when it was asked to do another).

The proposed solution emailed out: QA obviously is a problem getting in the way of the slop the customers clearly want to pay for, so we should consider skipping QA and working with the customers instead to fix it. I'm hoping this is the line that executives won't cross, most of them have been chewed out by customers enough over more mundane fuckups that they could see what a shitstorm it would be to skip the only thing protecting us from shit.

  • source
  • hideshow 38 child comments
  • [–] 7 points 1 day ago

    I was ready to type "just fire QA and have the LLM do it" then I read the last paragraph.

    I'm hoping this is the line that executives won't cross

    Hate to be a downer here buuuut....

  • source
  • parent
  • [–] 8 points 1 day ago

    As a former quality person this is everywhere. Manufacturing has basically relegated QA/QO to be task based (reactive, paper pushing, etc) and removed from the process completely unless deviations occur (which they do all the time, go figure).

    The further you are from the product the less value you have.

  • source
  • parent
  • [–] 1 point 21 hours ago*

    That’s unfortunate. But less unfortunate than it could be.

    The problem with pure vibecoding is that it can get really flashy results up front, just enough to impress and get green lights. But it’s completely unsustainable outside the simplest projects. At some point, vibecoded stuff gets issues, or one realizes certain features actually aren’t working, and no one can fix them.

    In other words, it’s great that QA is already catching the issues. Because they could have quite possibly vibecoded something functional, passed through QA, and ended your separate effort, only for it to collapse later.

    Note that I am strictly talking vibecoding, though. I know it’s not a popular opinion in this sub, but more mundane LLM “chore automation” is far more practical.

  • source
  • parent
  • [–] 100 points 2 days ago (8 children)

    QA obviously is a problem

    Oh lord...

  • source
  • parent
  • hideshow 8 child comments
  • [–] 4 points 1 day ago*

    The more things change the more they stay the same.

    If shit rolls downhill QA is at the bottom and I'm saying this as someone who comes from Ops. I deal with the shit when it hits production.

    Truth of the matter is in any org it goes like this

    Devs - literally walk on water. Give us new and shiny things

    Ops - support the half baked shit that devs promote to prod on lies that "it's production ready we will fix in future sprints"

    QA - magical role that is never filled because "we can just get devs to review their own code"

  • source
  • parent
  • [–] 22 points 1 day ago (5 children)

    Well, they're right.

    "If we stop testing right now, we'd have very few cases, if any," Trump asserted, "Without testing, or weak testing, we would be showing almost no cases."

  • source
  • parent
  • hideshow 5 child comments
  • [–] 3 points 1 day ago (4 children)
  • [–] 8 points 1 day ago* (3 children)

    In case you weren't being sarcastic: Yes, that's a direct quote from the Paedophile traitor, the Fuhrer in Orange, shitlord of Hamberder.

    He claimed that the only reason america had COVID cases to report was because we were testing for them, stating directly that testing was bad for publicity.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 20 hours ago (1 child)

    Covid kind of exposed how he operates.

    His problem was that the virus did not pay attention to the media or care about politics.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 20 hours ago

    Moron downplayed it because he thought it would hurt his reelection chances. If he had leaned into COVID and mad a huge public commitment to fight it he would have sailed right back into office.

  • source
  • parent
  • [–] 36 points 2 days ago

    And if you know how few of the real problems QA normally detects before a product is being shipped to the customer you know the shit show that those products will cause.

    Hand me the popcorn, that show has a lot of potential.

  • source
  • parent
  • [–] 19 points 1 day ago (5 children)

    One early example I heard about slop code being bad was that if you have a process that is repeated, an LLM will not write a function that is broadly scoped so it can handle a significant number of related use cases, it will write different code for each instance, so when that process fails you have no idea which version is responsible. Seems like the kind of thing QA would flag.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 2 points 22 hours ago* (2 children)

    I've never known QA evaluations to be design-based, only results-based. They would test each of the cases and reject the code if it failed any of them, but they wouldn't check for or reject the code just because it was handling them in stupid, non-generalized ways.

    Checking for bad design is more of a team lead code reviewer thing (if it happens at all).

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 20 hours ago (1 child)

    Way back in the oughties I worked on a consumer electronics project where our QA guy actually used the product, would come to design reviews, and filed bugs about the UI design confusing the user.

    It was awesome, so of course that job got eliminated as soon as possible.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 19 hours ago

    I would still call the user-facing look and feel of the UI more of a "result" in the sense I'm using it, though. When I said "design" I was really talking about the internal implementation that someone not looking at the source code would never experience.

    It doesn't necessarily hurt for QA people to have access to the source code, but they shouldn't need it to do their job.

  • source
  • parent
  • [–] 17 points 1 day ago

    QA obviously is a problem getting in the way

    Tbf, QA being seen as a problem slowing things down is nothing new in development. AI just gives it a new impulse.

    The same thing is happening in my work. Just as we got the higher ups convinced that yes, having QA is very valuable, we get called the bottleneck again because dev is supposedly speeding up due to AI (according to dubious metrics).

  • source
  • parent
  • [–] 20 points 2 days ago* (5 children)

    The problem: After 6 months, every single vibed up slop project has failed to get through QA, and the developers can’t get AI to address the issues QA is reporting (similar stuff as people have been reporting with the slop-clones of Office and Photoshop, UI elements that fail to do anything despite looking like they should, the software doing one thing when it was asked to do another).

    • Is AI imploding within itself through poisoning and no new content?
    • Are they putting in their own guardrails to take away the really good innovations because of greed?
    • AI was never as good as they claimed, even the super advanced models?
    • All of the above?

    I'm more curious than anything. It seems to be getting worse for the free models.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 9 points 1 day ago

    Is AI imploding within itself through poisoning and no new content?

    Model collapse doesn't retroactively affect models that have already been trained, so this option seems like a pretty nonrigorous belief

    Are they putting in their own guardrails to take away the really good innovations because of greed?

    This would be difficult and likely counterproductive, but they are known to adjust the amount of compute they allow models to use, so they could be throttling that

    AI was never as good as they claimed, even the super advanced models?

    Ding ding ding, we have a winner! AI has improved, but those who are selling it will always and forever oversell it, like they would with any product.

  • source
  • parent
  • [–] 11 points 1 day ago (3 children)

    I'd say it's never been as good as Musk and Altman especially make it sound.

    If you drill into the nuance of their proof points that their companies bother to share, it's generally a bit more clear that things aren't exactly 'asterisk free' even then. Most recently I read how one of them proudly proclaimed they were 100% manual code free, but for the task described they still had 200 developers, which is significantly more developers than I have traditionally seen. To get things vaguely working, you need to be relentless in providing a super comprehensive set of requirements and criteria that are impeccable at fully capturing what is needed and could reasonably go wrong. Also a fanatical amount of detail to audit how the solution was reached and correct missteps, like when it decides to delete a failing test case instead of fixing the correlated problem. The 'purist' approach is, in my opinion, worse for the sake of just signaling how 'AI-friendly' you are.

    I can at least buy that from a pragmatic perspective, the codegen can accelerate things if used under a more balanced approach, being able to be in the code and grasp in detail where it might have gone wrong and correct it before it gets lost in a mess of garbage. It will do things that a human might skip that they shouldn't, like hooking various unlikely errors and providing a nicer treatment. It is often useful to get a general idea of a good choice of library to do something and how it fits in your code, though you still have to beware that it may invent details that would be nice and convenient, but are wrong.

    The problem is that narrative is totally not what they need to justify the spend. They needed to take randos and prompt up things without full understanding. They needed AI codegen to be really valued on the same level as a full headcount. This trickles down to little folk like the team lead in my situation, where he thinks this is his only ticket to the 'big leagues' if he manages to drive a success story for 'AI native'. He doesn't understand any of it and his team doesn't understand any of it and so of course the AI can't bring what actually understanding the scenario would.

    Even more rough for that guy: he imagines a world of slopping up code and then selling the result, without ever having unique skill or insight. Assuming that can work, then why would any company buy your slop instead of just slopping it up themselves?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 7 points 1 day ago (1 child)

    Remarkably well said. I don't have much of substance to add, but it is funny to me that:

    To get things vaguely working, you need to be relentless in providing a super comprehensive set of requirements and criteria that are impeccable at fully capturing what is needed and could reasonably go wrong.

    is basically describing what programming already was.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 8 points 1 day ago

    I have found AI to be great as an extra pair of eyes, organizing, and finding obscure sources. So it's a great assistant, not creator. You have to know what you're doing.

    It's like sending a fresh set of very smart, right out of school, employees to do the job. They have been trained to do everything you want them to do and more, but guiding them takes expertise. These recruits can't do it on their own, they need a leader.

    Are the companies understanding this, or is greed blinding them?

  • source
  • parent
  • [–] 15 points 1 day ago (1 child)

    The proposed solution emailed out: QA obviously is a problem getting in the way of the slop the customers clearly want to pay for, so we should consider skipping QA and working with the customers instead to fix it.

    Narrator: the people who agreed to this never interacted with a single customer.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 12 points 1 day ago

    Well, so far, it's the "I've bet my entire career on AI" leader who has not yet gotten traction. He had a large benefit of the doubt but I feel like leadership is getting more and more skeptical about him and his band of merry sloppers.

  • source
  • parent
  • [–] 6 points 2 days ago (1 child)

    Might be an idea to start looking at vacancies in other companies. Just in case yours goes nuclear on the stupid.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 7 points 1 day ago

    As someone who went through the dot.com bust at a very unfortunate time in my career, I'm always looking for the escape hatch.

    But I feel like the prevaling opinion in the leadership guy is increasingly dubious of this guy, so I'm hopeful sanity prevails.

  • source
  • parent