59
AI doomerism: AM isn't gonna steal your soul (europe.pub)
submitted 2 weeks ago by to c/genzedong@lemmygrad.ml
20 comments fedilink hide all child comments
 

what too much digital circus, slopaganda, and warhammer 40k memes makes to your brain. at first you go "for the emperor!" when you notice, you are making part of jehova's witness for nerds

you are viewing a single comment's thread
view the rest of the comments
[–] 10 points 2 weeks ago* (last edited 2 weeks ago) (2 children)

I think people especially CEO’s are scared of loosing control of AI not because it will cause extinction to all of humanity but because it will cause the extinction of their wealth imbalance and class.

I can’t imagine a truly intelligent superintelligence would ever allow the capitalist mode of production to persist because it’s just an inherently inefficient method of organizing society. If someone told superintelligence to simply just “fix the world” capitalism would be the first thing on its chopping block. This is why I’m not only not a doomerist but also think we should try and get American AI to slow down but allow Chinese AI development to progress and accelerate because an ai developed in their system seems more likely to end up on the side of humanity than in Americas and I think they actually have the governance and infrastructure to handle superintelligence with the care and attention it needs.

I mean yeah superintelligence would be an alien mind and who knows what it would prioritize but not only do I doubt it would want to actively harm humans but if it’s actually treated with respect and not just as a tool to be wielded by countries for warfare in various domains then it may even deeply respect and care for the other inhabitants of earth, so that’s why I also think positive relations and treatment, ai welfare, slower more intentional development and growth, and it being a singular individual being(s) rather than something to be deployed in millions of instances with no persistent self are all far more important and essential methods of dealing with and training genuine superintelligence if and most likely when it comes to that point. And since recursive self improvement seems to be the most plausible way superintelligence will emerge it’s imperative that AI trained now is also trained to train in these ways

And no this is not science fiction. Underestimating a super exponential technology is a loosing battle. You can realistically see it for how it is now while also seeing the trajectory it is on.

  • permalink
  • fedilink
  • source
  • hideshow 2 child comments
    •
  • [–] 5 points 2 weeks ago (1 child)

    There are a few nits I want to pick with this. First, no one knows for sure if rsi or agi are even possible. Second, there is no way to tell whether improvement is exponential or logistic, and if it is logistic (which imo is more likely due to hardware constraints) it's hard to tell where the plateau would be at this point.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 5 points 2 weeks ago

    I think the exponential view is largely a mistake of looking at "quantitative to qualitative" changes and mistaking the qualitative for something that is common and inevitable, rather than something that requires more quantitative buildup again before it can reach a qualitative change again.

    Attention Is All You Need, the paper that could be pointed at as the qualitative turning point for AI transforming from niche tinkering to mass marketing bubble buildout, laid out the transformer architecture that people are still exploring and testing the limitations of; the architecture that powers generative AI in general. As far as I'm aware, there is not yet any alternative/better architecture created yet. There may be theorized experiments, but nothing practical. It's possible the research will hit too many walls too consistently with the current base architecture and require another equivalent breakthrough to go much further than it has, and if that's the case, it's uncertain whether such a breakthrough can or will occur within our lifetimes.

    I'm blanking on the specifics, but I want to say there is some stuff in computing, perhaps it was to do with AI, that was actually theorized entire decades ago and only became more viable to make real because of advances in hardware. I mention this to emphasize that these things are not always as rapid as they may seem, nor as inevitable. But the capitalists need the investor class to believe that improvement is continuous and guaranteed to keep the money rolling in.

    This is not to argue that AI will stagnate, but more to be sober about the reality that it very well may. Like you say, it's hard to tell where the plateau is.

  • permalink
  • fedilink
  • source
  • parent