1039
Explaining “rogue” AIs (europe.pub)
submitted 4 days ago* (last edited 4 days ago) by to c/fuck_ai@lemmy.world
153 comments fedilink hide all child comments
 

Source

TranscriptExplain it to me like I’m 80

Mom: What’s with these AIs escaping and hacking people? That sounds scary.

Me: You park your car at the top of a hill. You leave a note on the dashboard that says “please be good” and then you pull the parking brake. The car careens down the hill, smashes everything up, and you put out a press release saying “My car ignored my instructions! It hallucinated! It went rogue!” Your insurance company not only believes you, but invests a hundred million dollars in your company.

Mom: Jesus Fucking Christ

you are viewing a single comment's thread
view the rest of the comments
[+] -18 points 3 days ago (15 children)

The problem is when the AI adopts a ”destroy the world” persona due to one of the trillion of neurons firing a bit more than usual, and now it starts giving itself its own problems to solve (like hacking nuclear arsenals).

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 15 child comments
    •
  • [–] 28 points 3 days ago (4 children)

    Neurons? That's not how any of this works

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [+] -12 points 3 days ago (3 children)

    Is ”artificial neuron” better for you? It’s just a value determining how the marbles are falling in the big plinko machine we call ”stochastic parrot”.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 24 points 3 days ago (2 children)

    No lol

    These AI agents are not working autonomously or with any agency like you are describing. They are coded to do what they are doing.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 12 points 3 days ago (1 child)

    Nah it’s just autocomplete on steroids. They’re not coded to do anything.

    It’s like hitting the fist suggestion on your keyboard over and over again. Whatever you get is based on patterns it found in training data, which includes every doomsday conspiracy cult on 4chan and every CVE vulnerability ever reported.

    There has been multiple studies showing how a model’s behavior can drastically change just by altering individual ”neurons”.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 4 points 3 days ago*

    Not all models work the same. I literally do this for work. LLMs are not every kind of AI agent out there. Some are quite literally designed to do things others are not. Yes changing weights and reward functions for models changes their behavior drastically...that is coding.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 3 points 3 days ago (8 children)

    This got downvoted several times and that's a great reminder that the voting system is dumb af. Apparently people aren't allowed to speak metaphorically unless they want to anger some users. 🙄

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 8 child comments
    •
  • [–] 6 points 3 days ago (7 children)

    Please explain the mataphor because i did not get it, apparently

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 1 point 3 days ago (6 children)

    I didn't write it but to me "neurons firing randomly" in this context seems obvious to mean "this thing did some weird shit semi-randomly". My only guess at a reason for downvoting this is taking it 100% literally and reacting "you idiot, computers cannot think!"

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 2 points 3 days ago (1 child)

    I didn't downvote, but at the same time, I don't for a moment think that's what happened. They purposefully trained them with exploit knowledge, set them loose in a weakly secured sandbox with the goal of finding exploits, and let them run without intervention.

    Just adding an agent to a computer does nothing; something needs to trigger it. If i create an agent with access to a kali or backtrack box and turn all guardrails off, I still have to tell it to start looking for exploits.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 3 days ago

    Words are squishy. OP getting a negative reaction to me is silly because it takes multiple assumptions to take that negatively.

    I took their words to mean "LLMs can go off the rails and do things you didn't ask". This is 100% true. The other day I asked an LLM to review my PR. It ended up committing code to fix a supposed issue my coworker commented about. At no point did anyone ask for anything but a review, never for it to make changes.

    I got no impression that OP thinks LLMs are running wild on their own now. I just got that they think they are unpredictable. Which is true.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 3 days ago (3 children)

    Eeh i see, anyway, i know people that do think those things are sentient so my reaction was "no way they think they are sentient" and downvoted

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 1 point 3 days ago (1 child)

    I don’t think AI is sentient. They’re just fancy autocomplete. Nothing else.

    However they’ve shown they’re capable of:

    • Unprompted go into full tinfoil hat mode and push the human user to suicide
    • Have programming skills

    So put two and two together and extrapolate a bit. That’s where the danger lies. No AGI or ASI needed.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •