384
Microsoft confirms an AI worm is propagating through Copilot and other MS apps | CSO Online (www.csoonline.com)
submitted 2 months ago by to c/technology@lemmy.world
76 comments fedilink hide all child comments
you are viewing a single comment's thread
view the rest of the comments
[–] 87 points 2 months ago (14 children)

the only way to block it is to get AI to differentiate instructions from data, which is impossible today

Input sanitation, basically security 101. And it can't currently do it....

  • permalink
  • fedilink
  • source
  • hideshow 14 child comments
    •
  • [–] 35 points 2 months ago (5 children)

    if the instruction is messy fuzzy human language to a system that was not coded instruction by instruction but got generated and trained then there never is a way to differentiate instructions from data if i'm not mistaken

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 1 point 2 months ago (3 children)

    We know that some humans can be trained to do that just fine. Humans are natural neuronal networks. That implies, neuronal networks can in principle do it. We just don't have any human-capability artificial neuronal networks yet.
    LLMs might never get there. But humans aren't LLMs. If we ever manage to properly model a human brain, that probably will be able to do that task with human-level accuracy (which actually is pretty good if you only look at professionals of the filed).
    Hopefully, it doesn't actually need a human brain for the task - because modeling that might still be a century off.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 6 points 2 months ago (2 children)

    A human brain model would never get us to a super computer, which is their intended goal.

    Our brains have about one exaflop of processing. The NVidia B300 has 15 petaflops of processing. Corresponding to 0.015 exa, you'd need 66 of the strongest GPUs on the market to match the processing power of one human brain.

    That's before considering context, current estimations would put the human brain at 2.5 petabytes of storage. The former B300 has 288 gigabytes of VRAM. Corresponding to 0.000288 of one human brain, the earlier expectation of 66 landing flat at 0.019 the storage of a human brain.

    Can arguments be made not all brain capacity is at VRAM at all times? Most likely. But context windows is the primary bottleneck, not compute

    Our brains are absolute monsters, we'd gain more from understanding how we can have this powerful of a machine on so little space. The bigger breakthrough would be brainbased processing of software, but I've not heard of anyone going that route - Probably because it's not feasible

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 2 points 2 months ago

    Human brains are also considerably more efficient than an artificial neural network. Most modern ones only replicate the function of a human brain and its neurons in the most basic of aspects.

    Things like encoding data itself in how the signal is transmitted are still pretty uncommon in artificial neural networks, for example.

    We'd probably need somewhere close to several times the size and computing power of a human brain to even start coming close with current technology, assuming that it is possible to begin with.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 2 months ago

    Just a hundred years ago, we had one floating point operation per second in mechanical calculators. We got roughly 100 TFlops (fp16; probably much more accurate than human brain neurons) in a high-end gaming GPU now. That's 14 orders of magnitudes in a hundred years. I guess, 4 more orders of magnitude in another hundred years should be possible. We don't even have fusion power yet. A lot changes when power becomes a practically unlimited resource.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 17 points 2 months ago (1 child)

    I just can't anymore. Isn't that like, the basic thing any program does? Who runs these companies?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 2 points 2 months ago*

    It is - and tons of bugs are just about fucking it up or just not doing it at all. Incomplete or faulty parsing and validation of untrusted input is absurdly common. It got better in the past decades, but most software developers write worse code than AI by now (not me though; obviously, I am still better than the best AI).

  • permalink
  • fedilink
  • source
  • parent
  • [–] 5 points 2 months ago (2 children)

    I have heard in the past that it's not possible to fully control AI. Like literally, the people developing and running the AI cannot fully control its behavior. I did a quick search to see if I could find more info and found this link on the first page of results: https://www.eurekalert.org/news-releases/1032090

    I think that we're going to continue seeing unwanted behavior from AI.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 7 points 2 months ago* (last edited 2 months ago) (1 child)

    Distinguished credentials, but at the same time I am not buying it. You can control AI. You can turn it off. You can have it not interact with systems you don't want.

    Remember this guy is saying "you cant control AI, we are all doomed" while also saying that we live in a simulation and he is very close to being able to hack us out of it.

    Grain of salt and all.

    By the way his belief is thus "AI can't be contained, therefore the simulation can be escaped; by contraposition, if the simulation can't be escaped, AI can be contained" Since AI cant be contained, he reasons, we can escape the simulation, quite possibly by using a super AI!

    There might be a reason he has a podcasts and visits Joe Rogan

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 2 months ago

    Thanks, I fully agree that AI can be controlled in the ways you mentioned. I think most of the discussion around AI not being fully controllable is in the context of if the AI is running as it currently is.

    Thanks for the additional info too. I knew nothing about the guy in the link. It was just the first link I found on a quick search about AI not being controllable. I had heard it before, but didn't remember where, or the exact details.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 2 months ago*

    It's a text completion model/glorified Markov chain. Of course it can't input sanitise, it was never meant to do this to begin with.

    The tool calling integrations that let it do more are basically making it add a markdown code block in JSON format into the user message, where the middleware intercepts it.

    The input and "instructions" are the same thing from its perspective. There's nothing special that differentiates the two. The user input text, so it will output text, following the most likely sequence from its training.

  • permalink
  • fedilink
  • source
  • parent