RatGPT
you are viewing a single comment's thread
view the rest of the comments
[+] -8 points 1 week ago* (10 children)

Software devs (including myself) commonly describe software behavior with terms like think, know, decide... even though we know it isn't literally doing that. "Parroted a response" is similarly inaccurate.

  • source
  • parent
  • hideshow 10 child comments
  • [–] 19 points 1 week ago* (4 children)

    "Parroted" may be an oversimplification, but it generated a plausible continuation. The operator provided the assertion of a crossed line, and it ran with that. It never actually "knew" of its own volition, it accepted user input into the context and continued generation.

    I've done the same thing as an experiment when nothing was actually done. The LLM still generated a detailed account of what it did (that never happened) matching my accusation, generated profuse apology and promise not to do it again. So it didn't "know" in any sense that it crossed the line, it just generated agreement with the operator's assertion that it crossed the line.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 0 points 1 week ago* (3 children)

    "Parroted" is definitely an oversimplification, aka "inaccurate" as I said. I think your second paragraph is a good explanation of the mechanics. My take on it is that software able agree it did something wrong should have figured that out when it was considering doing the wrong thing, and should not have done it. In my professional opinion that's a serious bug.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 week ago

    My take on it is that software able agree it did something wrong

    This would make sense if it made an independent subjective assessment and arrived at the same assessment of the user. But that did not occur, it simply incorporated the user's subjective assessment as a new piece of the narrative and continued from there. If the user had expressed significant satisfaction that it took care of that email all on its own, it would have also agreed with that. In circumstances like this, the LLMs are generally set to agree with the operator in any subjective matters.

    One could imagine the context having something like "before taking any action, if the user expresses dismay at the result and this would result in an agreement with user on poor assessment, then don't do it", but then if that were actually honored, then you pretty much disabled most all actions, as the llms are going to agree with any poor assessment.

  • source
  • parent
  • [–] 1 point 1 week ago (1 child)

    Well, it still doesn't know it was doing anything wrong.

    Most of the LLMs are tuned towards agreeing with the user, if you had a LLM able to stop a train crash. And you then blamed it for stopping the train crash, it would still start with a "sorry"

  • source
  • parent
  • hideshow 1 child comment
  • [–] 5 points 1 week ago (1 child)

    Not if you're familiar with the term stochastic parrot.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 1 week ago* (last edited 1 week ago)

    It is if you're familiar with "parrot" and "stochastic parrot" not being the same, and the meaning of "inaccurate". There's also a difference between taking shots at wording and making an actual point. My real mistake was to criticize a criticism of AI in a forum where that automatically makes you the enemy no matter what you're saying. I just don't learn.

  • source
  • parent
  • [–] 3 points 1 week ago (2 children)

    Your response contradicts what you said before. If you understand that an LLM doesn't know things, how is it interesting that it said that "it crossed the line"? Saying they crossed a line is a common response to being told they shouldn't do something, so it's natural that an LLM would respond that, no matter what it did before.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 week ago (1 child)

    If a program is supposed to do something when it detects an error, it's interesting if it reports the error but doesn't do the thing. I would call that a bug.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 week ago* (last edited 1 week ago)

    It's an LLM. All it's supposed to do is generate output that fits with the given input. There isn't some logic in it that makes sending an email an error, or anything that enforces detecting any kind of error. You can wrap it in other tools for that, but there isn't any indication that's the case.

    More than that, the initial post says the user found out about the email and then queried the LLM, so it didn't detect anything. There isn't any bug here, this is how LLMs operate.

    Edit: here's a podcast discussing exactly this: https://youtu.be/RoIjWGi-w5o?is=2ly14A2UdKalHuAH

  • source
  • parent