I was recently reading Ben Aaronvitch’s ‘Rivers of London’ and I was struggling to follow the plot. So I engaged in discussions with both ChatGPT and Gemini and both really struggled to not make lots of mistakes. Both AIs would regularly give inaccurate info about major plot points, and when I questioned them about those issues, they would apologize profusely and then try to continue on. I basically had to coach them through to get them to remember important points of the plot! They also struggled to not reveal spoilers past a certain chapter. It was interesting to see the AIs repeatedly fail.

you are viewing a single comment's thread
view the rest of the comments
[–] -2 points 4 weeks ago (2 children)

Did the AI “fail,” or did you use AI in a suboptimal way?

If you had started the conversation by feeding in the actual text of the book, you would have very different results, because you would be grounding on context rather than asking the model to “figure it out somehow” based on training data.

Not sure what you were expecting, using AI is mostly about context management, and if you don’t set up the context correctly you’re asking for results similar to this.

  • source
  • hideshow 2 child comments
  • [–] [S] 0 points 4 weeks ago* (1 child)

    It appeared it had access to the entire book - it linked me to a website that somehow had the text. I was able to coax better responses out of it when I corrected its mistakes about plot etc.

  • source
  • parent
  • hideshow 1 child comment