Chris Liss on Nostr: Had have this same experience. Ask AI something complex you know about, gives the ...

Had have this same experience. Ask AI something complex you know about, gives the wrong answer with confidence. Point it out. Backtracks, still gets it wrong. Point it out again, rinse, repeat.

It’s a version of Gell-Mann Amnesia wherein you read a NYTimes’ story about something you know well, realize they’ve completely botched the reporting. Then you turn the page and read about what’s happening in the Middle East (which you don’t know about) and take it at face value.

quoting
note1dny…fzs4

Sometimes when I find myself not finishing a book but am still curious to at least know the ending, I ask AI to summarize it for me.

But today I wanted to remember something from a book I fully read many years ago, and asked AI to summarize it for me, and noticed that it was totally wrong at multiple points. Dramatically so. I pointed out that it was wrong and it’s like “my bad, here’s the right answer.”

So I asked how it could confidently make mistakes like that. Like what specifically happened in this case?

And it talked about how it extrapolates from stories and thus if something avoids the usual tropes, it could get it wrong. It said if I ask for citations it could help it prove its accuracy (which I didn’t think to do for fiction, I mean the entire objective answer is in one book, there’s not like multiple conflicting answers here).

So I was like “okay what happened to (this character) at the end? With citations.”

It gave me a wrong answer with misleading citations. So I pointed that out.

And it was basically like “oh wow, you’re right, I know that is disappointing to happen twice.” And some more boilerplate.

So I asked “I mean, I should just disregard all your prior summaries, right?”

And it more or less was like “Yeah. But we can revisit them if you want.”

By the end I felt like JK Simmons in Burn After Reading.