2 Millions people have already signed up to use ChatGPT that has been generating a lot of buzz in the AI community. What can it create, and where are the humans in the loop? How does this generalize? Cassie Kozyrkov, Chief Decision Scientist at Google, is trying to demystify this “Revolutionary New Tool for Conversation Generation”. Link

In particular, she wrote: “There’s something very important you need to know: ChatGPT is a bullshitter. The essence of bullshit is unconcern with truth. It’s not a liar because to be a liar, you must know the truth and intend to mislead. ChatGPT is indifferent to the truth.”

Read also Benedict Evans: “Lots of people have posted examples of ‘false facts’ asserted by ChatGPT. It often looks like an undergraduate confidently answering a question for which it didn’t attend any lectures. It looks like a confident bullshitter, that can write very convincing nonsense. OpenAI calls this ‘hallucinating’.”

“Instead of people trying to write rules for the machine to apply to data, we give the data and the answers to the machine and it calculates the rules. This works tremendously well, and generalises far beyond images, but comes with the inherent limitation that such systems have no structural understanding of the question.“

See also Gary Marcus: “How come GPT can seem so brilliant one minute and so breathtakingly dumb the next?”

Another example of “hallucinations” generated by ChatCRT from a discussion between serious scientists:

On 23/12/2022 15:52, Peter van Beek wrote:

The paper it cites with me as a co-author is not an actual paper and I have never done research with the co-author listed. I have heard from others that chatGPT will hallucinate examples, generalizing from the examples it has seen. The hallucinated examples are very plausible but not factual. A bit disturbing in my view that it was released in this form even if they did put a disclaimer on it.

More about GPT from Noam Chomsky