-
@ Robin Adams
2025-03-04 07:01:05
nostr:nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqhpy6zyt0pa05nezps2x8hhrntua2hng8t495ya4rnyj7ne83ey9qn39gmq nostr:nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqky223zcc4q69d8t0me4vg5uw8mw0yxeukjgvz6h92laqnenr0ajsgze5g0 nostr:nprofile1qy2hwumn8ghj7un9d3shjtnddaehgu3wwp6kyqpqwrpqzd5elcczztl3tzs29kat0gy7e0js88cvshcvjrtj2483l6nqp45a92 Of course ChatGPT can make things up - that's what it was designed for. GPTs are interesting because they can generate new original text. They were never intended as information sources, just an experiment in language generation.
It doesn't just repeat its training set or recombine fragments. It samples from the probability distribution inferred from the training data set. It produces a sequence of words such that, based on the training data, those words have a high probability of occurring in that sequence. Its one job is "produce a new piece of text that looks like the text in your training data".
So we give it a question and ask "If this question occurred in your training data, give me a piece of text that has a high probability of following it." Turns out the result is very often an answer to the question.
Often this gives you a correct answer to your question, especially if the question and a correct answer occurred many times in the training data. And sometimes it generates a wrong answer that looks very like a correct answer.
"Hallucination" is a misleading name because it makes it sound like a bug or glitch that could be eliminated, when it's a GPT's core function. It's doing exactly the same thing when it's hallucinating as when it's giving a correct answer.
See Xu, Jain, Kankanhalli. Hallucinations are Inevitable: https://arxiv.org/abs/2401.11817