Decision Management Community

Do Language Models Know When They’re Hallucinating References? Current state-of-the-art language models (LMs) are notorious for generating text with “hallucinations,” a primary example being book and paper references that lack any solid basis in their training data. However, we find that many of these fabrications can be identified using the same LM, using only black-box queries without consulting any external resources. Consistency checks done with direct queries about whether the generated reference title is real are compared to consistency checks with indirect queries which ask for ancillary details such as the authors of the work. This suggests that the hallucination may be more a result of generation techniques than the underlying representation. Link