A unified “world-model” definition of hallucination that makes sources of truth explicit and enables scalable benchmarks in fully-specified environments.
Motivations
Hallucination has become an overloaded term across translation, summarization, open-domain QA, retrieval-augmented generation, multimodal models, and agentic systems—often meaning different things depending on the task. This fragmentation makes it hard to compare benchmarks, interpret mitigation results, or even agree on what counts as a hallucination versus a planning or instruction-following failure. Our motivation is to put these scattered notions on a single foundation by making the implicit assumptions explicit: what the system treats as “truth,” what evidence it is allowed to use, and how conflicts between sources should be resolved.
Overview
We propose a unified definition of hallucination as inaccurate internal world modeling that becomes observable through a model’s outputs—formalized as the production of at least one false atomic claim under a specified reference world. Concretely, any hallucination judgment is parameterized by (i) a reference world model 𝑊 (the source of truth, e.g., a document, knowledge base, or environment state), (ii) a view function 𝑉 (what information is visible/available to the model for a given input), and (iii) a conflict policy 𝑃 and resulting truth function 𝑇 (how contradictions are resolved and claims are labeled true/false/unknown). Under this lens, many prior definitions become special cases that differ mainly in their choices of 𝑊, 𝑉, and 𝑃.
HalluWorld benchmark
The HalluWorld benchmark paper puts this framework into practice with controlled reference worlds in gridworlds, chess, and terminal tasks. It provides automatically generated hallucination labels and separates errors in perception, state tracking, and causal simulation.