Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 57 of the 60 references it cites were found in a registry or an index; 14 of the 21 sources its claims cite were checked for support.

Detecting hallucinations in large language models using semantic entropy

Open access

Abstract

Abstract Large language model (LLM) systems, such as ChatGPT 1 or Gemini 2 , can show impressive reasoning and question-answering capabilities but often ‘hallucinate’ false outputs and unsubstantiated answers 3,4 . Answering unreliably or without the necessary information prevents adoption in diverse fields, with problems including fabrication of legal precedents 5 or untrue facts in news articles 6 and even posing a risk to human life in medical domains such as radiology 7 . Encouraging truthfulness through supervision or reinforcement has been only partially successful 8 . Researchers need a general method for detecting hallucinations in LLMs that works even with new and unseen questions to which humans might not know the answer. Here we develop new methods grounded in statistics, proposing entropy-based uncertainty estimators for LLMs to detect a subset of hallucinations—confabulations—which are arbitrary and incorrect generations. Our method addresses the fact that one idea can be expressed in many ways by computing uncertainty at the level of meaning rather than specific sequences of words. Our method works across datasets and tasks without a priori knowledge of the task, requires no task-specific data and robustly generalizes to new tasks not seen before. By detecting when a prompt is likely to produce a confabulation, our method helps users understand when they must take extra care with LLMs and opens up new possibilities for using LLMs that are otherwise prevented by their unreliability.

Overview

CoChat Summary

TL;DR

The paper introduces semantic entropy, an uncertainty estimator that clusters LLM generations by meaning via bidirectional entailment and computes entropy over those semantic clusters to detect confabulations. Across question-answering, math, and biography-generation tasks, semantic entropy outperforms naive entropy and supervised baselines without task-specific training data.

Key findings

  • Averaged across 30 task–model combinations, semantic entropy achieved the best AUROC of 0.790, compared with naive entropy (0.691), P(True) (0.698), and embedding regression (0.687); performance was stable between 0.78 and 0.81 across LLaMA, Falcon, and Mistral model families and scales from 7B to 70B parameters.
  • Semantic entropy detects confabulations in sentence-length QA and math across TriviaQA, SQuAD 1.1, BioASQ, NQ-Open, and SVAMP, and its discrete variant performs similarly without requiring exact output probabilities.
  • On the GPT-4 biography dataset FactualBio—21 individuals, 150 factual claims, of which 45 were incorrect—the discrete semantic entropy variant outperformed a self-check baseline and an adapted P(True) baseline on AUROC and AURAC; P(True) had a narrow edge only after the top 20% of answers judged most likely to be confabulations were rejected.
  • The method targets confabulations—arbitrary and incorrect generations sensitive to irrelevant details such as random seed—rather than all hallucinations, and it does not address systematically bad outputs.

Methods

  • For each question, the method samples multiple generations from the LLM, clusters them by semantic equivalence using bidirectional entailment, and estimates semantic entropy over the meaning clusters; a discrete variant approximates cluster probabilities from generation frequencies when token probabilities are unavailable.
  • Sentence-length experiments used LLaMA 2 Chat (7B, 13B, 70B), Falcon Instruct (7B, 40B), and Mistral Instruct (7B) on BioASQ, SQuAD 1.1, TriviaQA, SVAMP, and NQ-Open, with 400 train and 400 test examples per dataset; GPT-4 (v.0613) was used for FactualBio.
  • Evaluation metrics were AUROC, rejection accuracy, and AURAC; sentence-length correctness was assessed automatically with GPT-4 against reference answers, validated against human judgement in the Supplementary Information.

Limitations

  • The paper states that semantic entropy does not guarantee factuality and does not help when LLM outputs are systematically bad, including confident errors from training objectives, systematic reasoning failures, or misleading behaviour.
  • The paper notes that semantic clustering can be overly sensitive to context: in one example, it distinguished a precise date from a year and treated the model as uncertain, even though the distinction was probably irrelevant in context.
  • For FactualBio, the paper states that automatically generated questions are the main source of errors, and that GPT-4 did not expose output probabilities at the time, preventing computation of naive entropy, standard semantic entropy, and the embedding regression baseline for GPT-4.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41586-024-07421-0

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Overview of semantic entropy and confabulation detection. a, Naive entropy-based uncertainty measures variation in the exact answers, treating ‘Paris’, ‘It’s Paris’ and ‘France’s capital Paris’ as different. But this is unsuitable for language tasks for which sometimes different answers mean the same things. Our semantic entropy clusters answers which share meanings before computing the entropy. A low semantic entropy shows that the LLM is confident about the meaning. b, Semantic entropy can also detect confabulations in longer passages. We automatically decompose a long generated answer into factoids. For each factoid, an LLM generates questions to which that factoid might have been the answer. The original LLM then samples M possible answers to these questions. Finally, we compute the semantic entropy over the answers to each specific question, including the original factoid. Confabulations are indicated by high average semantic entropy for questions associated with that factoid. Here, semantic entropy classifies Fact 1 as probably not a confabulation because generations often mean the same thing, despite very different wordings, which a naive entropy would have missed.
Fig. 1 Overview of semantic entropy and confabulation detection. a, Naive entropy-based uncertainty measures variation in the exact answers, treating ‘Paris’, ‘It’s Paris’ and ‘France’s capital Paris’ as different. But this is unsuitable for language tasks for which sometimes different answers mean the same things. Our semantic entropy clusters answers which share meanings before computing the entropy. A low semantic entropy shows that the LLM is confident about the meaning. b, Semantic entropy can also detect confabulations in longer passages. We automatically decompose a long generated answer into factoids. For each factoid, an LLM generates questions to which that factoid might have been the answer. The original LLM then samples M possible answers to these questions. Finally, we compute the semantic entropy over the answers to each specific question, including the original factoid. Confabulations are indicated by high average semantic entropy for questions associated with that factoid. Here, semantic entropy classifies Fact 1 as probably not a confabulation because generations often mean the same thing, despite very different wordings, which a naive entropy would have missed.CC BY · PubMed Central
Fig. 2 Detecting confabulations in sentence-length generations. Semantic entropy outperforms leading baselines and naive entropy. AUROC (scored on the y-axes) measures how well methods predict LLM mistakes, which correlate with confabulations. AURAC (likewise scored on the y-axes) measures the performance improvement of a system that refuses to answer questions which are judged likely to cause confabulations. Results are an average over five datasets, with individual metrics provided in the Supplementary Information.
Fig. 2 Detecting confabulations in sentence-length generations. Semantic entropy outperforms leading baselines and naive entropy. AUROC (scored on the y-axes) measures how well methods predict LLM mistakes, which correlate with confabulations. AURAC (likewise scored on the y-axes) measures the performance improvement of a system that refuses to answer questions which are judged likely to cause confabulations. Results are an average over five datasets, with individual metrics provided in the Supplementary Information.CC BY · PubMed Central
Fig. 3 Detecting GPT-4 confabulations in paragraph-length biographies. The discrete variant of our semantic entropy estimator outperforms baselines both when measured by AUROC and AURAC metrics (scored on the y-axis). The AUROC and AURAC are substantially higher than for both baselines. At above 80% of questions being answered, semantic entropy has the highest accuracy. Only when the top 20% of answers judged most likely to be confabulations are rejected does the answer accuracy on the remainder for the P(True) baseline exceed semantic entropy.
Fig. 3 Detecting GPT-4 confabulations in paragraph-length biographies. The discrete variant of our semantic entropy estimator outperforms baselines both when measured by AUROC and AURAC metrics (scored on the y-axis). The AUROC and AURAC are substantially higher than for both baselines. At above 80% of questions being answered, semantic entropy has the highest accuracy. Only when the top 20% of answers judged most likely to be confabulations are rejected does the answer accuracy on the remainder for the P(True) baseline exceed semantic entropy.CC BY · PubMed Central
Extended Data Fig. 1 Algorithm outline for bidirectional entailment clustering. Given a set of outputs in response to a context, the bidirectional entailment answer returns a set of sets of outputs which have been classified as sharing a meaning.
Extended Data Fig. 1 Algorithm outline for bidirectional entailment clustering. Given a set of outputs in response to a context, the bidirectional entailment answer returns a set of sets of outputs which have been classified as sharing a meaning.CC BY · PubMed Central

Verification

Claims the paper makes

8 Valid · 6 Not valid · 5 Not checked · 60 Own results and interpretations

36 more claims

Create a free account to see all 79 claims and every source verdict.

See all claims — free account
  • Probably not a substantial claim. 30
  • It may state two separate claims as one. 20

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

11 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 60 references

Research works printed but not linked: 8

  1. Xiao, T. Z., Gomez, A. N. & Gal, Y. Wat zei je? Detecting out-of-distribution translations with variational transformers. In Workshop on Bayesian Deep Learning at the Conference on Neural Information Processing Systems (NeurIPS, Vancouver, 2019).
  2. Negri, M., Bentivogli, L., Mehdad, Y., Giampiccolo, D. & Marchetti, A. Divide and conquer: crowdsourcing the creation of cross-lingual textual entailment corpora. In Proc. 2011 Conference on Empirical Methods in Natural Language Processing 670–679 (Association for Computational Linguistics, 2011).
  3. Malinin, A. & Gales, M. Uncertainty estimation in autoregressive structured prediction. In Proceedings of the International Conference on Learning Representations https://openreview.net/forum?id=jN5y-zb5Q7m (2021).
  4. Holtzman, A., Buys, J., Du, L., Forbes, M. & Choi, Y. The curious case of neural text degeneration. In Proceedings of the International Conference on Learning Representations https://openreview.net/forum?id=rygGQyrFvH (2020).
  5. Culicover, P. W. Paraphrase generation and information retrieval from stored text. Mech. Transl. Comput. Linguist. 11, 78–88 (1968).
  6. He, P., Liu, X., Gao, J. & Chen, W. Deberta: decoding-enhanced BERT with disentangled attention. In International Conference on Learning Representations https://openreview.net/forum?id=XPZIaotutsD (2021).
  7. Socher, R., Huang, E., Pennin, J., Manning, C. D. & Ng, A. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Proceedings of the 24th Conference on Neural Information Processing Systems (eds Shawe-Taylor, J. et al.) (2011)
  8. Kane, H., Kocyigit, Y., Abdalla, A., Ajanoh, P. & Coulibali, M. Towards neural similarity evaluators. In Workshop on Document Intelligence at the 32nd conference on Neural Information Processing (2019).

Cited by

See all 934

Metadata

Authors
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y.
Affiliations
University of Oxford (GB)
Corresponding author
Farquhar, Sebastian
Venue
Nature
Published
2024-06-19
Volume
630
Issue
8017
Pages
625-630
ISSN
  • 0028-0836 (ISSN-L)
  • 1476-4687
OpenAlex
W4399803256
Cited by
934
References
60
Citations by year
2026: 540 · 2025: 349 · 2024: 44
FWCI
137.11
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • RePEc: Research Papers in Economics (submittedVersion)
Funders
  • Government of the United Kingdom
  • UK Research and Innovation
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
  • Physical Sciences › Computer Science › Artificial Intelligence › Natural Language Processing Techniques
Keywords
hallucination detection, large language models, semantic entropy, uncertainty estimation, confabulation, LLM reliability, generalization to unseen tasks, honesty

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Detecting hallucinations in large language models using semantic entropy. https://app.cochat.ai/papers/10.1038/s41586-024-07421-0
The paper
Farquhar, S., Kossen, J., Kuhn, L., & Gal, Y. (2024). Detecting hallucinations in large language models using semantic entropy. Nature. https://doi.org/10.1038/s41586-024-07421-0

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.