CoChat
Sign Up
Cited by 152Artificial Intelligence in Healthcare and EducationTopic ModelingMultimodal Machine Learning Applications

Med-HALT: Medical Domain Hallucination Test for Large Language Models

Open access
Pal, Ankit, Umapathi, Logesh Kumar, Sankarasubbu, Malaikannan·2023·Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)
DOI 10.18653/v1/2023.conll-1.21
Open this paper in CoChat — free accountOpen PDF

Abstract

This research paper focuses on the challenges posed by hallucinations in large language models (LLMs), particularly in the context of the medical domain.Hallucination, wherein these models generate plausible yet unverified or incorrect information, can have serious consequences in healthcare applications.We propose a new benchmark and dataset, Med-HALT (Medical Domain Hallucination Test), designed specifically to evaluate and reduce hallucinations.Med-HALT provides a diverse multinational dataset derived from medical examinations across various countries and includes multiple innovative testing modalities.Med-HALT includes two categories of tests reasoning and memory-based hallucination tests, designed to assess LLMs' problem-solving and information retrieval abilities.Our study evaluated leading LLMs, including Text Davinci, GPT-3.5, LlaMa-2, MPT, and Falcon, revealing significant differences in their performance.The paper provides detailed insights into the dataset, promoting transparency and reproducibility.Through this work, we aim to contribute to the development of safer and more reliable language models in healthcare.Our benchmark can be found at med- halt.github.ioAIIMS PG (India) NEET PG (India) Exámenes médica (Spain) TWMLE (Taiwan) USMILE (U.S)

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.18653/v1/2023.conll-1.21

Cites

18 of 18 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Projecting to a large benchmark scenario (Med-HALT; 59,254 items), the cost of a full-length evaluation would be on the order of $1475 under stated pricing assumptions, whereas CAT-based evaluation would be under $5.

    “Using empirically observed testing characteristics (e.g., an average of 37 items per administration and per-item token usage) and projecting them to a large benchmark scenario (Med-HALT; 59,254 items10), we estimate that the cost of a full-length evaluation would be on the order of $1475 under the stated pricing assumptions, whereas the CAT-based evaluation would be under $5.”

    From Leveraging computerized adaptive testing for cost-effective evaluation of large language models in medical benchmarking (2026) · cites this paper as [10]

    Section 2, “Datasets Statistics” (page unavailable), identifies Med-HALT as a large benchmark and reports 18,866 samples per reasoning-hallucination task plus a separate PubMed portion. It does not give the cited figure of 59,254 items. The $1,475 and under-$5 costs are explicitly the citing authors’ estimates, not claims attributed to Med-HALT. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Some benchmarks incorporate custom metrics to capture trust facets.

    “To capture trust facets, some benchmarks incorporate custom metrics: e.g., Med-HALT's false confidence rate (Pal et al., 2023), or MedSafety-Bench's safety score (Han et al., 2024a).”

    From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Pal et al., 2023

    The False Confidence Test section describes a Med-HALT test of whether models answer with unnecessary certainty. But the paper calls it a test, not a “false confidence rate”; its Evaluation matrices section reports accuracy and pointwise score rather than a metric by that name. No page number is available in our copy. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Med-HALT incorporates a false confidence rate metric.

    “To capture trust facets, some benchmarks incorporate custom metrics: e.g., Med-HALT's false confidence rate (Pal et al., 2023), or MedSafety-Bench's safety score (Han et al., 2024a).”

    From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Pal et al., 2023

    The False Confidence Test (FCT) section describes a Med-HALT test for answers given with unnecessary certainty. But the Evaluation matrices section names accuracy and pointwise score as metrics, not a “false confidence rate”; the cited paper supports a false-confidence test, not the specific metric claimed. The indexed passages provide section labels but no page numbers. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Adversarial benchmarks like Med-HALT and targeted false-confidence probes stress-test models with challenging prompts designed to induce fabrications, thereby quantifying a model's propensity to hallucinate under duress.

    “Adversarial benchmarks like Med-HALT (Pal et al., 2023) and targeted”

    From Trustworthy Medical Question Answering: An Evaluation-Centric Survey (2025) · cites this paper as Pal et al., 2023

  • These frameworks typically focus on narrow clinical scenarios (1-28 datasets), rely on limited data sources or synthetic data, and assess primarily accuracy without comprehensive taxonomy validation or deployment considerations.

    “However, these frameworks typically focus on narrow clinical scenarios (1–28 datasets), rely on limited data sources or synthetic data, and assess primarily accuracy without comprehensive taxonomy validation or deployment considerations.”

    From Holistic evaluation of large language models for medical tasks with MedHELM (2026) · cites this paper as [24]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 1

  1. Ankit Pal, Logesh Kumar Umapathi, and Malaikannan Sankarasubbu. 2022. Medmcqa: A large-scale multi- subject multi-choice dataset for medical domain ques- tion answering. In Proceedings of the Conference on Health, Inference, and Learning, volume 174 of Proceedings of Machine Learning Research, pages 248-260. PMLR.

Cited by

Sorted by citations

Authors
Pal, A., Umapathi, L. K., & Sankarasubbu, M.
ORCID
Venue
Proceedings of the 27th Conference on Computational Natural Language Learning (CoNLL)
Publisher
Association for Computational Linguistics
Published
2023
Pages
314-334
DOI
10.18653/v1/2023.conll-1.21
OpenAlex
W4389519219
Cited by
152
References
18
Citations by year
2026: 62 · 2025: 59 · 2024: 28 · 2023: 3
FWCI
27.39
Type
conference paper
Language
en
Open access
Licence
cc-by
Locations
  • Conference on Computational Natural Language Learning (publishedVersion, cc-by)
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
Keywords
hallucinations, large language models, medical domain, benchmark, hallucination detection, reasoning tests, medical examination, medical applications, LLM evaluation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review