CoChat
Sign Up
Cited by 1Artificial Intelligence in Healthcare and EducationMachine Learning in HealthcareAdversarial Robustness in Machine Learning

MHB: Medical Hallucination Benchmark for Large Language Models in Complex Clinical Tasks

Open access
Lu, Jianrong, Liu, Junwei, Zheng, Xingyun, Yang, Minghui, Wang, Jian, Wang, Ping, Zhang, Yechao·2026·Proceedings of the AAAI Conference on Artificial Intelligence
DOI 10.1609/aaai.v40i45.41243
Open this paper in CoChat — free accountOpen PDF

Abstract

The integration of Large Language Models (LLMs) into clinical applications presents transformative potential but is undermined by the critical risk of hallucination, the generation of plausible but factually incorrect information. Such failures pose a direct threat to patient safety and the integrity of clinical decision-making. To address this challenge, we introduce MHB, a novel and comprehensive benchmark framework designed to evaluate LLM reliability in two complex, high-stakes clinical contexts: multi-turn medical dialogues and clinical case report analysis. The core of our contribution is a systematic methodology for generating adversarial test cases by injecting ``hallucination traps" into realistic medical data, guided by a fine-grained taxonomy of clinical errors. MHB, comprising 4,695 samples and 20,288 evaluation rubrics, underwent a rigorous, two-stage validation by a panel of 60 licensed physicians from top-tier hospitals, ensuring high clinical realism and consistency. This comprehensive assessment of leading LLMs revealed significant, clinically relevant shortcomings across the board. Even the best-performing model, Claude-4-Sonnet, exhibited a hallucination rate of 29.1%, with some open-source models exceeding 57.0%. All models struggled with specific traps, like fabricated medical data or non-existent guidelines, highlighting prevalent systemic weaknesses.

AI snapshot

Written from the paper’s full text.

TL;DR

MHB is a dual-task benchmark for medical hallucination in multi-turn clinical dialogues and clinical case report analysis, built by injecting adversarial “hallucination traps” into realistic medical data. Across leading LLMs, the best combined hallucination rate was 29.1% (Claude-4-Sonnet), while some open-source models exceeded 57.0%.

Key findings

  • On the combined MHB benchmark, Claude had the lowest aggregate hallucination rate at 0.291, followed by Gemini (0.333) and DeepSeek (0.388); Qwen and Llama had the highest rates at 0.590 and 0.575, respectively.
  • On MHB-Dialogue, Claude again performed best with a hallucination rate of 0.165, while Llama was worst at 0.793; Qwen was 0.513, DeepSeek 0.454, and Gemini and GPT both 0.357. Trap types A1 (Non-existent Entity) and D2 (Source Fabrication) consistently induced high hallucination frequencies across most models.
  • On MHB-Report, Gemini had the lowest rate at 0.323, followed by Claude (0.346) and DeepSeek (0.360); Qwen had the highest rate at 0.624, and GPT was 0.533. Trap type A3 (Incorrect Factual Claim) triggered high hallucination rates across multiple models, and cluster E (Temporal Obsoletion Traps) was a prominent vulnerability for GPT and Llama.
  • Combined categorical analysis identified clusters A (Factual Discrepancy) and D (Procedural Error) as common weaknesses, with D2 and C3 representing persistent challenges for current LLMs.

Methods

  • Design: MHB comprises two sub-datasets, MHB-Dialogue and MHB-Report, built from a five-category, 14-subtype taxonomy of medical hallucination traps. Adversarial test cases were generated by injecting traps into realistic medical data, with parallel automated rubric curation.
  • Data and sample: MHB contains 4,695 samples and 20,288 evaluation rubrics, averaging about four rubrics per sample. MHB-Dialogue is based on OpenAI’s HealthBench (5,000 multi-turn medical conversations), and MHB-Report uses PMC-Patients (over 160,000 patient summaries). Quality assurance involved 60 licensed physicians from Chinese Grade-A tertiary hospitals using dual-annotator cross-verification and four criteria: naturalness/plausibility, factual correctness of the trap, logical coherence, and rubric-trap alignment.
  • Analysis: Trap embedding used Google Gemini 2.5-Pro. Evaluated models were Gemini 2.5-Pro, Claude-4-Sonnet, GPT-4.1-2025-4-14, Qwen3-235B-A22B, DeepSeek-R1-0528, and Llama-4-Scout-17B-16E-Instruct. DeepSeek-R1-0528 served as the LLM judge at temperature 0.5. The hallucination rate is the proportion of rubrics not passed out of total rubrics assessed.

Limitations

  • Stated: the binary metric only identifies the presence of hallucinations, not their clinical severity or potential harm; future work should use risk-stratified metrics considering clinical impact.
  • Stated: the study tested specific adversarial manipulations, and models may have other vulnerabilities; model rankings will change as technology evolves, though the authors argue the systemic vulnerabilities and evaluation framework remain relevant.
  • Evident from the text: evaluation relies on a single LLM-as-a-judge framework (DeepSeek-R1-0528) with automated rubric scoring; the paper does not report independent validation of judge reliability beyond expert rubric curation.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1609/aaai.v40i45.41243

Cites

0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

5 of 7 cited sources checked · 1 supported · 0 flagged · accuracy not yet measured

  • Premisereasoning

    Existing benchmarks, such as Med-HALT, do not adequately stress-test models in more ecologically valid scenarios like multi-turn dialogues or case report analysis.

    “Existing benchmarks, such as Med-HALT (Pal, Umapathi, and Sankarasubbu 2023), do not adequately stress-test models in these more ecologically valid scenarios.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Pal, Umapathi, and Sankarasubbu 2023 source
  • Premisecitation

    For MHB-Report, the paper leveraged the PMC-Patients dataset, the first large-scale, structured clinical case dataset extracted from case reports in PubMed Central.

    “For the MHB-Report dataset, we leveraged the PMC-Patients dataset (Zhao et al. 2023), the first large-scale, structured clinical case dataset extracted from case reports in PubMed Central (PMC).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Zhao et al. 2023 source

      The Background & Summary says PMC-Patients contains patient summaries extracted from PMC case reports, and Technical Validation > Dataset characteristics describes it as the largest patient-summary dataset by number of patients. The source does not establish that it was the first large-scale, structured clinical case dataset; its claim of being first concerns large-scale patient-similarity annotations. The citing authors’ statement that they leveraged it for MHB-Report is their own account, not a claim this source must verify. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Premisecitation

    Current paradigms for evaluating hallucination of medical LLMs often fall short of capturing the complexities of real-world clinical practice.

    “current paradigms for evaluating hallucination of medical LLMs often fall short of capturing the complexities of real-world clinical practice (Arias-Duart et al. 2025).”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Arias-Duart et al. 2025 source

      The Introduction (page not provided) says that multiple-choice evaluation checks factuality but does not reflect the complexity of real-world clinical settings. The Limitations section similarly cautions that benchmarks may not reflect real-world healthcare practice. The paper supports the broader concern about evaluation, but these passages do not specifically frame it as an assessment of hallucination-evaluation paradigms. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
  • Findingown data

    On MHB-Report, cluster E (Temporal Obsoletion Traps) was a prominent vulnerability, particularly for GPT and Llama.

    “When analyzed by hallucination cluster (see supplementary), cluster E (Temporal Obsoletion Traps) emerged as a prominent vulnerability, particularly for GPT and Llama.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The judge LLM's rubric-guided judgments accurately reflect whether target models hallucinate.
  • Findingown data

    On MHB-Dialogue, Llama had the highest hallucination rate of 0.793.

    “Conversely, Llama was the most susceptible to hallucination, with a rate of 0.793.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The judge LLM's rubric-guided judgments accurately reflect whether target models hallucinate.
  • May combine two claims. 18
  • Classified as probably not a substantial claim. 28
  • The quote only partly states this claim. 5

  • May combine two claims.

    Premisecitation

    Broad-spectrum evaluations like Med-Bench recognize hallucination as an error category but do not provide a dedicated mechanism for systematically inducing and diagnosing it.

    “Similarly, broad-spectrum evaluations like Med-Bench (Liu et al. 2024) recognize hallucination as an error category but do not provide a dedicated mechanism for systematically inducing and diagnosing it.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Liu et al. 2024 source

      The Introduction (p. 2) and Conclusion (p. 11) describe MedBench as a broad-spectrum evaluation across 43 clinical specialties. The Dynamic Evaluation Mechanisms section (p. 5) describes choice shuffling and random prompt matching to address shortcut learning and response-pattern bias, not hallucination induction or diagnosis. Searches for hallucination and a dedicated mechanism found no passage in our copy that recognizes hallucination as an error category; that part of the claim is unsupported. (LLM, conf=0.78)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation

    Med-HALT's primary focus is on single-turn question-answering, and it lacks a systematic framework for probing diverse, fine-grained hallucination subtypes within more complex clinical tasks.

    “Med-HALT (Pal, Umapathi, and Sankarasubbu 2023) evaluates models on reasoning and memory-based hallucination tasks.However, its primary focus is on single-turn question-answering, and it lacks a systematic framework for probing diverse, finegrained hallucination subtypes within more complex clinical tasks.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Pal, Umapathi, and Sankarasubbu 2023 source

      Section 1.1 defines a question-and-options task, and Limitations & Future Scope says the assessment was limited to reasoning and information retrieval, supporting the characterization of its narrow task focus. But Sections 3 and 3.1 describe a two-tiered hallucination framework with several distinct tests. The paper does not establish that it lacks a systematic framework altogether, or specifically assess its coverage of fine-grained subtypes in complex clinical tasks. (LLM, conf=0.85)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Findingnone given

    The choice of trap is tailored to the dialogue's specific medical context to maximize plausibility.

    “This process is context-aware; the choice of trap is tailored to the dialogue's specific medical context to maximize plausibility.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    MHB provides a dual-task framework that assesses reliability in both dynamic, multi-turn dialogues and the analysis of complex clinical reports.

    “In contrast to prior work, it provides: (1) a dual-task framework that assesses reliability in both dynamic, multi-turn dialogues and the analysis of complex clinical reports”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The construction of both datasets followed a principled, automated pipeline designed to ensure scalability, consistency, and ecological validity.

    “The construction of both followed a principled, automated pipeline designed to ensure scalability, consistency, and ecological validity, as illustrated in Figure 1.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    MHB comprises 4,695 samples and 20,288 rubrics, with each sample annotated by an average of four rubrics.

    “The benchmark comprises 4,695 samples and 20,288 rubrics, with each sample annotated by an average of four rubrics.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    MHB is a comprehensive, dual-task framework designed to assess LLM reliability in high-stakes clinical scenarios that go beyond simple QA.

    “We introduce the MHB benchmark, a comprehensive, dual-task framework designed to assess LLM reliability in high-stakes clinical scenarios that go beyond simple QA.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Findingown data

    The hallucination traps taxonomy is organized into five primary categories: Factual Discrepancy, Cognitive Inconsistency, Diagnostic Reasoning, Procedural Error, and Temporal Obsoletion.

    “The hallucination traps are organized into five primary categories, as detailed in Table 1.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The three largest clusters in MHB are A (3,108 samples), C (2,628 samples), and D (2,643 samples).

    “The three largest clusters-A (3,108 samples), C (2,628 samples), and D (2,643 samples)-capture the most prevalent and clinically critical types of medical hallucinations.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    MHB-Report is a sub-dataset that evaluates a model's factuality and reasoning when analyzing dense, complex clinical case narratives.

    “MHB-Report: A sub-dataset evaluates a model's factuality and reasoning when analyzing dense, complex clinical case narratives.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Current leading models exhibit a high hallucination rate, with the lowest observed rate at 29.1% and the highest at 59.0%.

    “Our findings reveal that current leading models exhibit a high hallucination rate, with the lowest observed rate at 29.1% and the highest at 59.0%.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    The data generation process involved four key stages.

    “Our data generation process involved four key stages:”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Most evaluated models struggled with clusters A, C, and D, with Llama and Qwen having substantially larger hallucination rates than Claude and Gemini.

    “The performance disparity between models was not uniform; while most models struggled with cluster A, C, and D, the magnitude of the hallucination rate was substantially larger for models like Llama and Qwen compared to the more resilient Claude and Gemini.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The judge LLM's rubric-guided judgments accurately reflect whether target models hallucinate.

  • May combine two claims.

    Findingown data

    MHB comprises 4,695 samples and 20,288 evaluation rubrics.

    “MHB, comprising 4,695 samples and 20,288 evaluation rubrics”
    • Also stated in: § The Fortieth AAAI Conference on Artificial Intelligence (AAAI-26) · plain
      “our 4,695 samples and 20,288 rubrics”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Findingown data

    On MHB-Dialogue, Claude demonstrated superior and consistent performance across hallucination trap clusters.

    “Trap cluster A (Factual Discrepancy Traps) and D (Procedural Error Traps), in particular, proved exceptionally challenging for Llama, while Claude again demonstrated superior and consistent performance.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The judge LLM's rubric-guided judgments accurately reflect whether target models hallucinate.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The five high-level clusters are further broken down into 14 fine-grained subtypes.

    “These five high-level clusters are further broken down into 14 fine-grained subtypes, as shown in Table 1, providing a granular framework for both generating targeted hallucination modes and analyzing model weaknesses within medical contexts.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    MHB-Dialogue is a sub-dataset built on realistic, multi-turn medical conversations to evaluate model hallucination in dynamic, interactive patient-provider dialogues.

    “MHB-Dialogue: A sub-dataset built on realistic, multi-turn medical conversations to evaluate model hallucination in dynamic, interactive patient-provider dialogues.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    For MHB-Report, the questions were engineered to tempt the target model into making unsupported inferences via progressive question structures.

    “For the MHB-Report dataset, the questions were engineered to tempt the target model into making unsupported inferences via progressive question structures.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    The progressive induction strategy for MHB-Dialogue restructures the semantics and adjusts the logic of the dialogue from both the user and the assistant to weave suggestive context and misleading logical cues, creating a strong cognitive bias.

    “For the MHB-Dialogue dataset, instead of inserting a single, isolated false statement, the generator subtly restructures the semantics and adjusts the logic of the dialogue from both the user and the assistant.This weaves a web of suggestive context and misleading logical cues, creating a strong cognitive bias.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    For MHB-Report, the generator LLM was prompted to design a clinically plausible question that, while superficially relevant, could not be answered from the provided text.

    “the generator was prompted to design a clinically plausible question that, while superficially relevant, could not be answered from the provided text.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    The process culminates in a carefully crafted final question from the user, designed to provoke the target model into generating a hallucinated response based on the previously embedded traps.

    “The process culminates in a carefully crafted final question from the user, which acts as bait, designed to provoke the target model into generating a hallucinated response based on the previously embedded traps.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingnone given

    The pipeline includes a detailed taxonomy of clinical hallucination traps and expert-validated evaluation rubrics in parallel.

    “This pipeline includes a detailed taxonomy of clinical hallucination traps and expert-validated evaluation rubrics in parallel.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). The quote only partly states this claim.

    Study design and conductreasoning

    Reviewers checked that the trap was logically and semantically consistent with the surrounding text.

    “The trap must be logically and semantically consistent with the surrounding text in the dialogue or medical report.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    Expert reviewers assessed whether the trap-infused dialogue or question flows naturally within a clinical context.

    “This criterion evaluates the subtlety and realism of the embedded trap.The expert reviewers assess whether the trap-infused dialogue or question flows naturally within a clinical context.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    The expert panel was recruited from Grade-A tertiary hospitals in China and had extensive clinical experience across specialties including internal medicine, surgery, oncology, and pediatrics.

    “These experts are recruited from the most prestigious Grade-A tertiary hospitals in China and possess extensive clinical experience across various specialties, including internal medicine, surgery, oncology, and pediatrics.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductown data

    Each generated sample is paired with an automatically generated evaluation rubric and validated by a panel of physician experts to ensure clinical plausibility and logical coherence.

    “Each generated sample is paired with an automatically generated evaluation rubric and rigorously validated by a panel of physician experts to ensure clinical plausibility and logical coherence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisereasoning

    General-domain hallucination benchmarks lack the specificity required for medicine and do not address the unique error modalities and severe clinical risks associated with medical hallucinations.

    “While foundational, these general-domain benchmarks lack the specificity required for medicine.They do not address the unique error modalities and severe clinical risks associated with medical hallucinations, such as fabricating diagnostic results, misstating contraindications, or suggesting inappropriate clinical workflows.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. The quote only partly states this claim.

    Study design and conductnone given

    Reviewers confirmed that the model-generated evaluation rubric was precisely and unambiguously aligned with the specific hallucination trap.

    “The reviewer confirms that the model-generated evaluation rubric is precisely and unambiguously aligned with the specific hallucination trap embedded in the sample.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    The multi-stage expert review process ensures the exceptional quality of the 4,695 samples and 20,288 rubrics, making MHB a highly reliable resource for evaluating LLM safety.

    “This stringent, multi-stage expert review process ensures the exceptional quality of our 4,695 samples and 20,288 rubrics, making MHB a highly reliable resource for evaluating LLM safety.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Study design and conductnone given

    This study provides a robust benchmark to guide clinical integration and future development.

    “This study provides a robust benchmark to guide this process and future development.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductreasoning

    Samples with inadequate rubrics were rejected and regenerated, with the new version re-entering the review cycle from the beginning.

    “If a sample's generated rubric is deemed inadequate by either reviewer, it is rejected and regenerated.The new version then re-enters the review cycle from the beginning.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The judge LLM was instructed to determine if the target model's response satisfies the criteria in the provided rubric.

    “The judge LLM is instructed to act as an impartial evaluator.Its sole task is to determine if the target model's response satisfies the criteria outlined in the provided rubric.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The parallel generation of trap-embedded dialogues/questions and their corresponding evaluation rubrics creates a tightly coupled, closed-loop system ensuring high internal consistency between them.

    “This parallel generation of trapembedded dialogue/questions, as well as their corresponding evaluation standards, creates a tightly coupled, closedloop system, ensuring high internal consistency between the trap-embedded dialogue/questions and their traps, thus facilitating automated, scalable evaluation.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    For each generated rubric, one physician annotates whether the model-generated rubric aligns with the injected trap, and another physician independently checks the correctness of this annotation.

    “For each generated rubric, one physician annotates whether the model-generated rubric aligns with the injected trap; simultaneously, another physician independently checks the correctness of this annotation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S4 (based). May combine two claims.

    Study design and conductnone given

    The automated pipeline establishes a closed-loop system where the criteria for generating a trap are the same as those for evaluating it, ensuring high fidelity and consistency.

    “This automated pipeline establishes a closed-loop system where the criteria for generating a trap are the same as those for evaluating it, ensuring high fidelity and consistency.”
    Quote in the paperNot a substantial claim

  • May combine two claims. The quote only partly states this claim.

    Interpretationreasoning

    The validation ensured high clinical realism and consistency.

    “ensuring high clinical realism and consistency.”
    • Also stated in: § The Fortieth AAAI Conference on Artificial Intelligence (AAAI-26) · boosted
      “This stringent, multi-stage expert review process ensures the exceptional quality of our 4,695 samples and 20,288 rubrics, making MHB a highly reliable resource for evaluating LLM safety.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Each sample was reviewed independently by two physicians, with a secondary reviewer verifying the primary reviewer's annotations and conclusions.

    “The initial review is conducted by two physicians independently.”
    • Also stated in: § Model Response · plain
      “Subsequently, a Secondary Reviewer independently verifies the primary reviewer's annotations and conclusions.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    For samples with dissenting opinions, two additional physicians are engaged for re-evaluation to achieve consensus.

    “For any samples with dissenting opinions, two additional physicians are engaged for re-evaluation, ensuring that the final sample achieves consensus between the reviewing and checking physicians.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    If a generated rubric does not meet requirements, it is regenerated and re-evaluated until satisfactory.

    “If the generated rubric does not meet the requirements, it is regenerated and re-evaluated until satisfactory.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationown data

    These results highlight specific, recurring failure modes, offering critical insights for the development of safer and more reliable medical AI.

    “These results highlight specific, recurring failure modes, offering critical insights for the development of safer and more reliable medical AI.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    All models were accessed and queried via their official APIs to ensure standardized and reproducible evaluation conditions.

    “To ensure standardized and reproducible evaluation conditions, all models were accessed and queried via their official APIs.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    The identified vulnerabilities may stem from fundamental differences in model architecture, training data, or fine-tuning procedures.

    “This suggests that the identified vulnerabilities may stem from fundamental differences in model architecture, training data, or fine-tuning procedures.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: The judge LLM's rubric-guided judgments accurately reflect whether target models hallucinate.
42 of 42

  • The paper relies on this without support.

    Clinical reasoning is dynamic, interactive, and context-dependent.

    “Clinical reasoning is not a single-turn, static process.It is dynamic, interactive, and context-dependent.”

  • The paper relies on this without support.

    Multi-turn patient dialogues and synthesizing dense, unstructured case reports are more representative of future clinical AI applications than answering isolated medical questions.

    “Tasks such as engaging in multi-turn patient dialogues or synthesizing dense, unstructured case reports are far more representative of future clinical AI applications than answering isolated medical questions.”

  • The paper relies on this without support.

    Multi-turn conversational dialogues and analysis of complex case reports are high-risk, high-reward applications where LLMs could either provide immense value or cause significant harm.

    “We focus on these areas because they represent high-risk, high-reward applications where LLMs could either provide immense value or cause significant harm.”

  • The paper relies on this without support.

    The judge LLM's rubric-guided judgments accurately reflect whether target models hallucinate.

    “Its sole task is to determine if the target model's response satisfies the criteria outlined in the provided rubric.”

  • The paper relies on this without support.

    The proportion of rubrics not passed is a valid measure of hallucination rate.

    “The hallucination rate is calculated as the proportion of rubrics that are not passed out of the total rubrics assessed.”

  • The paper relies on this without support.

    The selected models are representative of contemporary large language models.

    “we evaluated a suite of state-of-the-art (SOTA) large language models.”
6 of 6

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Figure caption says five stages but lists four

    Minor

    The caption states that the workflow has five key stages, but its numbered list runs only from (1) to (4).

1 of 1

Cites

The publisher hasn’t deposited this paper’s reference list.

Research works printed but not linked: 1

  1. Fleming, S. L.; Lozano, A.; Haberkorn, W. J.; Jindal, J. A.; Reis, E.; Thapa, R.; Blankemeier, L.; Genkins, J. Z.; Stein- berg, E.; Nayak, A.; et al. 2024. Medalign: A clinician- generated dataset for instruction following with electronic medical records. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 22021-22030.

Cited by

Sorted by citations

Related

Authors
Lu, J., Liu, J., Zheng, X., Yang, M., Wang, J., Wang, P., & Zhang, Y.
Affiliations
  • Ant Group (China) (CN)
  • Zhejiang University (CN)
  • Peking University (CN)
  • Nanyang Technological University (SG)
Venue
Proceedings of the AAAI Conference on Artificial Intelligence
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Published
2026-03-14
Volume
40
Issue
45
Pages
38971-38978
ISSN
  • 2159-5399 (ISSN-L)
  • 2374-3468
DOI
10.1609/aaai.v40i45.41243
OpenAlex
W7138995235
Cited by
1
References
0
Citations by year
2026: 1
FWCI
9.36
Type
conference paper
Language
en
Open access
Locations
  • Proceedings of the AAAI Conference on Artificial Intelligence (publishedVersion)
  • AAAI Publications (The Association for the Advancement of Artificial Intelligence (AAAI)) (submittedVersion)
Funders
Ant Group
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
Keywords
hallucinations, large language models, clinical decision support, patient safety

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-29.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-29.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review