CoChat
Sign Up
PreprintCited by 1Topic ModelingMultimodal Machine Learning ApplicationsArtificial Intelligence in Healthcare and Education

Hallucination Detection-Guided Preference Optimization for Clinical Summarization

Open access
Seethakantha, Shamanth Kuthpadi, Thai, Dung Ngoc, Gudi, Vara Prasad, Tiwari, Simran, Matar, Rami, Mitra, Avijit, Zhao, Wenlong, McCallum, Andrew +1 more·2026·arXiv
DOI 10.48550/arxiv.2605.28910arXiv 2605.28910
Open this paper in CoChat — free accountOpen PDF

Abstract

Large language models (LLMs) have shown promise on summarization tasks, but they often produce hallucinations, which are unsupported or incorrect statements that limit their reliability in specialized healthcare applications. We introduce Hallucination Detection Guided Self-Refinement (HDSR), an inference-time method that leverages hallucination detectors to guide iterative summary revisions toward factual corrections. Building on this, we propose HDSR for Preference Learning (HDSR-PL), which converts detector-guided refinement trajectories into preference pairs for model finetuning. Extensive experiments show that our methods substantially reduce hallucinations for Llama and Gemma models in summarizing real-world clinical notes from MIMIC-IV-Note v2.2. For example, HDSR reduces 24% and HDSR-PL reduces 48% hallucinations in Llama-3.1-8B-Instruct. Importantly, both methods preserve summary fluency, coherence, and relevance according to human expert and LLM-Jury evaluations. Together, these results demonstrate that detection-informed refinement and preference learning offer an automated solution for improving factual faithfulness in clinical summarization.

AI snapshot

Written from the paper’s full text.

TL;DR

The paper introduces HDSR, an inference-time hallucination-detector-guided self-refinement method, and HDSR-PL, which converts refinement trajectories into preference pairs for DPO in clinical summarization. On MIMIC-IVNote v2.2 BHC-to-DI summarization, HDSR and HDSR-PL reduce hallucinations for Llama and Gemma models while preserving summary quality in human and LLM-jury evaluations.

Key findings

  • HDSR-PL with MedCat achieved the lowest hallucination count on LLaMA-3.1-8B-Instruct: 15, an approximate 48% reduction vs. zero-shot prompting (29) and a 74% reduction vs. SFT (57). HDSR with MedAlign reduced hallucinations to 22, about 24% below zero-shot.
  • SFT amplified hallucinations: hallucination count rose to 57 vs. 29 for zero-shot, while consistency dropped to 3.03 even though fluency (4.53) and coherence (4.43) improved.
  • HDSR improved summary quality: with MedAlign, hallucination count was 22, with consistency 4.13, coherence 4.48, fluency 4.53, relevance 3.95, and average 4.27. HDSR-PL with MedCat scored consistency 4.40, coherence 4.28, fluency 4.05, relevance 3.90, and average 4.16.
  • Detection signal mattered: Table 3 reports HDSR reduced hallucination count by 29% with MedAlign and 19% with MedCat vs. self-refinement without detectors (31 to 22 and 25, respectively). For smaller models, LLaMA-3.2-3B-Instruct HDSR-PL reached 13 hallucinations vs. 26 for prompting, and Gemma-3-4b-it HDSR-PL reached 13 vs. 15 for prompting.

Methods

  • HDSR applies a hallucination detector to a generated summary, uses detector feedback to prompt iterative revisions of unsupported or contradicted content, and repeats until a fixed iteration limit or no further hallucinations are detected. HDSR-PL treats the initial summary as non-preferred and the revised summary as preferred, then trains the summarization model with direct preference optimization.
  • Data: clinical summarization from Brief Hospital Course sections to Discharge Instructions using datasets derived from MIMIC-IVNote v2.2, evaluated on the hallucination-annotated subset Hallucination-Generated-DI. Detectors are MedCAT and a prompted detector following MedAlign annotation guidelines. Models include LLaMA-3.1-8B-Instruct, LLaMA-3.2-3B-Instruct, Gemma-3-4b-it, and GPT-5 zero-shot as a closed-source reference.
  • Evaluation: entity-level hallucination counts and clinician-based human evaluation on Consistency, Coherence, Fluency, and Relevance. For LLaMA-3.1-8B-Instruct, human evaluation is blind with double annotation and adjudication for hallucination labels; for LLaMA-3.2-3B-Instruct and Gemma-3-4b-it, hallucination counts are from a single clinician and qualitative evaluation uses automatic LLM-as-judge scores.

Limitations

  • Stated: both HDSR and HDSR-PL rely on the quality of the hallucination detector; detector errors such as false positives or false negatives can limit refinement and downstream preference learning.
  • Stated: evaluation is limited to clinical summarization from MIMIC-IV-Note v2.2, specifically BHC to Discharge Instructions generation, so transferability to other medical note types or non-clinical domains remains unexplored. HDSR incurs additional inference cost from multiple refinement iterations and detector calls, while HDSR-PL requires additional training resources and preference optimization.
  • Evident from the text: HDSR-PL slightly underperforms HDSR on fluency and coherence, and the smaller-model results use single-clinician hallucination counts and LLM-as-judge quality scores rather than the full blinded double-annotated human qualitative evaluation used for LLaMA-3.1-8B-Instruct.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

DataCite · 10.48550/arxiv.2605.28910

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

16 of 19 cited sources checked · 6 supported · 1 flagged · accuracy not yet measured

  • Premisecitation
    § 1 Introduction · p. 1

    Even domain-adapted models frequently introduce hallucinations expressed with medically plausible terminology, requiring expert review.

    “Even domain-adapted models frequently introduce hallucinations expressed with medically plausible terminology, requiring expert review (Hegselmann et al., 2024; Williams et al., 2025; Fang et al., 2024; Das et al., 2025), and underscoring the need for stricter factual reliability in clinical summarization than in general-domain tasks.”
    Quote in the paperSubstantial claim3 of 3 sources checked
    • cites Hegselmann et al., 2024 source

      Section 5.1 reports 114 hallucinations in 100 generated patient summaries, and §3.2 says the generating models had been trained for the study’s hallucination-reduction experiments. The Discussion (§6) emphasizes thorough evaluation by human domain experts. These passages support the concern about hallucinations and expert review, but do not establish that the hallucinations were expressed with medically plausible terminology. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Williams et al., 2025 source

      The Results report hallucinations in 42% of GPT-4 summaries, and the Discussion says some invented follow-up instructions were clinically appropriate despite being absent from the original notes. The Conclusion calls for clinician review and revision. However, the study evaluated GPT-4 and GPT-3.5-turbo, not domain-adapted models, and it does not specifically characterize the hallucinations as being expressed in medically plausible terminology (section labels available; page numbers not provided). (LLM, conf=0.87)

      Experimental — accuracy not yet measured
    • cites Fang et al., 2024 source

      Section 3.3 (page not provided) reports non-faithful hallucinations in plain biomedical summaries, supporting the general concern about factual reliability. It does not establish that domain-adapted models frequently hallucinate: it says the fine-tuned Flan-T5 had a low factual-hallucination rate. The passages also do not describe hallucinations as using medically plausible terminology or explicitly require expert review. (LLM, conf=0.85)

      Experimental — accuracy not yet measured
  • Premisecitation
    § 1 Introduction · p. 1

    Hallucinations remain prevalent in recent LLMs, with rates of roughly 40-50% on benchmarks such as FAITHBENCH.

    “Hallucinations remain prevalent even in recent LLMs, with rates of roughly 40-50% on benchmarks such as FAITHBENCH (Bao et al., 2025) indicating persistent factual unreliability.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Bao et al., 2025 source

      Table 1 reports unwanted-hallucination rates of 40–48% for four of the ten modern LLMs, supporting the claim that hallucinations remain prevalent. The other six models have higher rates, and §3.2 cautions that these results apply only to FaithBench’s challenging summarization samples, not necessarily to samples generally. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
  • Findingown data
    § Abstract · p. 1

    HDSR reduces hallucinations by approximately 24% in Llama-3.1-8B-Instruct.

    “For example, HDSR reduces 24% and HDSR-PL reduces 48% hallucinations in Llama-3.1-8B-Instruct.”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “our method reduces hallucinations by approximately 24% through self refinement”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The hallucination detectors (MedCat and MedAlign) reliably identify unsupported or hallucinated content in clinical summaries well enough to guide refinement and preference learning.
  • Findingown data
    § 3.2 Main Results · p. 3

    HDSR using MedAlign as the detector reduces hallucinations to 22.

    “Using MedAlign as the detector, HDSR reduces hallucinations to 22 while simultaneously achieving the best overall human-evaluated summary quality across all dimensions.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The hallucination detectors (MedCat and MedAlign) reliably identify unsupported or hallucinated content in clinical summaries well enough to guide refinement and preference learning.
  • Findingown data
    § Abstract · p. 1

    Our methods substantially reduce hallucinations for Llama and Gemma models in summarizing real-world clinical notes from MIMIC-IVNote v2.2.

    “Extensive experiments show that our methods substantially reduce hallucinations for Llama and Gemma models in summarizing real-world clinical notes from MIMIC-IVNote v2.2 .”
    Quote in the paperSubstantial claim
  • May combine two claims. 20
  • Classified as probably not a substantial claim. 9

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Training-time hallucination mitigation methods require large volumes of high-quality domain data and depend on factuality metrics that correlate weakly with clinical correctness, limiting their practical impact.

    “These methods require large volumes of high-quality domain data (Hegselmann et al., 2024) and depend on factuality metrics that correlate weakly with clinical correctness, limiting their practical impact.”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites Hegselmann et al., 2024 source

      The citation is attached to the claim that these methods require large volumes of high-quality domain data. The paper’s Related Work section says the opposite for sample-efficient LLMs: revising a small number of training examples is sufficient and feasible even manually. The later claim about factuality metrics is not the part attributed to this citation. (LLM, conf=0.90)

      Checked by a reasoning model — may be wrong.

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § Abstract · p. 1

    Large language models often produce hallucinations, which are unsupported or incorrect statements that limit their reliability in specialized healthcare applications.

    “Large language models (LLMs) have shown promise on summarization tasks, but they often produce hallucinations, which are unsupported or incorrect statements that limit their reliability in specialized healthcare applications.”
    • Also stated in: § 1 Introduction · p. 1 · plain
      “Large language models achieve strong summarization performance but often generate hallucinations, defined as content not supported by the source or inconsistent with world knowledge (Maynez et al., 2020; Ji et al., 2023a; Tang et al., 2023).”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites Maynez et al., 2020 source

      The BERTS2S passage reports frequent hallucinations in single-sentence summaries and says that over 90% of extrinsic hallucinations were erroneous. The Conclusion identifies hallucination as a critical challenge for abstractive summarization. The paper does not establish that large language models generally behave this way or assess their reliability in specialized healthcare applications. (LLM, conf=0.88)

      Experimental — accuracy not yet measured
    • cites Ji et al., 2023a

      Couldn’t be matched to a paper in Crossref, so it can’t be checked.

    • cites Tang et al., 2023 source

      The Discussion (‘What causes Factual Inconsistency?’; page not provided) says medical-evidence summaries often contain misinterpretation errors that can mislead readers, supporting the concern about incorrect statements. But it says fabricated, unsupported errors were relatively few, and the study evaluated medical-evidence summarization rather than specialized healthcare applications generally. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Retrieval-augmented generation grounds outputs in external documents but depends on retrieval quality.

    “retrieval-augmented generation grounds outputs in external documents yet depends on retrieval quality (Koopman and Zuccon, 2023; Wan et al., 2024; Kim et al., 2025)”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites Koopman and Zuccon, 2023 source
    • cites Wan et al., 2024 source
    • cites Kim et al., 2025 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    SYNFACEDIT generates synthetic edit feedback for preference optimization but relies on predefined error types and external edit models.

    “A closely related approach is SYNFACEDIT (Mishra et al., 2024), which generates synthetic edit feedback for preference optimization but relies on predefined error types and external edit models.”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Mishra et al., 2024 source

      Section 3 says SYNFAC-EDIT uses LLMs as synthetic edit functions to generate preference pairs, and Section 3.3 says it uses those pairs for preference training. Section 3.2 specifies ADD and OMIT operations, while the introduction identifies GPT-3.5 and GPT-4 as synthetic experts. These passages support the use of predefined edit operations and external models, but not the more specific claim that the method relies on predefined error types; Appendix C describes its four edit categories as observed from the pipeline and used for analysis. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation
    § 1 Introduction · p. 1

    LLMs achieve comparable performance with clinician-written summaries in clinical summarization.

    “Despite comparable performance with clinician-written summaries (Veen et al., 2024), hallucinations manifest as fabricated or distorted clinical statements that are hard to detect (Asgari et al., 2025; Kim et al., 2025).”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites Veen et al., 2024 source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Self-refinement and verification loops iteratively critique and revise generations, improving factuality at the cost of higher inference overhead and a risk of over-editing.

    “while self-refinement(Madaan et al., 2023) and verification loops (Tang et al., 2023) iteratively critique and revise generations, improving factuality at the cost of higher inference overhead and a risk of over-editing.”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites Madaan et al., 2023 source

      Figure 1 and Section 4 describe SELF-REFINE’s repeated feedback-and-revision loop. The Feedback and Refinement Error Analysis reports that revisions sometimes introduce new problems, and the repeated model calls imply additional inference work. The paper does not establish improved factuality for SELF-REFINE; the separately cited verification loops are outside this verdict. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites Tang et al., 2023 source

      The Discussion section on detecting factual inconsistency proposes using LLMs to identify potential errors in summaries, but describes this as a possible future approach. Our copy does not describe verification loops that iteratively revise generations, establish improved factuality from such loops, or discuss inference overhead or over-editing (searched: verification loops, revision, inference cost). (LLM, conf=0.75)

      Experimental — accuracy not yet measured

  • May combine two claims.

    Premisecitation
    § 1 Introduction · p. 1

    Hallucinations in clinical summarization manifest as fabricated or distorted clinical statements that are hard to detect.

    “Despite comparable performance with clinician-written summaries (Veen et al., 2024), hallucinations manifest as fabricated or distorted clinical statements that are hard to detect (Asgari et al., 2025; Kim et al., 2025).”
    Quote in the paperSubstantial claim2 of 2 sources checked
    • cites Asgari et al., 2025 source
    • cites Kim et al., 2025 source

  • May combine two claims.

    Findingown data
    § Abstract · p. 1

    Both HDSR and HDSR-PL preserve summary fluency, coherence, and relevance according to human expert and LLM-jury evaluations.

    “Importantly, both methods preserve summary fluency, coherence, and relevance according to human expert and LLMJury evaluations.”
    • Also stated in: § 1 Introduction · p. 2 · plain
      “while preserving summary quality and fluency according to both human and LLM-as-judge evaluations”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The hallucination detectors (MedCat and MedAlign) reliably identify unsupported or hallucinated content in clinical summaries well enough to guide refinement and preference learning.
    • Rests on an assumption: Automatic LLM-as-judge scores are a valid substitute for human qualitative evaluation for smaller models.

  • May combine two claims.

    Findingown data
    § 3.2 Main Results · p. 4

    Self-refinement without detection reduces some surface-level errors but leaves substantial unsupported content uncorrected.

    “Self-refinement without detection reduces some surface-level errors but leaves substantial unsupported content uncorrected.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning
    § 2.2 Hallucination Detection Guided Self-Refinement · p. 2

    Detector-guided refinement improves factual alignment while maintaining overall fluency.

    “This detector guided refinement improves factual alignment while maintaining overall fluency.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The hallucination detectors (MedCat and MedAlign) reliably identify unsupported or hallucinated content in clinical summaries well enough to guide refinement and preference learning.

  • Probably not a substantial claim: may not pass S4 (based).

    Findingnone given
    § 1 Introduction · p. 2

    Our method does not incur additional inference time cost.

    “and without incurring additional inference time cost.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data
    § 3.2 Main Results · p. 4

    SFT amplifies clinically critical error types, increasing unsupported conditions from 8 to 15 and procedures from 2 to 11.

    “SFT amplifies these clinically critical error types, increasing unsupported conditions from 8 to 15 and procedures from 2 to 11.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 3.2 Main Results · p. 3

    Supervised fine-tuning on clinician-written references substantially worsens factual alignment and increases hallucinations to 57, nearly twice the zero-shot baseline.

    “Supervised finetuning on clinician-written references substantially worsens factual alignment and increases hallucinations (57) to nearly twice the zero-shot baseline.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data
    § 3.2 Main Results · p. 3

    Supervised fine-tuning improves fluency (4.53) and coherence (4.43) but consistency drops sharply (3.03).

    “Although supervised fine-tuning improves fluency (4.53) and coherence (4.43), consistency drops sharply (3.03).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

    Study design and conductnone given
    § Medalign Chain-of-thought k-shot Prompt · p. 10

    To obtain reliable error counts, an annotated error span should contain only a single error.

    “To get reliable error counts a span should only contain a single error.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductown data
    § 3.1 Experimental Setup · p. 3

    Annotations for the main results were performed by a team of clinicians blinded to model identity; for the quantitative task, each summary was annotated independently by two annotators with adjudication of disagreements.

    “Annotations are performed by a team of clinicians blinded to model identity: for the quantitative task, each summary is annotated independently by two annotators (H1 and H2) with adjudication of disagreements, while for the qualitative analysis, two annotators score summaries independently without adjudication.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 2.3 Preference Learning from Detection Guided Self-Refinement · p. 3

    The DPO objective encourages the model to internalize detector-guided factual corrections, amortizing improvements from self-refinement into the model parameters and enabling faithful generation at inference time without additional refinement steps.

    “This objective encourages the model to internalize detector guided factual corrections, amortizing improvements from self refinement into the model parameters and enabling faithful generation at inference time without additional refinement steps.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The revised summary produced by HDSR has fewer hallucinations than the initial summary for each source document, making it the preferred output.

  • May combine two claims.

    Interpretationreasoning
    § Limitations · p. 5

    Both HDSR and HDSR-PL rely on the quality of the hallucination detector used to provide supervision.

    “First, both HDSR and HDSR-PL rely on the quality of the hallucination detector used to provide supervision.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § 3.2 Main Results · p. 3

    The trade-off between HDSR and HDSR-PL reflects that HDSR benefits from direct, entity-specific detector feedback, whereas HDSR-PL learns a more general preference for factual alignment that may smooth over fine-grained stylistic details but yields factually aligned outputs.

    “This trade-off reflects the difference between the two variants of our pipeline: HDSR benefits from direct, entity-specific detector feedback, whereas HDSR-PL learns a more general preference for factual alignment that may smooth over fine-grained stylistic details but yields factually aligned outputs.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § Limitations · p. 5

    Detector errors such as false positives that signal supported content or false negatives that miss subtle inconsistencies can limit the effectiveness of refinement and downstream preference learning.

    “While we treat detectors as black-box assessors, their errors-such as false positives that signal supported content or false negatives that miss subtle inconsistencies-can limit the effectiveness of refinement and downstream preference learning.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning
    § Limitations · p. 5

    HDSR-PL requires additional training resources and preference optimization, which may not be feasible in all deployment scenarios.

    “While HDSR-PL addresses this by amortizing refinement into training, it requires additional training resources and preference optimization, which may not be feasible in all deployment scenarios.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationreasoning
    § Limitations · p. 5

    Inference-time self-refinement (HDSR) incurs additional computational cost due to multiple refinement iterations and detector calls, which may limit its applicability in latency-sensitive settings.

    “Finally, inference-time self-refinement (HDSR) incurs additional computational cost due to multiple refinement iterations and detector calls, which may limit its applicability in latency-sensitive settings.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data
    § 3.1 Experimental Setup · p. 3

    For LLaMA-3.1-8B-Instruct, human evaluation was conducted in a blind setting with double annotation and adjudication for hallucination labels.

    “For LLaMA-3.1-8B-Instruct, human evaluation is conducted in a blind setting with double annotation and adjudication for hallucination labels, forming the basis of the main results in Table 1.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning
    § 3.1 Experimental Setup · p. 3

    For LLaMA-3.2-3B-Instruct and Gemma-3-4B-IT, due to resource constraints, hallucination counts were annotated by a single clinician and human qualitative evaluation was replaced with automatic LLM-as-judge scores.

    “For LLaMA-3.2-3B-Instruct and Gemma-3-4B-IT, due to resource constraints, we report clinicianprovided hallucination counts annotated by a single clinician (H1) and replace human qualitative evaluation with automatic LLM-as-judge scores.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Automatic LLM-as-judge scores are a valid substitute for human qualitative evaluation for smaller models.
24 of 24

  • The paper relies on this without support.

    The hallucination detectors (MedCat and MedAlign) reliably identify unsupported or hallucinated content in clinical summaries well enough to guide refinement and preference learning.

    “We use existing hallucination detectors from prior work, specifically MedCat (Kraljevic et al., 2021) and prompt based detectors following the MedAlign annotation guidelines (Fleming et al., 2024).”
    “While we treat detectors as black-box assessors, their errors-such as false positives that signal supported content or false negatives that miss subtle inconsistencies-can limit the effectiveness of refinement and downstream preference learning.”

  • The paper relies on this without support.

    The revised summary produced by HDSR has fewer hallucinations than the initial summary for each source document, making it the preferred output.

    “Detection-guided self-refinement produces pairs of summaries for each source document, consisting of an initial summary and a revised summary with fewer hallucinations. We treat the revised summary as the preferred output and the initial summary as the non-preferred output, thereby forming preference pairs without human annotation.”

  • The paper relies on this without support.

    Automatic LLM-as-judge scores are a valid substitute for human qualitative evaluation for smaller models.

    “For LLaMA-3.2-3B-Instruct and Gemma-3-4B-IT, due to resource constraints, we report clinicianprovided hallucination counts annotated by a single clinician (H1) and replace human qualitative evaluation with automatic LLM-as-judge scores.”

  • The paper relies on this without support.

    Most errors in after-visit summaries are due to unsupported facts.

    “Most errors are due to unsupported facts, so we further distinguish those based on their specific content.”
4 of 4

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • HDSR does not lead on every quality dimension

    Minor

    Table 1 gives HDSR-PL a higher consistency score than HDSR with MedAlign (4.40 versus 4.13), so MedAlign HDSR does not achieve the best score across all dimensions.

  • Three error types are not a majority for zero-shot generation

    Minor

    In Table 2, unsupported conditions, procedures, and medications total 11 errors for the prompting model (8 + 2 + 1), fewer than half of its 29 errors.

2 of 2

Cites

Flagged first, then by citations

11 of the 37 references couldn’t be matched to a known paper.

Research works printed but not linked: 1

  1. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Delong Chen, Wenliang Dai, Ho Shu Chan, Andrea Madotto, and Pascale Fung. 2023a. Survey of Hallucination in Natural Language Generation. ACM Computing Surveys , 55(12):1-38.

Cited by

Sorted by citations

Authors
Seethakantha, S. K., Thai, D. N., Gudi, V. P., Tiwari, S., Matar, R., Mitra, A., Zhao, W., McCallum, A., & Salloum, W.
Affiliations
  • Ensemble Therapeutics (United States) (US)
  • Columbia College (CA)
  • Manning Diversified Forest Products (Canada) (CA)
ORCID
Venue
arXiv
Publisher
arXiv
Published
2026
ISSN
2331-8422 (ISSN-L)
DOI
10.48550/arxiv.2605.28910
arXiv
2605.28910
OpenAlex
W7162774626
Semantic Scholar
114dabd1e7d9d56ec560def9b59729782384f3ca
Cited by
0
References
37
Type
posted content
Open access
Licence
Locations
  • arXiv (Cornell University) (submittedVersion, cc-by)
  • arXiv (Cornell University) (cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
hallucination detection, clinical summarization, preference learning, self-refinement, clinical notes, MIMIC-IV, large language models

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review