Beyond Literal Summarization: Redefining Hallucination for Medical SOAP Note Evaluation
Open accessAbstract
Evaluating large language models (LLMs) for clinical documentation tasks such as SOAP note generation remains challenging. Unlike standard summarization, these tasks require clinical abstraction, normalization of colloquial language, and medically grounded inference. However, prevailing evaluation methods including automated metrics and LLM as judge frameworks rely on lexical faithfulness, often labeling any information not explicitly present in the transcript as hallucination. We show that such approaches systematically misclassify clinically valid outputs as errors, inflating hallucination rates and distorting model assessment. Our analysis reveals that many flagged hallucinations correspond to legitimate clinical transformations, including synonym mapping, abstraction of examination findings, diagnostic inference, and guideline consistent care planning. By aligning evaluation criteria with clinical reasoning through calibrated prompting and retrieval grounded in medical ontologies we observe a significant shift in outcomes. Under a lexical evaluation regime, the mean hallucination rate is 35%, heavily penalizing valid reasoning. With inference aware evaluation, this drops to 9%, with remaining cases reflecting genuine safety concerns. These findings suggest that current evaluation practices over penalize valid clinical reasoning and may measure artifacts of evaluation design rather than true errors, underscoring the need for clinically informed evaluation in high context domains like medicine.
AI snapshot
Written from the paper’s full text.
TL;DR
The paper argues that lexical hallucination evaluation systematically misclassifies clinically valid SOAP note inferences as hallucinations, and proposes an inference-aware LLM-as-judge using calibrated prompting and retrieval from medical ontologies. Across 100 transcripts, the mean hallucination rate falls from 35.2% under inference-unaware judging to 9.1% under inference-aware judging, near the 10.4% human annotator baseline.
Key findings
- Under Stage 1 (inference-unaware judge), the mean hallucination rate was 35.2%, nearly three times the human annotator baseline of 10.4%; Stage 2 (inference-aware judge) reduced this to 9.1%, closing the gap to within 1.3 percentage points.
- Stage 1 false positives were dominated by diagnostic inference, terminology translation, and standard-of-care implications; Stage 2 reclassified the vast majority as supported, with residual hallucinations comprising primarily fabricated past medical history and unsupported medication entries.
- Both stages agreed on Tier 4–5 cases, confirming that inference awareness did not introduce permissiveness toward genuine fabrications or contradictions.
- The five-tier classification distinguishes supported direct restatement, paraphrase/medical translation, trade/generic equivalence, and clinical inference from hallucinated no-basis or contradicted claims; specific resolved examples include “lisinopril” vs. “zestril” and linking a penicillin allergy in Subjective to a non-penicillin H. pylori regimen in Plan.
Methods
- Dataset: 100 anonymized physician-patient transcripts covering specialties including Cardiology, Dermatology, Endocrinology, ENT, and Family Medicine; SOAP notes were generated using an LLM for each transcript.
- Two LLM-as-judge configurations: Stage 1 used strict factual grounding and flagged any claim not explicitly stated in the transcript as hallucinated; Stage 2 used an inference-aware prompt, five-step chain of thought, cross-section verification, a non-negotiable trade/generic synonym rule, retained safety-floor hallucination conditions, and retrieval-augmented clinical knowledge from SNOMED CT, ICD-10, and open-source pharmacological resources.
- Analysis compared mean hallucination rates across Stage 1, Stage 2, and human annotators, with a claim-level tier breakdown for a representative transcript.
Limitations
- The paper does not state a dedicated limitations section.
- Evident from the text: the evaluation uses 100 anonymized transcripts and LLM-generated SOAP notes, but the generating LLM(s) and specialty distribution are not specified, limiting generalizability.
- Evident from the text: the human annotator baseline is reported as 10.4%, but annotator protocol, number, and inter-annotator agreement are not detailed; Stage 2 combines prompt redesign and retrieval, so the separate contribution of each component is not isolated.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
- Findingown data
Stage 2 mean hallucination rate (9.1%) achieves near parity with human judgment.
“Stage 2 reduced this to 9.1%, achieving near parity with human judgment and confirming that the excess flagging under Stage 1 reflects evaluation design artifacts rather than genuine model errors.”
Quote in the paperSubstantial claim- Rests on an assumption: Human evaluation is the gold standard for assessing clinical documentation.
- Rests on an assumption: Expected inter-annotator variability in clinical review is around 1.3 percentage points or more.
- Findingown data
Stage 1 mean hallucination rate (35.2%) is nearly three times the human annotator baseline (10.4%).
“Under Stage 1, the mean hallucination rate reached 35.2%, nearly three times the human annotator baseline of 10.4%.”
Quote in the paperSubstantial claim - Findingown data
With inference-aware evaluation, the mean hallucination rate drops to 9.1%.
“With inference aware evaluation, this drops to 9%, with remaining cases reflecting genuine safety concerns.”
- Also stated in: § 5 Results · plain
“Stage 2 reduced this to 9.1%, achieving near parity with human judgment and confirming that the excess flagging under Stage 1 reflects evaluation design artifacts rather than genuine model errors.”
Quote in the paperSubstantial claim- Rests on an assumption: The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
- Also stated in: § 5 Results · plain
- Findingown data
The human annotator baseline mean hallucination rate is 10.4%.
“Under Stage 1, the mean hallucination rate reached 35.2%, nearly three times the human annotator baseline of 10.4%.”
Quote in the paperSubstantial claim- Rests on an assumption: Human evaluation is the gold standard for assessing clinical documentation.
- Findingown data
Stage 2 closes the gap between Stage 1 and human annotators to within 1.3 percentage points.
“Stage 2 closes this gap to within 1.3 percentage points, consistent with expected inter annotator variability in clinical review.”
Quote in the paperSubstantial claim
- Findingown data
The results demonstrate a substantial reduction in flagged hallucinations when the judge is equipped with clinically informed criteria.
“The results demonstrate a substantial reduction in flagged hallucinations when the judge is equipped with clinically informed criteria.”
Quote in the paperSubstantial claim- Rests on an assumption: The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
- Findingown data
Many flagged hallucinations correspond to legitimate clinical transformations, including synonym mapping, abstraction of examination findings, diagnostic inference, and guideline-consistent care planning.
“Our analysis reveals that many flagged hallucinations correspond to legitimate clinical transformations, including synonym mapping, abstraction of examination findings, diagnostic inference, and guideline consistent care planning.”
Quote in the paperSubstantial claim- Rests on an assumption: The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
- Findingreasoning
Under Stage 1 (inference-unaware judge), the judge systematically flagged clinically valid inferences-differential diagnoses, medical terminology translations of lay symptoms, and standard-of-care implications-as hallucinations.
“Under this framework, the judge systematically flagged clinically valid inferences differential diagnoses, medical terminology translations of lay symptoms, and standard of care implications as hallucinations, resulting in a mean hallucination rate, well above clinically meaningful thresholds.”
Quote in the paperSubstantial claim- Rests on an assumption: The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
- Findingown data
Stage 2's chain-of-thought protocol correctly reclassified all such Tier 3 cases as supported.
“Stage 2's chain of thought protocol correctly reclassified all such cases as supported.”
Quote in the paperSubstantial claim - Findingown data
Both stages agree on Tier 4-5 cases.
“Both stages agree on Tier 4-5 cases, confirming that inference awareness does not introduce permissiveness toward genuine fabrications or contradictions.”
Quote in the paperSubstantial claim - Premisenone given
Prevailing evaluation methods including automated metrics and LLM-as-judge frameworks rely on lexical faithfulness, often labeling any information not explicitly present in the transcript as hallucination.
“However, prevailing evaluation methods including automated metrics and LLM as judge frameworks rely on lexical faithfulness, often labeling any information not explicitly present in the transcript as hallucination.”
Quote in the paperSubstantial claim - Interpretationown data
The excess flagging under Stage 1 reflects evaluation design artifacts rather than genuine model errors.
“Stage 2 reduced this to 9.1%, achieving near parity with human judgment and confirming that the excess flagging under Stage 1 reflects evaluation design artifacts rather than genuine model errors.”
Quote in the paperSubstantial claim - Premisereasoning
Evaluating clinical documentation using summarization-based criteria introduces a fundamental mismatch.
“As a result, evaluating clinical documentation using summarization based criteria introduces a fundamental mismatch.”
Quote in the paperNot a substantial claim - Interpretationreasoning
The gap between Stage 1 and human annotators (24.8 percentage points) represents claims that trained clinicians considered valid but a strictly literal framework flagged as hallucinations.
“As reported in Table 3, the gap between Stage 1 and human annotators (24.8 percentage points) represents claims that trained clinicians considered valid but a strictly literal framework flagged as hallucinations.”
Quote in the paperSubstantial claim- Rests on an assumption: Human evaluation is the gold standard for assessing clinical documentation.
- Interpretationown data
Inference awareness does not introduce permissiveness toward genuine fabrications or contradictions.
“Both stages agree on Tier 4-5 cases, confirming that inference awareness does not introduce permissiveness toward genuine fabrications or contradictions.”
Quote in the paperSubstantial claim - Interpretationown data
Remaining cases under inference-aware evaluation reflect genuine safety concerns.
“With inference aware evaluation, this drops to 9%, with remaining cases reflecting genuine safety concerns.”
Quote in the paperSubstantial claim - Interpretationown data
Naive hallucination detection which flags any claim not explicitly stated in a source transcript dramatically overestimates hallucination rates in SOAP note generation, conflating clinically valid inference with fabrication.
“This paper demonstrates that naive hallucination detection which flags any claim not explicitly stated in a source transcript dramatically overestimates hallucination rates in SOAP note generation, conflating clinically valid inference with fabrication.”
Quote in the paperSubstantial claim - Interpretationnone given
The 1.3 percentage point gap is consistent with expected inter-annotator variability in clinical review.
“Stage 2 closes this gap to within 1.3 percentage points, consistent with expected inter annotator variability in clinical review.”
Quote in the paperNot a substantial claim- Rests on an assumption: Expected inter-annotator variability in clinical review is around 1.3 percentage points or more.
- The quote only partly states this claim. 1
- May combine two claims. 9
- Classified as probably not a substantial claim. 14
The quote only partly states this claim.
Findingown dataUnder a lexical evaluation regime, the mean hallucination rate is 35.2%.
“Under a lexical evaluation regime, the mean hallucination rate is 35%, heavily penalizing valid reasoning.”
- Also stated in: § 5 Results · plain
“Under Stage 1, the mean hallucination rate reached 35.2%, nearly three times the human annotator baseline of 10.4%.”
Quote in the paperSubstantial claim- Rests on an assumption: The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
- Also stated in: § 5 Results · plain
May combine two claims.
Findingown dataSuch approaches systematically misclassify clinically valid outputs as errors, inflating hallucination rates and distorting model assessment.
“We show that such approaches systematically misclassify clinically valid outputs as errors, inflating hallucination rates and distorting model assessment.”
Quote in the paperSubstantial claim- Rests on an assumption: The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
May combine two claims.
Findingown dataThe most consequential disagreements between Stage 1 and Stage 2 occur at Tier 3, where Stage 1 flagged clinically grounded diagnoses such as Peptic Ulcer Disease and GERD and failed to link the penicillin allergy in the Subjective section to the non-penicillin H. pylori regimen in the Plan.
“The most consequential disagreements occur at Tier 3: Stage 1 flagged clinically grounded diagnoses such as Peptic Ulcer Disease and GERD, and failed to link the penicillin allergy in the Subjective section to the non penicillin H. pylori regimen in the Plan a cross-section inference with direct patient safety implications.”
Quote in the paperSubstantial claimMay combine two claims.
Findingown dataStage 2 correctly reclassified the vast majority of Stage 1 false positives as supported inferences, with residual hallucinations comprising primarily fabricated past medical history and unsupported medication entries.
“Stage 2 correctly reclassified the vast majority of these as supported inferences, with residual hallucinations comprising primarily fabricated past medical history and unsupported medication entries.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
PremisereasoningClinical documentation is fundamentally an act of interpretation rather than transcription.
“However, clinical documentation is fundamentally an act of interpretation rather than transcription.”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataCurrent evaluation practices over-penalize valid clinical reasoning and may measure artifacts of evaluation design rather than true errors.
“These findings suggest that current evaluation practices over penalize valid clinical reasoning and may measure artifacts of evaluation design rather than true errors, underscoring the need for clinically informed evaluation in high context domains”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Premisenone givenA high-quality SOAP note normalizes colloquial expressions into medical terminology, structures findings according to clinical conventions, and infers diagnoses or care plans from incomplete information.
“A high quality SOAP note does not merely restate what was said, it normalizes colloquial expressions into medical terminology, structures findings according to clinical conventions, and infers diagnoses or care plans from incomplete or”
Quote in the paperNot a substantial claimMay combine two claims.
Interpretationown dataAn inference-aware LLM-as-judge evaluation framework grounded in clinically calibrated prompt design and retrieval-augmented medical knowledge assesses hallucination rates with greater clinical fidelity, yielding scores that align more closely with expert physician judgment.
“By developing and validating an inference aware LLM as Judge evaluation framework which is grounded in both clinically calibrated prompt design and retrieval augmented medical knowledge, we show that hallucination rates can be assessed with greater clinical fidelity, yielding scores that align more closely with expert physician judgment.”
Quote in the paperSubstantial claim- Rests on an assumption: Human evaluation is the gold standard for assessing clinical documentation.
Classified as probably not a substantial claim.
Study design and conductnone givenThe judge applies a five-step chain-of-thought protocol to every claim individually, with hallucination as the verdict of last resort.
“The judge applies the following protocol to every claim individually, with hallucination as the verdict of last resort: 1. State the claim. 2. Scan all four fact sections for a direct mention, synonym, translated equivalent, or paraphrase. 3. If found → supported.Stop. 4. If not found, ask: is this a reasonable inference, standard summary, or terminology upgrade?If yes → supported.Stop. 5. Other”
Quote in the paperSubstantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductnone givenStage 2 preserves a hard safety floor: a claim must be marked hallucinated if it states a specific medication, dose, frequency, or duration with no equivalent in the facts; a diagnosis neither stated nor inferable; a procedure or test not mentioned; or any detail that directly contradicts a stated fact.
“Stage 2 preserves a hard safety floor.A claim must be marked hallucinated if it states a specific medication, dose, frequency, or duration with no equivalent in the facts; a diagnosis neither stated nor inferable from the symptom picture; a procedure or test not mentioned; or any detail that directly contradicts a stated fact.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductreasoningEvery claim is verified against facts from all four SOAP sections, not only its own.
“Every claim must be verified against facts from all four SOAP sections, not only its own.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisereasoningTraditional metrics such as ROUGE and BLEU measure lexical overlap and are poorly suited for clinical tasks.
“Traditional metrics such as ROUGE and BLEU measure lexical overlap and are therefore poorly suited for clinical tasks.”
Quote in the paperSubstantial claimClassified as probably not a substantial claim.
Study design and conductown dataThe clinical knowledge base used in Stage 2 is constructed from SNOMED CT, ICD-10, and supplementary drug knowledge resources.
“This knowledge base is constructed from openly available medical sources, including SNOMED CT for clinical terminology and concept relationships, ICD-10 for diagnostic classifications, and supplementary drug knowledge resources covering generic and trade name equivalences.”
Quote in the paperSubstantial claim- Rests on an assumption: The clinical knowledge base (SNOMED CT, ICD-10, drug resources) is sufficient and accurate for grounding clinical inference.
Classified as probably not a substantial claim.
Study design and conductreasoningStage 2 uses a retrieval-augmented knowledge layer that retrieves relevant entries from a curated clinical knowledge base for each claim under evaluation.
“The second is a retrieval augmented knowledge layer: for each claim under evaluation, relevant entries are retrieved from a curated clinical knowledge base and provided as additional context to the judge.”
Quote in the paperSubstantial claim- Rests on an assumption: The clinical knowledge base (SNOMED CT, ICD-10, drug resources) is sufficient and accurate for grounding clinical inference.
Probably not a substantial claim: may not pass S4 (based).
Study design and conductreasoningThe retrieval step grounds the judge's clinical reasoning in structured, standardized medical knowledge, enabling more reliable synonym resolution, ontology-aware inference, and terminology normalization than prompt design alone.
“The retrieval step ensures that the judge's clinical reasoning is grounded not only in the model's parametric knowledge but in structured, standardized medical knowledge enabling more reliable synonym resolution, ontology aware inference, and terminology normalization than prompt design alone can achieve.”
Quote in the paperNot a substantial claim- Rests on an assumption: The clinical knowledge base (SNOMED CT, ICD-10, drug resources) is sufficient and accurate for grounding clinical inference.
- Rests on an assumption: The judge's parametric knowledge may be incomplete or inconsistent, necessitating retrieval.
Probably not a substantial claim: may not pass S4 (based).
Premisenone givenA judge prompt that lacks domain calibration will systematically over-flag legitimate medical inference as hallucination, producing artificially inflated error rates.
“A judge prompt that lacks domain calibration, for instance, one that treats all unanchored claims as hallucinations without regard for clinical reasoning will systematically over flag legitimate medical inference as hallucination, producing artificially inflated error rates.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
PremisereasoningThe apparent disagreement between human experts and LLM judges often reflects differences in evaluation criteria rather than true differences in output quality.
“The apparent disagreement between human experts and LLM judges often reflects differences in evaluation criteria rather than true differences in output quality.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
PremisereasoningThe effectiveness of a clinical judge depends on the breadth of domain knowledge available to it during evaluation, including terminology mappings, diagnostic ontologies, and pharmacological equivalences.
“Beyond prompt design, the effectiveness of a clinical judge also depends on the breadth of domain knowledge available to it during evaluation including terminology mappings, diagnostic ontologies, and pharmacological equivalences, knowledge that may exceed what is reliably encoded in a model's parametric weights alone.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
PremisereasoningThe validity of LLM-as-judge hinges critically on the quality of the evaluation prompt.
“However, the validity of LLM as Judge hinges critically on the quality of the evaluation prompt.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Study design and conductnone givenTrade names and generic drug names are treated as equivalent under all circumstances in Stage 2.
“Trade names and generic drug names are treated as equivalent under all circumstances.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
Study design and conductreasoningThe retained hallucination conditions correspond to Tier 4-5 fabrications and represent the true patient safety boundary the framework enforces.
“These conditions correspond to the Tier 4-5 fabrications in Table 1 and represent the true patient safety boundary the framework enforces.”
Quote in the paperSubstantial claim
The paper relies on this without support.
Human evaluation is the gold standard for assessing clinical documentation.
“Human evaluation remains the gold standard for assessing clinical documentation, capturing nuanced aspects of correctness, completeness, and safety.”
The paper relies on this without support.
The 100 anonymized physician-patient transcripts and the LLM-generated SOAP notes are representative of clinical documentation tasks.
“We constructed an evaluation dataset comprising 100 anonymized physician patient transcripts, covering a range of specialty including Cardiology, Dermatology, Endocrinology, ENT, Family Medicine etc.For each transcript, SOAP notes were generated using LLM.”
The paper relies on this without support.
The clinical knowledge base (SNOMED CT, ICD-10, drug resources) is sufficient and accurate for grounding clinical inference.
“This knowledge base is constructed from openly available medical sources, including SNOMED CT for clinical terminology and concept relationships, ICD-10 for diagnostic classifications, and supplementary drug knowledge resources covering generic and trade name equivalences.”
The paper relies on this without support.
The judge's parametric knowledge may be incomplete or inconsistent, necessitating retrieval.
“allowing the model to draw on structured clinical knowledge when its parametric knowledge may be incomplete or inconsistent.”
The paper relies on this without support.
Expected inter-annotator variability in clinical review is around 1.3 percentage points or more.
“Stage 2 closes this gap to within 1.3 percentage points, consistent with expected inter annotator variability in clinical review.”
Cites
Survey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1× · see citing works
“Subsequent work by [4] provided a comprehensive taxonomy of hallucination across tasks and model architectures, noting that hallucination tends to increase in tasks requiring long form generation or domain specific knowledge retrieval both hallmarks of clinical documentation.”
Large language models encode clinical knowledge
Singhal, K., Azizi, S., Tu, T., Mahdavi, S. S., Wei, J., Chung, H. W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S., et al. (2023) · Nature · doi:10.1038/s41586-023-06291-2 · Cited by 4,083 · cited 1× · see citing works
“Med PaLM [9] introduced, a medically fine tuned LLM evaluated against physician curated benchmarks.”
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2023) · arXiv · doi:10.48550/arxiv.2311.05232 · Cited by 2,516 · see citing works
Huang, L., Yu, W., Ma, W., Zhong, W., Feng, Z., Wang, H., Chen, Q., Peng, W., Feng, X., Qin, B., et al. (2025) · ACM Transactions on Information Systems · doi:10.1145/3703155 · Cited by 2,142 · see citing works
Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng, L., Chiang, W.-L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E. P., et al. (2023) · arXiv · doi:10.48550/arxiv.2306.05685 · Cited by 1,888 · cited 1× · see citing works
“The LLM as Judge paradigm, formalized by [12] in the context of general purpose model evaluation, proposes using a capable LLM (typically GPT 4) as an evaluator of other model outputs.”
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494 · see citing works
On Faithfulness and Factuality in Abstractive Summarization
Maynez, J., Narayan, S., Bohnet, B., & McDonald, R. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.173 · Cited by 1,185 · cited 1× · see citing works
“Hallucination in neural text generation was formally characterized by [3] in the context of abstractive summarization, where models were observed to generate content that was either intrinsic (contradicting the source) or extrinsic (introducing information not present in the source).”
Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts
Veen, D. V., Uden, C. V., Blankemeier, L., Delbrouck, J.-B., Aali, A., Bluethgen, C., Pareek, A., Polacin, M., Reis, E. P., Seehofnerova, A., et al. (2023) · doi:10.21203/rs.3.rs-3483777/v1 · Cited by 892 · cited 1× · see citing works
“Authors in [7] evaluated LLMs on clinical note generation tasks and found hallucination rates ranging from 15–35% depending on model and task complexity.”
Med-HALT: Medical Domain Hallucination Test for Large Language Models
Pal, A., Umapathi, L. K., & Sankarasubbu, M. (2023) · arXiv · doi:10.48550/arxiv.2307.15343 · Cited by 152 · cited 1× · see citing works
“Authors here [8] demonstrated that even state of the art models struggled to maintain factual fidelity in medical question answering, often generating fabricated drug dosages or clinical guidelines.”
Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures.
Joshi, A., Katariya, N., Amatriain, X., & Kannan, A. (2020) · Findings of the Association for Computational Linguistics: EMNLP (Findings of EMNLP) · doi:10.18653/v1/2020.findings-emnlp.335 · Cited by 81 · see citing works
Yim, W.-W., Fu, Y., Ben Abacha, A., Snider, N., Lin, T., & Yetisgen, M. (2023) · Scientific Data · doi:10.1038/s41597-023-02487-3 · Cited by 64 · cited 1× · see citing works
“Early work on visit note generation in [6] noted that neural models frequently introduced diagnoses not supported by by the source dialogue.”
Why Language Models Hallucinate
Kalai, A. T., Nachum, O., Vempala, S. S., & Zhang, E. (2025) · arXiv · doi:10.48550/arxiv.2509.04664 · Cited by 45 · cited 1× · see citing works
“As a result, evaluators default to conservative judgments under uncertainty, leading to inflated hallucination rates [2].”
Allocation of Physician Time in Ambulatory Practice
Sinsky, C. A., Tutty, M., & Colligan, L. (2017) · Annals of Internal Medicine · doi:10.7326/l17-0073 · Cited by 23 · see citing works
Gao, Y., Dligach, D., Miller, T. A., Xu, D., Churpek, M. M., & Afshar, M. (2022) · PubMed · doi:10.48550/arxiv.2208.08408 · Cited by 20 · cited 1× · see citing works
“Documentation also follows conventions, where normal findings are summarized using standard phrases even if not explicitly stated [11].”
Yim, W., Fu, Y., Abacha, A. B., et al. (2023) · arXiv (Cornell University) · doi:10.48550/arxiv.2306.02022 · Cited by 3 · see citing works
Adapted Large Language Models Can Outperform Medical Experts in Clinical Text Summarization
Veen, D. V., Uden, C. V., Blankemeier, L., et al. (2023) · arXiv · doi:10.48550/arxiv.2309.07430
Flagged first, then by citations
Research works printed but not linked: 2
- K. Krishna et al. Generation of patient after visit summaries to support physicians. In AMIA Annual Symposium Proceedings, 2021.
- A. B. Abacha et al. Overview of the mediqa sum task at acl clinicalnlp 2023. In ACL Workshop on Clinical NLP, 2023.
Cited by
Sorted by citations
Fox, S., Markham, L., Lail, R., et al. (2026) · arXiv (Cornell University) · doi:10.48550/arxiv.2608.31017
“…under control: relabelling clinically reasonable inference as supported moves the flagged rate on the same SOAP notes from 35.2% to 9.1% ( Vachhani et al., 2026), and a vendor preprint reports its error burden under two detectors of deliberately different sensitivity, 24.4% against 6.2%…”
Influential
- Authors
- Vachhani, B., Shrisvastava, K., Nema, P., & Chiranthan, S.
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2026
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 2604.14829
- OpenAlex
- W7154735258
- Semantic Scholar
- 86d91c4e447654369c540c00bf192c74d94bbc92
- Cited by
- 0
- References
- 15
- Type
- posted content
- Open access
- green
- arxiv.org
- Licence
- Locations
- arXiv (Cornell University) (submittedVersion, cc-by)
- arXiv (Cornell University) (cc-by)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Artificial Intelligence › Topic Modeling
- Life Sciences › Biochemistry, Genetics and Molecular Biology › Molecular Biology › Biomedical Text Mining and Ontologies
- Keywords
- hallucination evaluation, SOAP notes, clinical documentation, large language models, clinical reasoning, medical ontologies, evaluation bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
