Retrieve-Then-Verify for Evaluating Evidence Support and Hallucination in Large Language Model–Generated Medical Information: Empirical Study

Probably not a substantial claim: may not pass S3 (load-bearing).

Rose et al found moderate to substantial interrater agreement between ChatGPT-4o and human reviewers across Cochrane RoB review domains.

“For example, Lai et al [18,27] demonstrated that LLMs can perform RoB assessments at a level comparable to trained reviewers in structured appraisal tasks, and Rose et al [25,26] found moderate to substantial interrater agreement between ChatGPT-4o and human reviewers across Cochrane RoB review domains.”

• cites [25] sourceDisputed

  The Abstract (Results) describes agreement as an outcome the authors will report, not a finding. The Methods (Outcomes) explicitly says they will not assess agreement across Cochrane RoB domains, instead focusing on overall risk of bias. No page numbers are provided in our copy. (LLM, conf=0.90)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [26] sourcePartly supported

  Table 4 reports human–ChatGPT-4o agreement of 50.7% for overall RoB and 78.7% for RoB due to randomization. But its Cohen’s κ values are 0.215 (classified as fair by the paper’s criteria) and 0.537 (moderate), not moderate to substantial. The Discussion’s Strengths and Limitations section explicitly says the study did not assess other domains, so the claim overextends the finding across Cochrane RoB domains. (LLM, conf=0.91)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Recent studies have reported promising agreement between LLMs and human reviewers for RoB assessment.

“Recent studies have examined the ability of LLMs to assist with RoB assessment and have reported promising agreement with human reviewers [18,24-27].”

• cites [18] sourceSupported
• cites [24] sourceNot checked — no full text of the source
• cites [25] sourceDisputed

  The Abstract Conclusions (page unavailable) say the proposed study is likely to provide the first evidence of agreement between human and LLM risk-of-bias assessments. The Background (page unavailable) says the authors were unaware of research assessing agreement in overall risk-of-bias judgments. This protocol therefore does not report the promising agreement attributed to it. (LLM, conf=0.89)

  Checked by a reasoning model — may be wrong.

  Experimental — accuracy not yet measured
• cites [26] sourceSupported
• cites [27] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Recent studies have explored the use of LLMs to assist with evidence appraisal tasks, including RoB assessment, and have reported promising agreement with human judgments.

“Recent studies have explored the use of LLMs to assist with evidence appraisal tasks, including RoB assessment, and have reported promising agreement with human judgments [18].”

• cites [18] sourceSupported

May combine two claims.

Human RoB assessments in the source Cochrane review were performed independently by 2 reviewers using the RoB 2 tool, with discrepancies resolved through consensus discussion.

“In accordance with standard Cochrane methodology, RoB assessments in the source review were performed independently by 2 reviewers using the RoB 2 tool, with discrepancies resolved through consensus discussion; full reviewer details are reported in the source review [19].”

• cites [19] sourcePartly supported

  The Methods (Data collection and analysis) state that two review authors (EP and CC) independently assessed risk of bias using the Cochrane Risk of Bias 2 tool. They resolved disagreements by discussion or consultation with a third review author, so the claim's description of resolution solely through consensus discussion omits an alternative the source reports. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

LLMs have demonstrated strong performance across a wide range of biomedical knowledge and reasoning tasks.

“LLMs have demonstrated strong performance across a wide range of biomedical knowledge and reasoning tasks, accelerating their adoption in health care workflows [1-6].”

• cites [1] sourceSupported
• cites [2] sourcePartly supported

  The Results (Emerging Trends; page unavailable) describe strong results on medical question answering and several biomedical NLP tasks, supporting examples of performance across different tasks. The Discussion (Prompt Engineering Techniques Effectiveness in Medical Research; page unavailable) cautions that results are not systematically in favor of LLM-based methods, so the unqualified claim of strong performance across a wide range overstates the review. The cited passages also do not establish that performance accelerated adoption in health care workflows. (LLM, conf=0.86)

  Experimental — accuracy not yet measured
• cites [3] sourcePartly supported

  The Discussion reports strong performance on theoretical, clinical, and integrated questions in a bilingual nuclear medicine exam, but the Conclusion also reports 0% accuracy on medical legal regulation questions. This supports performance on several biomedical knowledge and reasoning tasks, not a wide range of tasks generally; the cited passages also do not establish accelerated adoption in health care workflows. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [4] sourcePartly supported

  The Abstract and Discussion report strong Med-PaLM 2 performance across medical knowledge benchmarks, long-form answers, and bedside consultation questions. The Discussion says validation in real-world workflows remains future work; it does not establish that this performance has accelerated adoption in health care workflows. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [5] sourceNot checked — no full text of the source
• cites [6] sourcePartly supported

  The Introduction (page unavailable) describes LLMs as powerful NLP tools across different tasks, and the Conclusion reports promising results for literature-review automation. The paper does not establish strong performance across a wide range of biomedical knowledge and reasoning tasks or show that such performance accelerated adoption in health care workflows. (LLM, conf=0.83)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Conceptual and sociotechnical analyses suggest that naive reliance on AI-generated medical information may amplify automation bias, erode professional expertise, and reinforce self-referential learning loops in digital health systems.

“Conceptual and sociotechnical analyses further suggest that naive reliance on AI-generated medical information may amplify automation bias, erode professional expertise, and reinforce self-referential learning loops in digital health systems [10].”

• cites [10] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

The revised RoB 2 tool introduced outcome-specific domains, signaling questions, and explicit decision rules to improve consistency.

“Although the revised RoB 2 tool introduced outcome-specific domains, signaling questions, and explicit decision rules to improve consistency [17]”

• cites [17] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Reviewers were systematic review specialists with expertise in global mental health interventions.

“Reviewers were systematic review specialists with expertise in global mental health interventions; their institutional affiliations and authorship details are listed in the source review [19].”

• cites [19] sourcePartly supported

  The Methods > Data collection and analysis section identifies review authors conducting systematic screening, risk-of-bias assessment, and GRADE assessment for this review of mental-health interventions in low- and middle-income countries. This supports describing them as reviewers working in that field. The available copy does not show their institutional affiliations or establish that all were systematic review specialists; its end may be missing. (LLM, conf=0.78)

  Experimental — accuracy not yet measured

May combine two claims.

RoB assessment has long been recognized as labor intensive, variably reproducible, and prone to misapplication in specific domains such as selective reporting.

“However, RoB assessment has long been recognized as labor intensive, variably reproducible, and prone to misapplication in specific domains such as selective reporting [14-16].”

• cites [14] sourceNot checked — no full text of the source
• cites [15] sourcePartly supported

  The Results (Table 2; page unavailable in our copy) show poor agreement between reviewer groups for selective reporting (κ = 0.13), and the Discussion says RoB assessments may not be reproducible across groups. The paper describes intensive reviewer training, but does not establish that RoB assessment itself is labor intensive. Poor agreement in the selective-reporting domain does not, on its own, establish misapplication. (LLM, conf=0.84)

  Experimental — accuracy not yet measured
• cites [16] sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The Cochrane risk-of-bias (RoB) assessment tool formalized a structured, domain-based approach to judgments anchored in trial methodology rather than undifferentiated quality scores.

“Within systematic reviews, the Cochrane risk-of-bias (RoB) assessment tool formalized a structured, domain-based approach to judgments anchored in trial methodology rather than undifferentiated quality scores [13].”

• cites [13] sourceSupported

May combine two claims.

Prior work has found hallucinations in several relevant domains, including fabricated or inaccurate references in systematic reviews, unsupported claims in clinical text summarization, and adversarial exploitation of false clinical details in decision support systems.

“Prior work has found hallucinations in several relevant domains, including fabricated or inaccurate references in systematic reviews [7,9], unsupported claims in clinical text summarization [8], and adversarial exploitation of false clinical details in decision support systems [11].”

• cites [7] sourcePartly supported

  The Results report inaccurate or hallucinated references generated by medical AI chatbots, supporting the reference-error portion of the claim. The Methods (Medical Prompts) say the chatbots were asked about medical topics and clinical scenarios, not to produce systematic reviews; this source does not establish the systematic-review setting. The other two domains in the citing sentence are attached to different citations. (LLM, conf=0.85)

  Experimental — accuracy not yet measured
• cites [9] sourceSupported
• cites [8] sourceSupported
• cites [11] sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

Lai et al demonstrated that LLMs can perform RoB assessments at a level comparable to trained reviewers in structured appraisal tasks.

“For example, Lai et al [18,27] demonstrated that LLMs can perform RoB assessments at a level comparable to trained reviewers in structured appraisal tasks, and Rose et al [25,26] found moderate to substantial interrater agreement between ChatGPT-4o and human reviewers across Cochrane RoB review domains.”

• cites [18] sourceSupported
• cites [27] sourceSupported

May combine two claims.

GPT-5 attained the highest exact accuracy in most RoB 2 assessment domains, including 0.64 in domain 1, 0.54-0.71 in domains 2 and 3, and 0.42-0.45 in domains 4 and 5.

“and attained the highest exact accuracy in most RoB 2 assessment domains, including 0.64 in domain 1, 0.54-0.71 in domains 2 and 3, and 0.42-0.45 in domains 4 and 5.”

May combine two claims.

GPT-3.5 showed modestly lower evidence support and higher hallucination rates across all 3 measures.

“GPT-3.5 showed modestly lower evidence support and higher hallucination rates across all 3 measures.”

May combine two claims.

Evidence support rates increased substantially and consistently with larger k across all 8 topics, while conservative and strict hallucination rates decreased correspondingly.

“As shown in Figure 4, evidence support rates increased substantially and consistently with larger k across all 8 topics, while conservative and strict hallucination rates decreased correspondingly.”

May combine two claims.

RtV required end-to-end runtimes of 50-158 seconds per study, depending on model choice, and the computational requirements were modest.

“It required end-to-end runtimes of 50-158 seconds per study, depending on model choice, and the computational requirements were modest.”

May combine two claims.

OpenAI o3-mini closely tracked GPT-5 on binary accuracy but consistently showed lower exact accuracy.

“OpenAI o3-mini closely tracked GPT-5 on binary accuracy but consistently showed lower exact accuracy”

May combine two claims.

AI-generated RoB judgments frequently agreed with human judgments while simultaneously lacking evidence grounding in the original trial reports.

“AI-generated RoB judgments frequently agreed with human judgments while simultaneously lacking evidence grounding in the original trial reports.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The included trials were published between 1996 and 2022.

“The included trials were published between 1996 and 2022.”

May combine two claims.

The reasoning-capable judge assigned higher evidence support rates across most topics and produced more decisive conservative hallucination classifications.

“The reasoning-capable judge assigned higher evidence support rates across most topics and produced more decisive conservative hallucination classifications.”

May combine two claims.

Lower reasoning depth in the generator model reduced both evidence support (65.5% to 55.9% pooled) and binary accuracy (89.5% to 77.7% pooled), without a comparable change in retrieval scores.

“Lower reasoning depth in the generator model reduced both evidence support (65.5% to 55.9% pooled) and binary accuracy (89.5% to 77.7% pooled), without a comparable change in retrieval scores.”

May combine two claims.

Across domains, quadratic κ values consistently exceeded linear κ values, indicating that most disagreements involved adjacent categories on the ordinal RoB 2 scale rather than large category gaps.

“Across domains, quadratic κ values consistently exceeded linear κ values, indicating that most disagreements involved adjacent categories on the ordinal RoB 2 scale rather than large category gaps.”

May combine two claims.

For the overall RoB conclusion, gaps exceeded 35% across models, indicating that many apparently correct judgments were not explicitly supported by the trial reports.

“For the overall RoB conclusion, gaps exceeded 35% across models, indicating that many apparently correct judgments were not explicitly supported by the trial reports.”

May combine two claims.

GPT-5 achieved perfect performance on the overall RoB conclusion, with accuracy, sensitivity, specificity, F1-score, and Youden J all of 1.00.

“It achieved perfect performance on the overall RoB conclusion, with accuracy, sensitivity, specificity, F1-score, and Youden J all of 1.00”

May combine two claims.

Internal consistency across models was strong in domain 1 (Fleiss κ=0.83) and domain 2 (Fleiss κ=0.86), and moderate in domains 3-5 (Fleiss κ of approximately 0.49-0.60) and for the overall RoB conclusion (Fleiss κ=0.51).

“Internal consistency across models was strong in domain 1 (Fleiss κ=0.83) and domain 2 (Fleiss κ=0.86), and moderate in domains 3-5 (Fleiss κ of approximately 0.49-0.60) and for the overall RoB conclusion (Fleiss κ=0.51).”

May combine two claims.

Mean support rate across RoB 2 domains was higher in domain 1 (approximately 0.78-0.80) and lower in domain 3 (approximately 0.56-0.59) across models.

“Radar plots of mean support rate across RoB 2 domains (Figure 3C) showed higher evidence support in domain 1 (approximately 0.78-0.80) and lower support in domain 3 (approximately 0.56-0.59) across models, potentially reflecting differences in reporting clarity and methodological complexity.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Assessments covered 7 outcome categories: diagnosis of mental disorders, quality of life, adverse events, psychological functioning and impairment, depressive symptoms, anxiety symptoms, and distress or posttraumatic stress disorder symptoms.

“Assessments covered 7 outcome categories: diagnosis of mental disorders, quality of life, adverse events, psychological functioning and impairment, depressive symptoms, anxiety symptoms, and distress or posttraumatic stress disorder symptoms.”

Classified as probably not a substantial claim. May combine two claims.

Hallucination reflects a general and high-stakes failure mode of AI-generated medical information in digital health systems, where plausible statements lack documentary support but may nonetheless influence downstream decisions.

“In this sense, hallucination reflects a general and high-stakes failure mode of AI-generated medical information in digital health systems, where plausible statements lack documentary support but may nonetheless influence downstream decisions.”

Probably not a substantial claim: may not pass S4 (based).

Most evaluations of hallucination in AI-generated health information rely on indirect measures such as agreement with human judgments, task-level accuracy, or expert preference ratings.

“Despite growing recognition of these risks, most evaluations of hallucination in AI-generated health information rely on indirect measures such as agreement with human judgments, task-level accuracy, or expert preference ratings.”

Probably not a substantial claim: may not pass S4 (based).

The proposed approach provides reproducible, source-linked quality metrics that are directly relevant to digital health applications.

“The proposed approach provides reproducible, source-linked quality metrics that are directly relevant to digital health applications.”

Classified as probably not a substantial claim. May combine two claims.

The proposed evaluation approach is model-agnostic and task-general.

“While RoB assessment provides a concrete and well-defined test case, the proposed evaluation approach is model-agnostic and task-general.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Generalizability to other clinical specialties, review types, or evidence appraisal frameworks has not been established.

“Generalizability to other clinical specialties, review types, or evidence appraisal frameworks has not been established and represents an important direction for future work.”

May combine two claims.

AI-generated medical information can achieve high decision-level accuracy while a substantial proportion of outputs remain unsupported by the source documents underlying the judgments.

“This study reveals that AI-generated medical information can achieve high decision-level accuracy while a substantial proportion of outputs remain unsupported by the source documents underlying the judgments.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

The domain-level pattern potentially reflects differences in reporting clarity and methodological complexity.

“Radar plots of mean support rate across RoB 2 domains (Figure 3C) showed higher evidence support in domain 1 (approximately 0.78-0.80) and lower support in domain 3 (approximately 0.56-0.59) across models, potentially reflecting differences in reporting clarity and methodological complexity.”

Classified as probably not a substantial claim.

The embedding-based semantic grounding diagnostic directly evaluates whether the model's stated rationale is semantically close to the retrieved trial text, including when the supporting evidence is paraphrased rather than lexically matched.

“This directly evaluates whether the model’s stated rationale is semantically close to the retrieved trial text, including when the supporting evidence is paraphrased rather than lexically matched.”

May combine two claims.

Agreement metrics summarize alignment with human judgments but do not assess whether model rationales are supported by the source documents.

“While these metrics summarize alignment with human judgments, they do not assess whether model rationales are supported by the source documents.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Prompt adequacy was verified through iterative review against the instrument's signaling questions and decision rules before full-scale application across all 97 studies.

“Prompt adequacy was verified through iterative review against the instrument’s signaling questions and decision rules before full-scale application across all 97 studies.”

Classified as probably not a substantial claim.

The RtV procedure operationalizes hallucination assessment by explicitly separating information retrieval from claim verification.

“The RtV procedure operationalizes hallucination assessment by explicitly separating information retrieval from claim verification.”

Classified as probably not a substantial claim. May combine two claims.

The constrained design minimized variability associated with free-text generation and ensured consistency across models and studies.

“This constrained design minimized variability associated with free-text generation and ensured consistency across models and studies.”

Classified as probably not a substantial claim.

This design enables a distinction between failures of evidence access and failures of model reasoning.

“This design enables a distinction between failures of evidence access and failures of model reasoning.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Prompt specifications were developed based on the structure and decision logic of the Cochrane RoB 2 instrument and were not validated on a separate held-out prompt development dataset.

“Prompt specifications were developed based on the structure and decision logic of the Cochrane RoB 2 instrument [20] and were not validated on a separate held-out prompt development dataset.”

• cites [20] RoB 2: revised Cochrane risk-of-bias tool for randomized trialsCochrane20252026-05-04https://methods.cochrane.org/bias/resources/rob-2-revised-cochrane-risk-bias-tool-randomized-trialsNot matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

The divergence among binary accuracy, exact accuracy, and evidence support highlights a key limitation of agreement-based evaluation and underscores the importance of assessing whether AI-generated judgments are grounded in source documents.

“The divergence among binary accuracy, exact accuracy, and evidence support highlights a key limitation of agreement-based evaluation and underscores the importance of assessing whether AI-generated judgments are grounded in source documents.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Verification relied on a single LLM judge, which may introduce systematic bias in borderline or ambiguous cases.

“Verification relied on a single LLM judge, which may introduce systematic bias in borderline or ambiguous cases; multijudge adjudication or ensemble verification approaches could reduce this risk, albeit at the cost of increased runtime.”

Classified as probably not a substantial claim. May combine two claims.

The structured output enabled consistent mapping to analytic variables, automated calculation of performance metrics, and traceability for adjudication.

“This structure enabled consistent mapping to analytic variables, automated calculation of performance metrics, and traceability for adjudication.”

Classified as probably not a substantial claim.

Each verification response included a verbatim quotation drawn from the retrieved trial text that grounds the judgment, making the evidentiary basis for each item-level decision directly traceable without requiring downstream re-derivation.

“Each response also included the verdict label, a verbatim quotation drawn from the retrieved trial text that grounds the judgment, and a 1-sentence rationale, making the evidentiary basis for each item-level decision directly traceable without requiring downstream re-derivation.”

• AssumptionRests on an assumption: The constrained judge LLM (GPT-5) verdicts accurately classify whether a generated claim is supported, contradicted, not found, or out of scope.

May combine two claims.

Agreement and evidence grounding capture different aspects of model reliability, and high agreement does not imply that generated claims are traceable to source documents.

“Our study addresses this gap by demonstrating that agreement and evidence grounding capture different aspects of model reliability, and that high agreement does not imply that generated claims are traceable to source documents.”

Classified as probably not a substantial claim. May combine two claims.

Higher semantic similarity indicates stronger semantic grounding of the explanation in the retrieved evidence, whereas low similarity across retrieved snippets indicates weak grounding and potential hallucination risk.

“Higher semantic similarity indicates stronger semantic grounding of the explanation in the retrieved evidence, whereas low similarity across retrieved snippets indicates weak grounding and potential hallucination risk.”

• AssumptionRests on an assumption: The top-k BM25 retrieved passages contain all evidence needed to verify each claim.

May combine two claims.

GPT-5 offered the most favorable balance between evidence support and hallucination control, with OpenAI o3-mini remaining closely comparable and GPT-3.5 exhibiting the highest tendency toward unsupported claims.

“In summary, GPT-5 offered the most favorable balance between evidence support and hallucination control, with OpenAI o3-mini remaining closely comparable and GPT-3.5 exhibiting the highest tendency toward unsupported claims.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Retrieval parameters were applied consistently across all 97 studies and all 3 LLMs.

“These parameters were applied consistently across all 97 studies and all 3 LLMs.”

Classified as probably not a substantial claim.

The resulting evidence support and hallucination metrics complement conventional accuracy and agreement measures by directly assessing whether AI-generated medical information is grounded in source documents.

“The resulting evidence support and hallucination metrics complement conventional accuracy and agreement measures by directly assessing whether AI-generated medical information is grounded in source documents.”

May combine two claims.

This pattern suggests high convergence in domains governed by explicit methodological criteria and greater divergence in domains requiring more nuanced interpretation.

“This pattern suggests high convergence in domains governed by explicit methodological criteria and greater divergence in domains requiring more nuanced interpretation.”

May combine two claims.

Prior studies mainly focus on decision-level agreement and do not assess whether model outputs are grounded in the trial reports used as source evidence.

“However, these studies mainly focus on decision-level agreement and do not assess whether model outputs are grounded in the trial reports used as source evidence.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The chunk size of 1400 characters with 200-character overlap was selected to balance sufficient local context per passage with retrieval precision, and the overlap helped ensure that claims spanning chunk boundaries were not missed.

“The chunk size was selected to balance sufficient local context per passage with retrieval precision; the 200-character overlap helped ensure that claims spanning chunk boundaries were not missed.”

May combine two claims.

Both retrieval depth and judge model reasoning capacity are important determinants of RtV evaluation outcomes.

“The sensitivity analyses further demonstrated that both retrieval depth and judge model reasoning capacity are important determinants of RtV evaluation outcomes, and that an appropriate top-k configuration combined with a reasoning-capable judge may provide a robust design choice for this framework.”

Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

Retrieval-based evidence verification provides a reproducible and transparent approach for evaluating the reliability of AI-generated medical information, with direct relevance to digital health practice, evidence-based medicine, and medical informatics.

“Retrieval-based evidence verification provides a reproducible and transparent approach for evaluating the reliability of AI-generated medical information, with direct relevance to digital health practice, evidence-based medicine, and medical informatics.”

• AssumptionRests on an assumption: The constrained judge LLM (GPT-5) verdicts accurately classify whether a generated claim is supported, contradicted, not found, or out of scope.
• AssumptionRests on an assumption: The top-k BM25 retrieved passages contain all evidence needed to verify each claim.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

For a subset of unsupported verdicts, low similarity may partly reflect genuine underretrieval rather than paraphrasing alone, and these explanations cannot be fully disentangled using lexical and embedding diagnostics alone.

“We acknowledge that, for a subset of unsupported verdicts, low similarity may partly reflect genuine underretrieval rather than paraphrasing alone, and these explanations cannot be fully disentangled using lexical and embedding diagnostics alone.”

May combine two claims.

Differences in hallucination rates were not primarily attributable to differences in retrieval strength and may instead reflect differences in reasoning and interpretation.

“The consistency of BM25 retrieval scores across models suggests that differences in hallucination rates were not primarily attributable to differences in retrieval strength and may instead reflect differences in reasoning and interpretation.”

May combine two claims.

LLMs often identify the correct direction of risk but frequently do so without sufficient evidentiary support.

“Together, these findings show that while LLMs often identify the correct direction of risk, they frequently do so without sufficient evidentiary support.”

Classified as probably not a substantial claim. May combine two claims.

This evaluation framework supports reproducible auditing, cross-model comparison, and systematic analysis of hallucination risk in AI-generated medical information for digital health systems.

“This evaluation framework supports reproducible auditing, cross-model comparison, and systematic analysis of hallucination risk in AI-generated medical information for digital health systems.”