Answering real-world clinical questions using large language model, retrieval-augmented generation, and agentic systems

OpenEvidence produced actionable results for 48% of questions with existing evidence.

“OpenEvidence produced actionable results for 48% of questions with existing evidence, compared to 37% for ChatRWD and <5% for the general-purpose LLMs.”

Other LLMs provided actionable results for <10% of questions that lacked existing literature.

“ChatRWD provided actionable results for 52% of questions that lacked existing literature compared to <10% for other LLMs.”

ChatRWD provided actionable results for 52% of questions that lacked existing literature.

“ChatRWD provided actionable results for 52% of questions that lacked existing literature compared to <10% for other LLMs.”

OpenEvidence produced relevant, evidence-based answers for 24% of questions.

“RAG-based and agentic LLM systems, respectively, produced relevant, evidence-based answers for 24% (OpenEvidence) to 58% (ChatRWD) of questions.”

• Also stated in: § Relevance, evidence, and actionability · plain

  “The physician reviewers found that OpenEvidence and ChatRWD produced relevant, evidence-based answers for 24% and 58% of the questions, respectively.”

General-purpose LLMs rarely produced relevant, evidence-based answers, meeting this standard for 2-10% of questions.

“General-purpose LLMs rarely produced relevant, evidence-based answers (2–10% of questions).”

• Also stated in: § Relevance, evidence, and actionability · plain

  “For comparison, the physician reviewers rarely judged answers from the general-purpose LLMs to be relevant and evidence-based, meeting this standard for 2–10% of the responses.”

ChatRWD and OpenEvidence were not significantly different from each other in actionability (p = 0.21).

“This pattern of differences was also statistically significant (p < 0.0001 by chi-square test); ChatRWD and OpenEvidence each significantly outperformed the general-purpose LLMs in pairwise comparisons (p < 0.009 for all) but were not significantly different from each other (p = 0.21) (Supplemental Table 5).”

ChatGPT, Gemini, or Claude were never the most common choice for best answer.

“ChatGPT, Gemini, or Claude were never the most common choice for best answer.”

ChatRWD produced relevant, evidence-based answers for 58% of questions.

“RAG-based and agentic LLM systems, respectively, produced relevant, evidence-based answers for 24% (OpenEvidence) to 58% (ChatRWD) of questions.”

• Also stated in: § Relevance, evidence, and actionability · plain

  “The physician reviewers found that OpenEvidence and ChatRWD produced relevant, evidence-based answers for 24% and 58% of the questions, respectively.”

The pattern of differences in actionability between models was statistically significant (p < 0.0001 by chi-square test).

“This pattern of differences was also statistically significant (p < 0.0001 by chi-square test); ChatRWD and OpenEvidence each significantly outperformed the general-purpose LLMs in pairwise comparisons (p < 0.009 for all) but were not significantly different from each other (p = 0.21) (Supplemental Table 5).”

ChatRWD produced actionable results for 37% of questions with existing evidence.

“OpenEvidence produced actionable results for 48% of questions with existing evidence, compared to 37% for ChatRWD and <5% for the general-purpose LLMs.”

ChatRWD's rate of relevant, evidence-based answers was significantly higher than OpenEvidence (p = 0.007).

“In pairwise comparisons, ChatRWD's rate of relevant, evidence-based answers was significantly higher than the general-purpose LLMs (p < 0.0001 by Fisher exact test for all comparisons) and OpenEvidence (p = 0.007); OpenEvidence significantly outperformed Gemini (p = 0.011) but not the other models (Supplemental Table 5).”

The pattern of differences in relevant, evidence-based answers between models was statistically significant (p < 0.0001 by chi-square test).

“This pattern of differences was statistically significant (p < 0.0001 by chi-square test).”

General-purpose LLMs produced actionable results for <5% of questions with existing evidence.

“OpenEvidence produced actionable results for 48% of questions with existing evidence, compared to 37% for ChatRWD and <5% for the general-purpose LLMs.”

ChatRWD's rate of relevant, evidence-based answers was significantly higher than the general-purpose LLMs (p < 0.0001 by Fisher exact test for all comparisons).

“In pairwise comparisons, ChatRWD's rate of relevant, evidence-based answers was significantly higher than the general-purpose LLMs (p < 0.0001 by Fisher exact test for all comparisons) and OpenEvidence (p = 0.007); OpenEvidence significantly outperformed Gemini (p = 0.011) but not the other models (Supplemental Table 5).”

General-purpose LLMs produced answers for 58-78% of questions.

“The RAG and agentic systems produced answers for 86% (OpenEvidence) and 94% (ChatRWD) of the questions (Table 1, Supplemental Figure 2), compared to 58–78% of questions in the three general-purpose LLMs (Supplemental Table 3) used for benchmarking.”

For ChatRWD, the most common problem was misspecification of the study design (44.7%).

“For ChatRWD, the most common problem was misspecification of the study design (44.7%, Table 3).”

The frequency of actionable answers increased from 30% for OpenEvidence alone and 44% for ChatRWD alone to 60% for the two combined.

“ChatRWD and OpenEvidence were complementary: the frequency of actionable answers increased from 30% for OpenEvidence alone and 44% for ChatRWD alone to 60% for the two combined (Table 4).”

Among novel questions, ChatRWD produced relevant and evidence-based answers 65.2% of the time.

“Among the novel questions, ChatRWD could produce answers that were actionable (52.2%) as well as answers that were relevant and evidence-based (65.2%).”

Systematic searching of PubMed and LLM-generated URLs identified citation errors in 40-80% of relevant answers.

“Systematic searching of PubMed and LLM-generated URLs identified citation errors in 40–80% of relevant answers (Table 3).”

The typical "null" result from the general-purpose LLMs was along the lines of "I'm sorry, but I couldn't find any study comparing [outcome] between patients with [condition] who were treated with [treatment] or [comparator]".

“In contrast, the typical “null” result from the general-purpose LLMs was along with the lines of, “I’m sorry, but I couldn’t find any study comparing [outcome] between patients with [condition] who were treated with [treatment] or [comparator]” (Supplemental Table 3).”

When answers by LLMs were deemed less relevant, it was often due to minor study misspecification such as the study population not being fully relevant to the question.

“When the answers by LLMs were deemed less relevant, it was often due to minor study misspecification such as the study population (e.g., ulcerative colitis) not being fully relevant to the question (e.g., Crohn's disease).”

Qualitatively, the reviewers found little value in the responses from the general-purpose LLMs.

“Qualitatively, the reviewers found little value in the responses from the general-purpose LLMs (Supplemental Table 7).”

When faced with novel questions, OpenEvidence produced relevant and evidence-based answers 8.7% of the time.

“When faced with novel questions, OpenEvidence was rarely able to produce actionable answers (8.7%) or answers that were relevant and evidence-based (8.7%).”

Almost half of the selected questions were novel at the time of this study.

“Given that almost half of our selected questions were novel at the time of this study, there is a need to rapidly conduct new studies on demand.”

Among novel questions, ChatRWD produced actionable answers 52.2% of the time.

“Among the novel questions, ChatRWD could produce answers that were actionable (52.2%) as well as answers that were relevant and evidence-based (65.2%).”

The general-purpose LLMs' most common failure was the inclusion of hallucinated or irrelevant citations.

“The general-purpose LLMs’ most common failure was the inclusion of hallucinated or irrelevant citations.”

When faced with novel questions, OpenEvidence produced actionable answers 8.7% of the time.

“When faced with novel questions, OpenEvidence was rarely able to produce actionable answers (8.7%) or answers that were relevant and evidence-based (8.7%).”

Within each LLM system, the inter-rater reliability for response category ranged from fair (ChatRWD: 0.30; OpenEvidence: 0.38) to good (Gemini: 0.63).

“Within each LLM system, the inter-rater reliability for response category ranged from fair (ChatRWD: 0.30; OpenEvidence: 0.38) to good (Gemini: 0.63).”

Special-purpose LLM systems, when augmented with specialized knowledge (24%, OpenEvidence) or a causal inference engine (58%, ChatRWD), far outperformed off-the-shelf LLMs (2-10%) in producing relevant and evidence-based answers for the clinical questions examined.

“We demonstrated that special-purpose LLM systems, when augmented with specialized knowledge (24%, OpenEvidence) or a causal inference engine (58%, ChatRWD), far outperformed off-the-shelf LLMs (2–10%) in producing relevant and evidence-based answers for the clinical questions examined.”

Probable authors, journals, and article titles were often composed together into nonexistent citations, making up nearly 40% of citations reported.

“We observed that probable authors, journals, and article titles were often composed together into nonexistent citations, making up nearly 40% of citations reported.”

Retrieval-augmented generation-based LLM (OpenEvidence) performed well when existing data were available.

“Retrieval-augmented generation-based LLM (OpenEvidence) performed well when existing data were available, while only the agentic ChatRWD was able to provide actionable answers when preexisting studies were lacking.”

Special-purpose LLM systems greatly outperformed general-purpose LLMs in producing answers to clinical questions.

“Special-purpose LLM systems greatly outperformed general-purpose LLMs in producing answers to clinical questions.”

Even when studies exist, they can have conflicting findings arising from heterogeneous patient populations, study designs of variable quality, or nonstandard endpoints.

“Second, even when studies exist, they can have conflicting findings arising from heterogeneous patient populations, study designs of variable quality, or nonstandard endpoints.”

• cites [9] Zipkin DA Greenblatt L Kushinka JT. Evidence-based medicine and primary care: keeping up is hard to do. Mt Sinai J Med 2012; 79: 545– 554. 22976360 sourceNot checked yet

  Not checked in the free run.

• cites [10] Davies K Harrison J. The information-seeking behaviour of doctors: a review of the evidence. Health Inf Libr J 2007; 24: 78– 94. 17584211 sourceNot checked yet

  Not checked in the free run.

Clinical trials often lack generalizability to complex patients who often fail to qualify for trials.

“First, clinical trials often lack generalizability 4 to complex patients who often fail to qualify for trials.5,6”

• cites [4] He J Morales DR Guthrie B. Exclusion rates in randomized controlled trials of treatments for physical conditions: a systematic review. Trials 2020; 21: 228. 32102686 PMC7045589 sourceNot checked yet

  Not checked in the free run.

• cites [5] Fanaroff AC Califf RM Windecker S, et al. Levels of evidence supporting American College of Cardiology/American Heart Association and European Society of Cardiology Guidelines, 2008–2018. JAMA 2019; 321: 1069– 1080. 30874755 PMC6439920 sourceNot checked yet

  Not checked in the free run.

• cites [6] Stewart WF Shah NR Selna MJ, et al. Bridging the inferential gap: the electronic health record and clinical evidence. Health Aff (Millwood) 2007; 26: w181– w191. PMC2670472 17259202 sourceNot checked yet

  Not checked in the free run.

Only the agentic ChatRWD was able to provide actionable answers when preexisting studies were lacking.

“Retrieval-augmented generation-based LLM (OpenEvidence) performed well when existing data were available, while only the agentic ChatRWD was able to provide actionable answers when preexisting studies were lacking.”

In some specialties less than 20% of daily medical decisions are supported by quality evidence.

“However, in some specialties less than 20% of daily medical decisions are supported by quality evidence.2,3”

• cites [2] Darst JR Newburger JW Resch S, et al. Deciding without data. Congenit Heart Dis 2010; 5: 339– 342. 20653700 PMC4283550 sourceNot checked yet

  Not checked in the free run.

• cites [3] Ishman SL Tang A Cohen AP, et al. Decision making for children with obstructive sleep apnea without tonsillar hypertrophy. Otolaryngol Head Neck Surg 2016; 154: 527– 531. 26645533 sourceNot checked yet

  Not checked in the free run.

It was not possible to blind the reviewers to the models because each model has a distinctive output format.

“Second, it was not possible to blind the reviewers to the models because each model has a distinctive output format.”

OpenEvidence can provide relevant, evidence-based responses to questions where literature already exists, often making use of high-quality sources such as randomized controlled trials and meta-analyses.

“OpenEvidence can provide relevant, evidence-based responses to questions where literature already exists, often making use of high-quality sources such as randomized controlled trials and meta-analyses.”

Large language models struggle with appraising sources for relevance, quality, and trustworthiness.

“Large language models struggle with appraising sources for relevance, quality, and trustworthiness, a critical task for RWE.”

ChatRWD, specifically designed to conduct comparative studies on demand, outperformed the general LLMs and the RAG-based OpenEvidence for novel questions.

“Thus, it is unsurprising that ChatRWD, specifically designed to conduct comparative studies on demand, outperformed the general LLMs and the RAG-based OpenEvidence for novel questions.”

• AssumptionRests on an assumption: General-purpose LLMs are not designed for the complex tasks required for RWE generation (study design classification, PICO extraction, and clinical named-entity recognition).
• AssumptionRests on an assumption: An LLM cannot provide responses to novel medical questions whose answer is in content created after the LLM's most recent training completion date.

Nearly 80% of care decisions lack high-quality evidence due to no specific study being available.

“Challenges in obtaining evidence stem from two sources: (1) nearly 80% of care decisions lack high-quality evidence due to no specific study being available 2 and (2) difficulty in contextualizing available studies for the specific intricacies of the patient at hand.”

• cites [2] Darst JR Newburger JW Resch S, et al. Deciding without data. Congenit Heart Dis 2010; 5: 339– 342. 20653700 PMC4283550 sourceNot checked yet

  Not checked in the free run.

OpenEvidence and ChatRWD are complementary.

“This makes OpenEvidence and ChatRWD complementary.”

The study could not control for possible reviewer preference for a given model.

“This means we could not control for possible reviewer preference for a given model.”

Even LLMs that perform well on medical benchmarks such as the United States Medical Licensing Examination may not generalize across all medical tasks.

“Even LLMs that perform well on medical benchmarks such as the United States Medical Licensing Examination 32 may not generalize across all medical tasks. 33”

• cites [32] Singhal K Azizi S Tu T, et al. Large language models encode clinical knowledge. Nature 2023; 620: 172–180. PMC10396962 37438534 sourceNot checked yet

  Not checked in the free run.

• cites [33] Feng H Ronzano F LaFleur J, et al. Evaluation of large language model performance on the biomedical language understanding and reasoning benchmark. medRxiv 2024 May 17. sourceNot checked yet

  Not checked in the free run.

ChatRWD can generate new evidence for questions that have not previously been studied in the published literature.

“On the other hand, ChatRWD can generate new evidence for questions that have not previously been studied in the published literature.”

ChatRWD's LLM-driven user interface translates clinical queries into a structured PICO study design for processing through a causal inference engine that can generate on-demand RWE.

“Its LLM-driven user interface translates clinical queries into a structured population–intervention–control–outcome (PICO) 23 study design for processing through a causal inference engine that can generate on-demand RWE 8 in response to the question.”

• cites [23] Miller SA Forrest JL. Enhancing your practice through evidence-based decision making: PICO, learning how to ask good questions. J Evid Based Dental Pract 2001; 1: 136– 141. sourceNot checked yet

  Not checked in the free run.

• cites [8] Callahan A Gombar S Cahan EM, et al. Using aggregate patient data at the bedside via an on-demand consultation service. NEJM Catal 2021; 2. DOI: 10.1056/CAT.21.0224. sourceNot checked yet

  Not checked in the free run.

A different group of reviewers might change the specific percentages.

“We also note that we observed only fair inter-rater reliability across the core measures; this indicates that a different group of reviewers might change the specific percentages.”

Using RAG on existing medical literature reduces the likelihood of hallucinating information because OpenEvidence can summarize the relevant literature retrieved and present the conclusions to the requester.

“Using RAG on existing medical literature in this manner reduces the likelihood of hallucinating information because OpenEvidence can summarize the relevant literature retrieved and present the conclusions to the requester.”

ChatRWD and OpenEvidence were complementary.

“ChatRWD and OpenEvidence were complementary: the frequency of actionable answers increased from 30% for OpenEvidence alone and 44% for ChatRWD alone to 60% for the two combined (Table 4).”

The physicians were not blinded to the system that generated the answer.

“Because the various LLM systems all have easily identifiable response structures, the physicians were not blinded to the system that generated the answer.”

• AssumptionRests on an assumption: The unblinded review did not bias the physicians' ratings of the systems.

The study restricted clinical questions to those potentially answerable using RWD: treatment comparisons assessable with a cohort study design and using phenotypes already in the phenotype library.

“Foremost is that we restricted our clinical questions to those potentially answerable using RWD: specifically, treatment comparisons that could be assessed with a cohort study design and that used phenotypes already in our phenotype library.”

Relying only on past studies is inadequate.

“However, because there is a considerable time lag between asking a clinical question and publishing a comparative study to answer the question, relying only on past studies is inadequate.”

• AssumptionRests on an assumption: There is a considerable time lag between asking a clinical question and publishing a comparative study to answer the question.

LLMs excel at summarizing and contextualizing existing literature, either internalized during training or retrieved from external RAG sources.

“While LLMs excel at summarizing and contextualizing existing literature, either internalized during training or retrieved from external RAG sources, they cannot perform new real-world studies for the exact question at hand unless integrated with a causal inference engine to do so.”

A combination of purpose-built literature retrieval and an agentic system to perform on-demand studies can do a fair job of surfacing relevant, evidence-based, and actionable responses.

“However, a combination of purpose-built literature retrieval and an agentic system to perform on-demand studies can do a fair job of surfacing relevant, evidence-based, and actionable responses.”

LLMs cannot perform new real-world studies for the exact question at hand unless integrated with a causal inference engine.

“While LLMs excel at summarizing and contextualizing existing literature, either internalized during training or retrieved from external RAG sources, they cannot perform new real-world studies for the exact question at hand unless integrated with a causal inference engine to do so.”

General-purpose LLMs are not fit for the task of providing evidence for clinical decisions.

“By evaluating the response of five LLMs and having independent reviewers evaluate them for relevance, evidence, and actionability, we demonstrated general-purpose LLMs are not fit for the task of providing evidence for clinical decisions.”

• AssumptionRests on an assumption: General-purpose LLMs are not designed for the complex tasks required for RWE generation (study design classification, PICO extraction, and clinical named-entity recognition).