The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; 18 of the 24 sources its claims cite were checked for support.
The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians
Open accessAbstract
Background: Artificial intelligence (AI) platforms can potentially enhance clinical decision-making (CDM) in primary care settings. OpenEvidence (OE), an AI tool, draws from trusted sources to generate evidence-based medicine (EBM) recommendations to address clinical questions. However, its effectiveness in real-world primary care cases remains unknown. Objective: To evaluate the performance of OE in providing EBM recommendations for five common chronic conditions in primary care: hypertension, hyperlipidemia, diabetes mellitus type 2, depression, and obesity. Methods: Five patient cases were retrospectively analyzed. Physicians posed specific clinical questions, and OE responses were evaluated on clarity, relevance, evidence support, impact on CDM, and overall satisfaction. Four independent physicians provided ratings using a 0 to 4 scale. Results: OE provided accurate, evidence-based recommendations in all cases, aligning with physician plans. OE was scored on a scale of zero to four, where zero was very unclear, and four was very clear. Mean scores across cases were clarity (3.55 ± 0.60), relevance (3.75 ± 0.44), support (3.35 ± 0.49), and satisfaction (3.60 ± 0.60). However, the impact on CDM was limited (1.95 ± 1.05), as OE primarily reinforced rather than modified plans. Conclusion: OE was rated high in clarity, relevance, and evidence-based support, reinforcing physician decisions in common chronic conditions. While the impact on CDM was minimal due to the study’s retrospective nature, OE shows promise in augmenting the primary care physician. Prospective trials are needed to evaluate its utility in complex cases and multidisciplinary settings.
Overview
CoChat Summary
TL;DR
In a retrospective five-case series, OpenEvidence provided accurate, evidence-based recommendations that aligned with physician plans for common chronic conditions in primary care. It was rated high in clarity, relevance, evidence support, and satisfaction, but had limited impact on clinical decision-making because it mainly reinforced rather than modified plans.
Key findings
- Four independent physicians rated OpenEvidence across five cases on a 0–4 scale. In the Results/Table 1, mean scores were clarity 3.55 ± 0.21, relevance 3.75 ± 0.43, evidence support 3.35 ± 0.22, satisfaction 3.60 ± 0.38, and impact on CDM 1.95 ± 0.48. The Abstract reports the same means but different SDs: clarity 3.55 ± 0.60, relevance 3.75 ± 0.44, support 3.35 ± 0.49, satisfaction 3.60 ± 0.60, and impact 1.95 ± 1.05.
- OpenEvidence recommendations aligned with physician plans in all five cases and primarily reinforced rather than modified decisions. Examples included statin initiation for hyperlipidemia, spironolactone for resistant hypertension, sertraline increase or mirtazapine augmentation for depression, SGLT2 inhibitor plus glipizide discontinuation for type 2 diabetes with HFrEF, and switching semaglutide to tirzepatide for obesity plateau.
- The authors conclude OpenEvidence performed comparably to physician CDM for common chronic conditions and may help address primary care knowledge gaps, but its impact was limited in this retrospective design.
Methods
- Retrospective case series of five patients with common chronic conditions—hypertension, hyperlipidemia, type 2 diabetes mellitus, depression, and obesity—identified from internal or family medicine clinics. Cases had initially been managed without AI using usual decision-making tools and physician knowledge.
- For each case, the treating physician retrospectively posed a patient-centric point-of-care question without protected health information to OpenEvidence. Four independent physician authors who had not evaluated the patients reviewed the cases, questions, responses, and ratings using a five-question assessment tool covering clarity, relevance, evidence-based support, impact on CDM, and overall satisfaction on a 0–4 scale.
- Main analysis was descriptive: mean ± SD scores across the five cases. No comparative or inferential statistics were reported.
Limitations
- Stated by the paper: OpenEvidence was used retrospectively days to weeks after patient visits, and the literature access time differed from the time of CDM; the study included only five cases from a single institution using common conditions; accuracy may differ in more complex or undifferentiated cases; prospective trials, diverse presentations of the same diagnosis, and multidisciplinary team members are needed; an additional question rating reference quality was not included.
- Stated by the paper: the small sample and retrospective design limit conclusions about real-world impact and clinical outcomes.
- Evident from the text: there was no control or comparator AI platform or physician-only arm; ratings came from four physicians with no reported blinding or inter-rater reliability metrics; the Abstract and Results/Table 1 report inconsistent SDs for the mean scores; no clinical outcomes were measured.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
14 Valid · 2 Not valid · 4 Not checked · 50 Own results and interpretations
Some cited sources support this claim and some don’t — read each source’s verdict below.
In the paper
“There are knowledge gaps in PCPs when compared to specialists in other common conditions, such as sexual dysfunction and HLD management.13,14”
§ The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians > Discussion
What the sources say
- Partly supportedcites [13] Stanley EE Pfoh E Lipold L Martinez K. Gap in sexual dysfunction management between male and female patients seen in primary care: an observational study. J Gen Intern Med. 2025; 40: 847- 853. doi: 10.1007/s11606-024-09004-1 39231849 PMC11914646 sourceExperimental — accuracy not yet measured
The Background says PCPs report feeling underprepared to manage female sexual dysfunction, and the Conclusion reports fewer management orders for those patients when diagnosed by PCPs than by OB/GYNs. The study does not establish that a knowledge gap causes this difference; it also finds that PCPs order erectile-dysfunction management more often than urologists. HLD management appears to be attributed to the other citation and is not assessed here. (LLM, conf=0.85)
- Disputedcites [14] Langer A Tan M Goodman SG, et al. Does management of lipid lowering differ between specialists and primary care: insights from GOAL Canada. Int J Clin Pract. 2021; 75(4): e13861. doi: 10.1111/ijcp.13861 33244861 sourceExperimental — accuracy not yet measured
For lipid-lowering management, the Discussion identifies knowledge or treatment gaps in both PCPs and specialists, rather than a greater knowledge gap in PCPs. The Abstract conclusion says the groups experienced similar care gaps, and recommended LDL-C attainment was similar (44% versus 42%). This paper does not address the sentence’s separate point about sexual dysfunction. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“A recent global survey of Metabolic Dysfunction Associated Steatotic Liver Disease (MASLD) was conducted to assess knowledge among PCPs, hepatologists (HEP), gastroenterologists (GI), and endocrinologists (ENDO). The knowledge scores were significantly higher in HEP, ENDO, and GI physicians in every domain (i.e., diagnosis, epidemiology, and treatment) when compared to PCPs. 12”
§ The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians > Discussion
What the sources say
- Partly supportedcites [12] Younossi ZM Ong JP Takahashi H, et al. A global survey of physicians knowledge about nonalcoholic fatty liver disease. Clin Gastroenterol Hepatol. 2022; 20(6):e1456-e1468. doi: 10.1016/j.cgh.2021.06.048 34229038 sourceExperimental — accuracy not yet measured
The Awareness and Knowledge section (page not specified) reports significantly higher scores for endocrinologists than PCPs in all three domains. It compares hepatologists with gastroenterologists on a separate survey version, not either group with PCPs; the Methods section confirms the two versions differed. The paper describes its subject as NAFLD, rather than MASLD. (LLM, conf=0.91)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“The role of AI in augmenting physician CDM in the management of chronic conditions for individual patients in the PCC is still largely unknown. 5”
§ The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians > Introduction
What the sources say
- Partly supportedcites [5] Liu J Wang C Liu S. Utility of ChatGPT in clinical practice. J Med Internet Res. 2023; 25:e48568. doi: 10.2196/48568 PMC10365580 37379067 sourceExperimental — accuracy not yet measured
The Introduction says studies evaluating ChatGPT in clinical practice are lacking, and Possible Risks and Challenges says its recommendations need clinical assessment. That supports uncertainty about its role in physician decision-making, but the paper does not establish the claim specifically for chronic-condition management of individual patients in primary care clinics. Its discussion also concerns ChatGPT rather than AI generally. (LLM, conf=0.84)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Despite these knowledge differences, to date, most AI CDM tools have been focused on specific organ or disease problems because of the narrower data elements (i.e., liver enzymes and imaging) needed for the LLMs compared to the undifferentiated nature of PCC patients. 2”
§ The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians > Discussion
What the sources say
- Partly supportedcites [2] Young RA Martin CM Sturmberg JP, et al. What complexity science predicts about the potential of artificial intelligence/machine learning to improve primary care. J Am Board Fam Med. 2024; 37(2): 332- 345. doi: 10.3122/jabfm.2023.230219R1 38740483 sourceExperimental — accuracy not yet measured
The Background section describes successful AI/ML applications for detecting specific diseases, while the Abstract and Discussion say tools are more likely to work on narrow, well-defined tasks than in complex primary care decision making (page numbers unavailable). The paper does not establish that most clinical decision-making tools target organ- or disease-specific problems, or that LLMs require narrower data elements such as liver enzymes and imaging. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Although the present study demonstrated that OE reinforced the CDM by the PCPs in commonly treated conditions, the sheer volume of medical knowledge makes staying up to date regarding management of chronic medical conditions challenging. 12”
§ The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians > Discussion
What the sources say
- Partly supportedcites [12] Younossi ZM Ong JP Takahashi H, et al. A global survey of physicians knowledge about nonalcoholic fatty liver disease. Clin Gastroenterol Hepatol. 2022; 20(6):e1456-e1468. doi: 10.1016/j.cgh.2021.06.048 34229038 sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) reports knowledge gaps about NAFLD, especially among primary care physicians, and recommends regular webinars on new updates. This supports a narrower point about keeping physicians informed on NAFLD, but the paper does not attribute the difficulty to the sheer volume of medical knowledge or address management of chronic conditions generally. (LLM, conf=0.78)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
39 more claims
Create a free account to see all 70 claims and to verify the 1 reference still open.
- Probably not a substantial claim. 22
- It may state two separate claims as one. 11
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“physicians participating in Hurt et al.’s (2025) study noted that seeing official citations increased their confidence in the AI’s suggestions (Hurt et al., 2025).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Hurt et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“for example, Hurt et al. (2025) found that providing clear citations in AI-generated answers made them highly credible to physicians (Hurt et al., 2025).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Hurt et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In practice, clinicians say they prefer recommendations that explicitly cite medical guidelines or high-quality studies (Hurt et al., 2025).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Hurt et al., 2025
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“OpenEvidence (OpenEvidence, Inc.) is one such platform; it is a clinician-facing LLM that grounds responses in cited, peer-reviewed medical sources, which may improve transparency and reduce hallucinations but does not eliminate them entirely (Hurt et al. 2025).”
From Evidence on Demand: OpenEvidence versus Established LLMs on the Orthopaedic In-Training Examination (2026) · cites this paper as Hurt et al. 2025
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“OpenEvidence is an AI system that answers doctors’ questions by drawing on “trusted sources” in evidence-based medicine (Hurt et al., 2025).”
From An auditable and source-verified framework for clinical AI decision support: integrating retrieval-augmented generation with data provenance (2026) · cites this paper as Hurt et al., 2025
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Whelton, P. K., Carey, R. M., Aronow, W. S., Casey, D. E., Collins, K. J., Dennison Himmelfarb, C., DePalma, S. M., Gidding, S., Jamerson, K. A., Jones, D. W., et al. (2018) · Hypertension · doi:10.1161/hyp.0000000000000065 · Cited by 7,391
Grundy, S. M., Stone, N. J., Bailey, A. L., Beam, C., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Circulation · doi:10.1161/cir.0000000000000625 · Cited by 6,185
Tirzepatide Once Weekly for the Treatment of Obesity
Jastreboff, A. M., Aronne, L. J., Ahmad, N. N., Wharton, S., Connery, L., Alves, B., Kiyosue, A., Zhang, S., Liu, B., Bunck, M. C., et al. (2022) · New England Journal of Medicine · doi:10.1056/nejmoa2206038 · Cited by 3,838
Grundy, S. M., Stone, N. J., Bailey, A. L., Beam, C., Birtcher, K. K., Blumenthal, R. S., Braun, L. T., de Ferranti, S., Faiella-Tommasino, J., Forman, D. E., et al. (2019) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2018.11.003 · Cited by 2,691
Davies, M. J., D’Alessio, D. A., Fradkin, J., Kernan, W. N., Mathieu, C., Mingrone, G., Rossing, P., Tsapas, A., Wexler, D. J., & Buse, J. B. (2018) · Diabetologia · doi:10.1007/s00125-018-4729-5 · Cited by 1,455
Williams, B., MacDonald, T. M., Morant, S., Webb, D. J., Sever, P., McInnes, G., Ford, I., Cruickshank, J. K., Caulfield, M. J., Salsbury, J., et al. (2015) · The Lancet · doi:10.1016/s0140-6736(15)00257-3 · Cited by 1,308
Carey, R. M., Calhoun, D. A., Bakris, G. L., Brook, R. D., Daugherty, S. L., Dennison-Himmelfarb, C. R., Egan, B. M., Flack, J. M., Gidding, S. S., Judd, E., et al. (2018) · Hypertension · doi:10.1161/hyp.0000000000000084 · Cited by 1,153
Whelton, P. K., Carey, R. M., Aronow, W. S., Casey, D. E., Collins, K. J., Dennison Himmelfarb, C., DePalma, S. M., Gidding, S., Jamerson, K. A., Jones, D. W., et al. (2018) · Circulation · doi:10.1161/cir.0000000000000596 · Cited by 1,127
Bupropion-SR, Sertraline, or Venlafaxine-XR after Failure of SSRIs for Depression
Rush, A. J., Trivedi, M. H., Wisniewski, S. R., Stewart, J. W., Nierenberg, A. A., Thase, M. E., Ritz, L., Biggs, M. M., Warden, D., Luther, J. F., et al. (2006) · New England Journal of Medicine · doi:10.1056/nejmoa052963 · Cited by 1,033
Dunlay, S. M., Givertz, M. M., Aguilar, D., Allen, L. A., Chan, M., Desai, A. S., Deswal, A., Dickson, V. V., Kosiborod, M. N., Lekavich, C. L., et al. (2019) · Circulation · doi:10.1161/cir.0000000000000691 · Cited by 639
Cited by
Akpınar, H. (2025) · BMC Oral Health · doi:10.1186/s12903-025-06294-7 · Cited by 30
“A study evaluating the use of MediSearch examined its broader implications for the healthcare system, highlighting its potential impact on clinical communication, health literacy, and data governance within digital health infrastructures [53–57].”
Kremer, P., Langballe, E., Haase, I., Bamberger, J., Kuhn, S., Krusche, M., & Knitza, J. (2026) · Rheumatology International · doi:10.1007/s00296-025-06068-y · Cited by 17
“In a small retrospective evaluation, physicians rated its answers highly in terms of clarity, clinical relevance, and evidence-based support [14].”
Alu, F. F., & Oluwadare, S. (2026) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2026.1737532 · Cited by 22
How Physicians Can Prepare for Generative AI
Morgan, D. J., Rodman, A. M., & Goodman, K. E. (2025) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2025.4914 · Cited by 16
Ngo, K., Mekhail, S., Chan, V., Li, X., Yin, A., Choi, H. Y., Allman-Farinelli, M. A., & Chen, J. (2025) · Nutrients · doi:10.3390/nu17223515 · Cited by 19
“…Accelerate Program, OpenEvidence, have been evaluated using primary care cases (hypertension, hyperlipidaemia, type 2 diabetes mellitus, depression, and obesity) to deliver pertinent outputs that support a physician’s clinical judgement in medical decision-making rather than replacing it [137].”
Metadata
- Authors
- Hurt, R. T., Stephenson, C. R., Gilman, E. A., Aakre, C. A., Croghan, I. T., Mundi, M. S., Ghosh, K., & Edakkanambeth Varayil, J.
- Affiliations
- Mayo Clinic (US)
- Mayo Clinic, Rochester, MN, USA
- ORCID
- Venue
- Journal of Primary Care & Community Health
- Publisher
- SAGE Publications
- Published
- 2025-04-16
- Volume
- 16
- Pages
- 21501319251332215-21501319251332215
- ISSN
- 2150-1319 (ISSN-L)
- 2150-1327
- PMC
- PMC12033599
- OpenAlex
- W4409493818
- Cited by
- 78
- References
- 33
- Citations by year
- 2026: 59 · 2025: 19
- FWCI
- 18.61
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- creativecommons.org/licenses/by-nc/4.0/ (unspecified)
- journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
- Locations
- Journal of Primary Care & Community Health (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- OpenEvidence, artificial intelligence in clinical decision-making, primary care physicians, medical question answering, hypertension, hyperlipidemia, type 2 diabetes, depression, obesity, chronic disease, physician satisfaction, clinical decision support
- MeSH terms
- Clinical Decision-Making, Artificial Intelligence, Chronic Disease, Depression, Diabetes Mellitus, Type 2, Female, Humans, Hyperlipidemias, Hypertension, Male, Middle Aged, Obesity, Primary Health Care, Retrospective Studies, Evidence-Based Medicine, Physicians, Primary Care
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians. https://app.cochat.ai/papers/10.1177/21501319251332215
- The paper
- Hurt, R. T., Stephenson, C. R., Gilman, E. A., Aakre, C. A., Croghan, I. T., Mundi, M. S., Ghosh, K., & Edakkanambeth Varayil, J. (2025). The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians. Journal of Primary Care & Community Health. https://doi.org/10.1177/21501319251332215
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.