Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 36 of the 36 references it cites were found in a registry or an index; 18 of the 24 sources its claims cite were checked for support.

The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians

Open access
Hurt, Ryan T., Stephenson, Christopher R., Gilman, Elizabeth A., Aakre, Christopher A., Croghan, Ivana T., Mundi, Manpreet S., Ghosh, Karthik, Edakkanambeth Varayil, Jithinraj · 2025 · Journal of Primary Care & Community Health

Abstract

Background: Artificial intelligence (AI) platforms can potentially enhance clinical decision-making (CDM) in primary care settings. OpenEvidence (OE), an AI tool, draws from trusted sources to generate evidence-based medicine (EBM) recommendations to address clinical questions. However, its effectiveness in real-world primary care cases remains unknown. Objective: To evaluate the performance of OE in providing EBM recommendations for five common chronic conditions in primary care: hypertension, hyperlipidemia, diabetes mellitus type 2, depression, and obesity. Methods: Five patient cases were retrospectively analyzed. Physicians posed specific clinical questions, and OE responses were evaluated on clarity, relevance, evidence support, impact on CDM, and overall satisfaction. Four independent physicians provided ratings using a 0 to 4 scale. Results: OE provided accurate, evidence-based recommendations in all cases, aligning with physician plans. OE was scored on a scale of zero to four, where zero was very unclear, and four was very clear. Mean scores across cases were clarity (3.55 ± 0.60), relevance (3.75 ± 0.44), support (3.35 ± 0.49), and satisfaction (3.60 ± 0.60). However, the impact on CDM was limited (1.95 ± 1.05), as OE primarily reinforced rather than modified plans. Conclusion: OE was rated high in clarity, relevance, and evidence-based support, reinforcing physician decisions in common chronic conditions. While the impact on CDM was minimal due to the study’s retrospective nature, OE shows promise in augmenting the primary care physician. Prospective trials are needed to evaluate its utility in complex cases and multidisciplinary settings.

Overview

CoChat Summary

TL;DR

In a retrospective five-case series, OpenEvidence provided accurate, evidence-based recommendations that aligned with physician plans for common chronic conditions in primary care. It was rated high in clarity, relevance, evidence support, and satisfaction, but had limited impact on clinical decision-making because it mainly reinforced rather than modified plans.

Key findings

  • Four independent physicians rated OpenEvidence across five cases on a 0–4 scale. In the Results/Table 1, mean scores were clarity 3.55 ± 0.21, relevance 3.75 ± 0.43, evidence support 3.35 ± 0.22, satisfaction 3.60 ± 0.38, and impact on CDM 1.95 ± 0.48. The Abstract reports the same means but different SDs: clarity 3.55 ± 0.60, relevance 3.75 ± 0.44, support 3.35 ± 0.49, satisfaction 3.60 ± 0.60, and impact 1.95 ± 1.05.
  • OpenEvidence recommendations aligned with physician plans in all five cases and primarily reinforced rather than modified decisions. Examples included statin initiation for hyperlipidemia, spironolactone for resistant hypertension, sertraline increase or mirtazapine augmentation for depression, SGLT2 inhibitor plus glipizide discontinuation for type 2 diabetes with HFrEF, and switching semaglutide to tirzepatide for obesity plateau.
  • The authors conclude OpenEvidence performed comparably to physician CDM for common chronic conditions and may help address primary care knowledge gaps, but its impact was limited in this retrospective design.

Methods

  • Retrospective case series of five patients with common chronic conditions—hypertension, hyperlipidemia, type 2 diabetes mellitus, depression, and obesity—identified from internal or family medicine clinics. Cases had initially been managed without AI using usual decision-making tools and physician knowledge.
  • For each case, the treating physician retrospectively posed a patient-centric point-of-care question without protected health information to OpenEvidence. Four independent physician authors who had not evaluated the patients reviewed the cases, questions, responses, and ratings using a five-question assessment tool covering clarity, relevance, evidence-based support, impact on CDM, and overall satisfaction on a 0–4 scale.
  • Main analysis was descriptive: mean ± SD scores across the five cases. No comparative or inferential statistics were reported.

Limitations

  • Stated by the paper: OpenEvidence was used retrospectively days to weeks after patient visits, and the literature access time differed from the time of CDM; the study included only five cases from a single institution using common conditions; accuracy may differ in more complex or undifferentiated cases; prospective trials, diverse presentations of the same diagnosis, and multidisciplinary team members are needed; an additional question rating reference quality was not included.
  • Stated by the paper: the small sample and retrospective design limit conclusions about real-world impact and clinical outcomes.
  • Evident from the text: there was no control or comparator AI platform or physician-only arm; ratings came from four physicians with no reported blinding or inter-rater reliability metrics; the Abstract and Results/Table 1 report inconsistent SDs for the mean scores; no clinical outcomes were measured.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1177/21501319251332215

Cites

36 of 36 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1. (a) Question submitted to the OpenEvidence platform for Case 1: hyperlipidemia and (b) OpenEvidence response for Case 1: hyperlipidemia.
Figure 1. (a) Question submitted to the OpenEvidence platform for Case 1: hyperlipidemia and (b) OpenEvidence response for Case 1: hyperlipidemia.CC BY-NC · PubMed Central
Figure 2. (a) Question submitted to the OpenEvidence platform for Case 2: hypertension and (b) OpenEvidence response for Case 2: hypertension.
Figure 2. (a) Question submitted to the OpenEvidence platform for Case 2: hypertension and (b) OpenEvidence response for Case 2: hypertension.CC BY-NC · PubMed Central
Figure 3. (a) Question submitted to the OpenEvidence platform for Case 3: depression and (b) OpenEvidence response for Case 3: depression.
Figure 3. (a) Question submitted to the OpenEvidence platform for Case 3: depression and (b) OpenEvidence response for Case 3: depression.CC BY-NC · PubMed Central
Figure 4. (a) Question submitted to the OpenEvidence platform for Case 4: diabetes and (b) OpenEvidence response for Case 4: diabetes.
Figure 4. (a) Question submitted to the OpenEvidence platform for Case 4: diabetes and (b) OpenEvidence response for Case 4: diabetes.CC BY-NC · PubMed Central
Figure 5. (a) Question submitted to the OpenEvidence platform for Case 5: obesity and (b) OpenEvidence response for Case 5: obesity.
Figure 5. (a) Question submitted to the OpenEvidence platform for Case 5: obesity and (b) OpenEvidence response for Case 5: obesity.CC BY-NC · PubMed Central

Verification

Claims the paper makes

14 Valid · 2 Not valid · 4 Not checked · 50 Own results and interpretations

39 more claims

Create a free account to see all 70 claims and to verify the 1 reference still open.

See all claims — free account
  • Probably not a substantial claim. 22
  • It may state two separate claims as one. 11

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

5 more claims about this paper

Create a free account to see every claim other papers make about this one.

See all claims — free account

Citations

Cites

Show all 33 references

Cited by

See all 78

Metadata

Authors
Hurt, R. T., Stephenson, C. R., Gilman, E. A., Aakre, C. A., Croghan, I. T., Mundi, M. S., Ghosh, K., & Edakkanambeth Varayil, J.
Affiliations
  • Mayo Clinic (US)
  • Mayo Clinic, Rochester, MN, USA
Venue
Journal of Primary Care & Community Health
Published
2025-04-16
Volume
16
Pages
21501319251332215-21501319251332215
ISSN
  • 2150-1319 (ISSN-L)
  • 2150-1327
OpenAlex
W4409493818
Cited by
78
References
33
Citations by year
2026: 59 · 2025: 19
FWCI
18.61
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Primary Care & Community Health (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
Keywords
OpenEvidence, artificial intelligence in clinical decision-making, primary care physicians, medical question answering, hypertension, hyperlipidemia, type 2 diabetes, depression, obesity, chronic disease, physician satisfaction, clinical decision support
MeSH terms
Clinical Decision-Making, Artificial Intelligence, Chronic Disease, Depression, Diabetes Mellitus, Type 2, Female, Humans, Hyperlipidemias, Hypertension, Male, Middle Aged, Obesity, Primary Health Care, Retrospective Studies, Evidence-Based Medicine, Physicians, Primary Care

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-30, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-30 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians. https://app.cochat.ai/papers/10.1177/21501319251332215
The paper
Hurt, R. T., Stephenson, C. R., Gilman, E. A., Aakre, C. A., Croghan, I. T., Mundi, M. S., Ghosh, K., & Edakkanambeth Varayil, J. (2025). The Use of an Artificial Intelligence Platform OpenEvidence to Augment Clinical Decision-Making for Primary Care Physicians. Journal of Primary Care & Community Health. https://doi.org/10.1177/21501319251332215

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.