CoChat
Sign Up
Cited by 33Artificial Intelligence in Healthcare and EducationECG Monitoring and AnalysisElectronic Health Records Systems

Clinician Adoption of an Artificial Intelligence Algorithm to Detect Left Ventricular Systolic Dysfunction in Primary Care.

Open access
Rushlow, David R., Croghan, Ivana T., Inselman, Jonathan W., Thacher, Tom D., Friedman, Paul A., Yao, Xiaoxi, Pellikka, Patricia A., Lopez-Jimenez, Francisco +8 more·2022·Mayo Clinic Proceedings
DOI 10.1016/j.mayocp.2022.04.008
Open this paper in CoChat — free accountOpen PDF

Abstract

OBJECTIVE: To compare the clinicians' characteristics of "high adopters" and "low adopters" of an artificial intelligence (AI)-enabled electrocardiogram (ECG) algorithm that alerted for possible low left ventricular ejection fraction (EF) and the subsequent effectiveness of detecting patients with low EF. METHODS: Clinicians in 48 practice sites of a US Midwest health system were cluster-randomized by the care team to usual care or to receive a notification that suggested ordering an echocardiogram in patients flagged as potentially having low EF based on an AI-ECG algorithm. Enrollment was between June 26, 2019, and July 30, 2019; participation concluded on March 31, 2020. This report is focused on those clinicians randomized to receive the notification of the AI-ECG algorithm. At the patient level, data were analyzed for the proportion of patients with positive AI-ECG results. Adoption was defined as the clinician order of an echocardiogram after prompted by the alert. RESULTS: A total of 165 clinicians and 11,573 patients were included in this analysis. Among patients with positive AI-ECG, high adopters (n=41) were twice as likely to diagnose patients with low EF (33.9%) vs low adopters, n=124, (16.9%); odds ratio, 1.62; 95% CI, 1.21 to 2.17). High adopters were more often advanced practice providers (eg, nurse practitioners and physician assistants) vs physicians, Family Medicine vs Internal Medicine specialty, and tended to have less complex patients. CONCLUSION: Clinicians who most frequently followed the recommendations of an AI tool were twice as likely to diagnose low EF. Those clinicians with less complex patients were more likely to be high adopters. TRIAL REGISTRATION: Clinicaltrials.gov Identifier: NCT04000087.

AI snapshot

Written from the paper’s full text.

TL;DR

In a secondary analysis of the intervention arm of the EAGLE trial, primary care clinicians who most often acted on an AI-ECG alert for possible low ejection fraction were twice as likely to diagnose low EF as low adopters. High adoption was associated with being an advanced practice provider, family medicine specialty, and lower patient complexity in unadjusted analyses, but these associations did not remain significant after multivariable adjustment.

Key findings

  • Among patients with an AI-positive ECG, high adopters had a diagnostic yield for low EF of 33.9% vs 16.3% for low adopters (P<.001); the adjusted odds ratio was 1.62 (95% CI, 1.21-2.17).
  • High adopters ordered echocardiograms after 78.7% (100/127) of AI-positive ECGs vs 43.0% (243/565) for low adopters; among those echocardiograms, low EF was found in 43.0% (43/100) vs 37.9% (92/243), respectively.
  • In unadjusted analyses, high adopters were more often NP/PAs than physicians (37.7% vs 18.8%; P=.008), more often family medicine than internal medicine (28.0% vs 15.6%; P=.010), and had lower average patient ACG complexity scores (P=.002); these variables were no longer significantly associated with adoption in multivariable logistic regression.
  • For AI-negative ECGs, there was no difference between high and low adopters in echocardiogram ordering or low EF detection (P=.46).

Methods

  • Secondary analysis of the intervention arm of a pragmatic, cluster randomized controlled trial (EAGLE) across 48 Mayo Clinic primary care practices in Minnesota and Wisconsin; enrollment was June 26, 2019, to July 30, 2019, with participation concluding March 31, 2020.
  • Included 165 primary care clinicians from 60 care teams and 11,573 adult patients with ECGs; patients were excluded for known EF ≤50%, prior heart failure, or lack of research authorization. Adoption was defined as clinician ordering an echocardiogram within 3 months after an AI-positive ECG; high adopters were the top quartile (ordering for at least 64.3% of AI-positive ECGs), and low adopters were the lower three quartiles.
  • Analyses used unadjusted t tests and chi-square tests, multivariable logistic regression for high-adopter characteristics, and mixed-effect logistic regression comparing low EF incidence with care team as a random effect; sensitivity analyses included negative binomial regression and a propensity-matched cohort.

Limitations

  • The paper states that findings may not generalize beyond 165 primary care clinicians in an integrated, multispecialty Midwest health system, and other organizations may differ in patient population, cardiac disease prevalence, race/ethnicity, socioeconomic status, rural/urban distribution, APP-to-physician ratios, scope of practice, care team structure, and specialty availability.
  • The paper states that not ordering an echocardiogram may sometimes be appropriate (eg, patient preference, cost, or other clinical justification), and the rationale behind clinicians’ decisions not to order after a positive ECG is not fully understood; adoption was also defined by echocardiogram order and completion rate rather than a validated adoption scale.
  • Evident from the text: this is a secondary analysis of the intervention arm only, high adopters numbered only 41, and the unadjusted clinician-characteristic associations lost significance in adjusted analysis; the text also reports inconsistent low-adopter diagnostic yields across sections (16.9% in the abstract, 16.3% in results/table, and 16.2% in the discussion).

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1016/j.mayocp.2022.04.008

Cites

38 of 38 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

12 of 25 cited sources checked · 4 supported · 0 flagged · accuracy not yet measured

  • Premisecitation

    Primary care has been slower to realize the benefits of AI.

    “Primary care has been slower to realize the benefits of AI. [6][7][8]”
    Quote in the paperNot a substantial claim3 of 3 sources checked
    • cites [6] Lin, 2019, Ten ways artificial intelligence will transform primary care, J Gen Intern Med source

      The Introduction describes AI’s potential in primary care, and the section on the patient–physician relationship says further research is needed to determine its impact. Neither passage says primary care has been slower than other fields to realize AI’s benefits. Searches for slower adoption and comparisons with other specialties found no such statement in our copy; page numbers were not available. (LLM, conf=0.74)

      Experimental — accuracy not yet measured
    • cites [7] Kueper, 2020, Artificial intelligence and primary care research: a scoping review, Ann Fam Med source
    • cites [8] Gottliebsen, 2020, Limited evidence of benefits of patient operated intelligent primary care triage tools: findings of a literature review, BMJ Health Care Inform source

      The Discussion ('Limited evidence'; page unavailable) says the reviewed digital triage systems were immature and that evidence of their benefits in primary care was limited. This supports the idea that benefits had not yet been fully realized for those tools, but the paper does not compare primary care’s pace with other settings or assess AI across primary care generally. (LLM, conf=0.84)

      Experimental — accuracy not yet measured
  • Premisecitation

    Early diagnosis of low EF, particularly in asymptomatic patients, is critical in reducing lifetime risk of morbidity and mortality.

    “Early diagnosis of low EF, particularly in asymptomatic patients, is critical in reducing lifetime risk of mobility and mortality.[15][16][17]”
    Quote in the paperSubstantial claim2 of 3 sources checked
    • cites [15] Bowling, 2013, Effects of enalapril in systolic heart failure patients with and without chronic kidney disease: insights from the SOLVD Treatment trial, Int J Cardiol source

      The Introduction and Discussion report that treatment with enalapril reduces mortality and hospitalization in patients with systolic heart failure and reduced EF. But the Methods describe a predominantly symptomatic trial population, and the paper does not establish that early diagnosis—particularly in asymptomatic patients—reduces lifetime risk. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [16] Ponikowski, 2016, Eur Heart J source

      Section 6 recommends treatment for asymptomatic patients with reduced left ventricular function to prevent or delay heart failure and, in some groups, prolong life. This supports the value of identifying low EF before symptoms develop, but the guidelines attribute the benefits to treatment, not to early diagnosis itself, and do not establish a reduction in lifetime risk. (LLM, conf=0.82)

      Experimental — accuracy not yet measured
    • cites [17] Wang, 2003, The epidemiology of “asymptomatic” left ventricular systolic dysfunction: implications for screening, Ann Intern Med source
  • Premisecitation

    An AI tool's effectiveness in improving clinical outcomes depends on its adoption by front-line clinicians.

    “An AI tools' effectiveness in improving clinical outcomes depends on its adoption by front-line clinicians, where “the rubber meets the road.”12”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [12] He, 2019, The practical implementation of artificial intelligence technologies in medicine, Nat Med source

      The 'Education of an AI-literate workforce' section says stakeholders must be involved in implementation and healthcare providers need AI knowledge to work effectively on human–machine teams. 'Future directions' describes improved outcomes as a promise of these technologies. The paper does not specifically establish that improved clinical outcomes depend on adoption by front-line clinicians. (LLM, conf=0.78)

      Experimental — accuracy not yet measured
  • Findingown data

    High adopters were twice as likely to identify low EF as low adopters in patients with an AI-positive ECG.

    “High adopters were twice as likely to identify low EF as low adopters in patients with an AI-positive ECG (Table 3).”
    • Also stated in: § Abstract · plain
      “Among patients with positive AI-ECG, high adopters (n¼41) were twice as likely to diagnose patients with low EF (33.9%) vs low adopters, n¼124, (16.9%); odds ratio, 1.62; 95% CI, 1.21 to 2.17).”
    • Also stated in: § Study Limitations · plain
      “Primary care clinicians who were higher adopters of an AI-enabled clinical decision support tool were twice as likely to diagnose low EF than low adopters.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: Adoption of AI can be validly measured by the proportion of AI-positive ECGs for which the clinician ordered an echocardiogram (and completion).
    • Rests on an assumption: Patients with an AI-positive ECG who did not undergo an echocardiogram did not have low EF, or low EF status is accurately captured by echocardiogram.
  • Findingown data

    A greater proportion of NP/PAs (37.7%) were high adopters compared with physicians (18.8%; P=.008).

    “A greater proportion of the NP/PAs 20, (37.7%) were high adopters compared with physicians 21, (18.8%0;P¼.008).”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 19
  • May combine two claims. 11

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Hypertension, diabetes, myocardial infarction, peripheral artery disease, and atrial fibrillation were determined using a validated natural language processing program to abstract clinical notes.

    “Hypertension, diabetes, myocardial infarction, peripheral artery disease, and atrial fibrillation were determined using a validated natural language processing program to abstract clinical notes.18”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [18] Wen, 2019, Desiderata for delivering NLP to accelerate healthcare AI advancement and a Mayo Clinic NLP-as-a-service implementation, NPJ Digit Med source

      The Backend and cardiac-sarcoidosis sections (pages unavailable) describe an NLP program that extracts information from clinical documents and identifies clinical concepts in unstructured text. They do not establish that the program was validated to determine hypertension, diabetes, myocardial infarction, peripheral artery disease, or atrial fibrillation from clinical notes. (LLM, conf=0.84)

      Experimental — accuracy not yet measured

  • Classified as probably not a substantial claim.

    Premisecitation

    Echocardiograms are considered the test of choice for the definitive diagnosis of left ventricular dysfunction.

    “Echocardiograms are considered the test of choice for the definitive diagnosis of left ventricular dysfunction.16”
    Quote in the paperSubstantial claim1 of 1 sources checked
    • cites [16] Ponikowski, 2016, Eur Heart J source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Other researchers have used validated scales to more precisely characterize clinicians who are more or less likely to use digital tools.

    “Other researchers have used validated scales to more precisely characterize clinicians who are more or less likely to use digital tools.33”
    Quote in the paperNot a substantial claim1 of 1 sources checked
    • cites [33] Hatz, 2017, The PMA scale: a measure of physicians' motivation to adopt medical devices, Value Health source

      The Conclusions describe the PMA scale as a validated measure of physicians’ motivation to adopt medical devices. The Discussion says whether scale scores indicate a higher or lower likelihood of adoption remains for future research to establish; the paper also does not validate the scale for digital tools generally. (LLM, conf=0.88)

      Experimental — accuracy not yet measured

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Several studies have shown improved diagnostic accuracy, patient triage, and clerical burden reduction.

    “On the other hand, several studies have shown improved diagnostic accuracy, patient triage, and clerical burden reduction.10,11”
    Quote in the paperNot a substantial claim1 of 2 sources checked
    • cites [10] Dankwa-Mullan, 2019, Transforming diabetes care through artificial intelligence: the future is here, Popul Health Manag source

      The Results report high sensitivity and specificity for AI-based diabetic-retinopathy screening, and the Discussion describes screening decisions that lead to immediate specialist referrals. These support diagnostic performance and a form of patient triage. Our copy does not report that the studies reduced clerical burden. (LLM, conf=0.86)

      Experimental — accuracy not yet measured
    • cites [11] Halamka, 2019, An FP's guide to AI-enabled clinical decision support, J Fam Pract

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Computer literacy is an important predictor in the successful adoption of clinical decision support tools.

    “In general, computer literacy is an important predictor in the successful adoption of clinical decision support tools and AI-enabled tools may require even higher technical proficiency.25,26”
    Quote in the paperNot a substantial claim2 of 2 sources checked
    • cites [25] Sutton, 2020, An overview of clinical decision support systems: benefits, risks, and strategies for success, NPJ Digit Med source
    • cites [26] Laka, 2021, Factors That Impact the Adoption of Clinical Decision Support Systems (CDSS) for Antibiotic Management, Int J Environ Res Public Health source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A total of 11,573 adult patients had ECGs ordered by intervention-arm clinicians.

    “A total of 11,573 adult patients had ECGs ordered by clinicians in the intervention arm.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Among the 165 intervention-arm clinicians, 112 (67.9%) were physicians and 53 (32.1%) were NP/PAs.

    “Among the 165 clinicians randomized to the intervention arm, 112 (67.9%) were physicians, and 53 (32.1%) were NP/PAs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Among the 165 intervention-arm clinicians, 118 (72.4%) were family medicine and 45 (27.6%) internal medicine.

    “Specialty representation was 118 (72.4%) family medicine and 45 (27.6%) internal medicine.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    In the intervention arm of the EAGLE trial, 165 primary care clinicians from 60 care teams were included in this secondary analysis.

    “This analysis includes 165 clinicians from 60 care teams that were randomized to the intervention group (ie, received results of an AI ECG that detects possible ventricular dysfunction) (Figure).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The study did not find significant differences in age, sex, number of years in practice, or number of patients cared for in a panel.

    “We did not find significant differences in several other important variables, including age, sex, number of years in practice, or the number of patients cared for in a panel.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    When analyzing AI-negative ECGs, there was no difference between high and low adopters in the number of echocardiograms ordered or the number of patients identified with low EF (P=.46).

    “When analyzing AI-negative ECGs, we found no difference between high and low adopters in both the number of echocardiograms ordered or the number of patients identified with low EF (P¼.46).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    In multivariable logistic regression, NP/PA vs physician, family medicine vs internal medicine, and patient panel complexity were no longer significantly associated with adoption.

    “Variables that were significant in the univariate model (NP/PA vs physicians, family medicine vs internal medicine, and patient panel complexity) were no longer significantly associated with adoption in a multivariable logistic regression analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    High adopters reported feeling less comfortable caring for patients with low EF (P=.06).

    “Although not statistically significant, high adopters reported feeling less comfortable caring for patients with low EF (P¼.06).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    The study found little evidence suggesting APPs, or clinicians with less complex patients, show higher adoption rates of decision support tools.

    “We found little evidence suggesting APPs, or clinicians with less complex patients, show higher adoption rates of decision support tools.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Interpretationown data

    When trusted, the AI algorithm improves accuracy and reduces variability in detecting patients with low EF.

    “This finding would indicate that, when trusted, the algorithm improves accuracy and reduces variability in detecting patients with low EF.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: There is no unmeasured confounding between high/low adopter groups and low EF detection; differences in diagnostic yield are attributable to adoption or trust of the AI algorithm.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Mixed-effect logistic regression was used to compare low EF incidence between the high-adopter and low-adopter groups with the care team as a random effect.

    “Mixed-effect logistic regression was used to compare low EF incidence between the high-adopter and the low-adopter groups with the care team as a random effect.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationown data

    The high-adopter group tended to be APPs and have less complicated patient panels, suggesting their patients may be less likely to have undiagnosed low EF.

    “This is rather remarkable given that the high-adopter group tended to be an APP and have less complicated patient panels, suggesting that their patients may be less likely to have undiagnosed low EF.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Electrocardiograms were considered positive if the probability of having low EF was higher than a threshold selected to have similar sensitivity and specificity for a cohort with a prevalence of 9.0%.

    “Electrocardiograms were considered positive if the probability of having low EF was higher than a threshold selected to have similar sensitivity and specificity for a cohort with a prevalence of 9.0%.14”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [14] Yao, 2020, ECG AI-Guided Screening for Low Ejection Fraction (EAGLE): rationale and design of a pragmatic cluster randomized trial, Am Heart J source

  • May combine two claims.

    Premiseown data

    In the EAGLE study, 358 primary care clinicians were randomized to intervention or control across 48 upper Midwest clinics/hospitals; the primary endpoint was a new diagnosis of EF less than or equal to 50% within 3 months.

    “In this study, 358 primary care clinicians were randomized to either an intervention group (access to AI screening results) or a control group (usual care) across 48 upper Midwest clinics/hospitals.The primary endpoint was a new diagnosis of EF less than or equal to 50% within 3 months.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [13] Yao, 2021, Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial, Nat Med source
    • cites [14] Yao, 2020, ECG AI-Guided Screening for Low Ejection Fraction (EAGLE): rationale and design of a pragmatic cluster randomized trial, Am Heart J source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    The trial was registered (ClinicalTrials.gov number NCT04000087).

    “The trial was registered (ClinicalTrials.govnumber NCT04000087), and the study protocol and materials have been previously published.13”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [13] Yao, 2021, Artificial intelligence-enabled electrocardiograms for identification of patients with low ejection fraction: a pragmatic, randomized clinical trial, Nat Med source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The original study was approved by the Mayo Clinic Institutional Review Board, in accordance with the Declaration of Helsinki.

    “The original study was approved by the Mayo Clinic Institutional Review Board, in accordance with the Declaration of Helsinki.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    There is some evidence that primary care physicians are more likely to adopt EHR technology.

    “There is some evidence that primary care physicians are more likely to adopt EHR technology.27,30”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [27] Decker, 2012, Physicians in nonprimary care and small practices and those age 55 and older lag in adopting electronic health record systems, Health Aff (Millwood) source
    • cites [30] DesRoches, 2008, Electronic health records in ambulatory care — a national survey of physicians, N Engl J Med source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    One study found early adopters of personal digital assistant to be younger with less clinical experience and more facile with technology.

    “One study assessing characteristics of early and late adopters in the clinical use of personal digital assistant, found early adopters to be younger with less clinical experience and more facile with the use of technology.29”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Vishwanath, 2009, Patterns and changes in prescriber attitudes toward PDA prescription-assistive technology, Int J Med Inform source

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Younger age is a characteristic of early adoption in the use of digital health tools.

    “Several studies have suggested younger age as a characteristic of early adoption in the use of digital health tools.27,28”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [27] Decker, 2012, Physicians in nonprimary care and small practices and those age 55 and older lag in adopting electronic health record systems, Health Aff (Millwood) source
    • cites [28] Wylie, 2014, Perceptions of electronic health record implementation: a statewide survey of physicians in Rhode Island, Am J Med source

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    The study reported clinician behavior (echocardiogram ordered and performed) and outcome (diagnosis of low EF), but there are many scenarios where not ordering an echocardiogram is appropriate.

    “In addition, we reported clinician behavior (ie, an echocardiogram was ordered and performed) and the outcome (eg, diagnosis of low EF), but there are many scenarios where not ordering an echocardiogram is appropriate (eg, patient preference, cost, or other clinical justification).”
    Quote in the paperNot a substantial claim
25 of 25

  • The paper relies on this without support.

    Few studies have examined the characteristics of clinicians who readily embrace AI tools versus those who are more hesitant and the clinical outcomes associated with these two approaches.

    “To date, few studies examine the characteristics of clinicians who readily embrace AI tools (high adopters) vs those who are more hesitant (low adopters) and the clinical outcomes associated with these two approaches.”

  • The paper relies on this without support.

    Adoption of AI can be validly measured by the proportion of AI-positive ECGs for which the clinician ordered an echocardiogram (and completion).

    “Adoption of AI was defined at the clinician level by dividing the number of echocardiograms for AI-positive ECGs by the total number of AI-positive ECGs.”

  • The paper relies on this without support.

    Patients with an AI-positive ECG who did not undergo an echocardiogram did not have low EF, or low EF status is accurately captured by echocardiogram.

    “This translates to a diagnostic yield (proportion of AI-positive with confirmed low EF) of 33.9% in high adopters, and 16.3% for low adopters (P<.001).”

  • The paper relies on this without support.

    There is no unmeasured confounding between high/low adopter groups and low EF detection; differences in diagnostic yield are attributable to adoption or trust of the AI algorithm.

    “This finding would indicate that, when trusted, the algorithm improves accuracy and reduces variability in detecting patients with low EF.”

  • The authors state this as an assumption.

    Younger clinicians are more facile with computer-based technology and received greater exposure during their training.

    “This hypothesis was assumed based on the perception that younger clinicians would be more facile with computer-based technology and received greater exposure during their training.”
5 of 5

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The main failure modes of safety-net AI-ECG deployment are excess downstream testing, incomplete follow-up, and alert fatigue.

    “Its main failure modes are excess downstream testing, incomplete follow-up, and alert fatigue.”

    From AI-ECG for Echocardiography Triage in Structural Heart Disease: Evidence, Implementation, and Future Directions (2026) · cites this paper as [16]

    The Effectiveness of Detecting Low EF section reports that low-adopting clinicians ordered echocardiograms after 43.0% of positive AI-ECG results, supporting a concern about incomplete follow-up. The Discussion says high adopters were not indiscriminately ordering more tests, and the paper does not identify excess downstream testing or alert fatigue as main failure modes. The study was conducted in a Midwest health system; it does not establish these as the main failure modes of safety-net deployment. Page numbers were not available in our copy. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Low-LVEF detection is the most mature use case, having already progressed from model development to randomized trials and early implementation studies.

    “Low-LVEF detection is the most mature use case, having already progressed from model development to randomized trials and early implementation studies.”

    From AI-ECG for Echocardiography Triage in Structural Heart Disease: Evidence, Implementation, and Future Directions (2026) · cites this paper as [16]

    The Abstract describes a randomized primary-care trial of AI-assisted low-EF detection, and the Methods identify this paper as a secondary analysis of that trial; its focus on clinician adoption supports the early-implementation portion of the claim. The paper does not compare low-LVEF detection with other AI use cases or establish that it is the most mature. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

  • Gatekeeper use carries a different clinical risk profile because false-negative or poorly calibrated low-risk outputs may delay diagnosis.

    “By contrast, gatekeeper use carries a different clinical risk profile because false-negative or poorly calibrated low-risk outputs may delay diagnosis.”

    From AI-ECG for Echocardiography Triage in Structural Heart Disease: Evidence, Implementation, and Future Directions (2026) · cites this paper as [16]

    The Treatment section says a negative AI-ECG result recommended no further testing unless symptoms or other conditions indicated it. Table 3 reports patients with negative AI-ECGs whose echocardiograms nevertheless showed low ejection fraction, supporting a potential false-negative concern. The paper does not assess calibration, describe the tool as a gatekeeper, or show that negative results delayed diagnosis. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

  • A safety-net model may be useful when positive AI-ECG results enrich echocardiography yield and lead to timely confirmatory evaluation.

    “A safety-net model may be useful when positive AI-ECG results enrich echocardiography yield and lead to timely confirmatory evaluation.”

    From AI-ECG for Echocardiography Triage in Structural Heart Disease: Evidence, Implementation, and Future Directions (2026) · cites this paper as [16]

  • The regulatory maturity of AI-ECG for SHD screening remains uneven.

    “The regulatory maturity of AI-ECG for SHD screening remains uneven.”

    From AI-ECG for Echocardiography Triage in Structural Heart Disease: Evidence, Implementation, and Future Directions (2026) · cites this paper as [16]

    The Methods and Treatment sections describe AI-ECG screening for low ejection fraction, while the Discussion addresses variation in clinician adoption. Our copy does not address the regulatory status or maturity of AI-ECG for structural heart disease screening (searched: regulatory approval, FDA clearance, SHD screening). (LLM, conf=0.75)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Family-medicine P value conflicts with Table 1

    Major

    The results report P=.010 and treat specialty as significant in the univariate analysis, whereas Table 1 reports P=.10 for specialty.

  • Low-adopter AI-negative diagnostic yield is incorrect

    Minor

    Table 3 reports a 1.15% yield, but 85 low-EF results among 8,788 AI-negative ECGs yield approximately 0.97%.

    Reported 1.15 · recomputed 0.967228

  • High-adopter AI-negative diagnostic yield is incorrect

    Minor

    Table 3 reports a 0.97% yield, but 24 low-EF results among 2,093 AI-negative ECGs yield approximately 1.15%. The two AI-negative yield values appear to be interchanged.

    Reported 0.97 · recomputed 1.146679

  • Study locations are described as spanning two and three states

    Minor

    The methods place the included practices across Minnesota and Wisconsin, but the limitations describe the included clinicians as being across three Midwest states.

4 of 4

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Rushlow, D. R., Croghan, I. T., Inselman, J. W., Thacher, T. D., Friedman, P. A., Yao, X., Pellikka, P. A., Lopez-Jimenez, F., Bernard, M. E., Barry, B. A., Attia, I. Z., Misra, A., Foss, R. M., Molling, P. E., Rosas, S. L., & Noseworthy, P. A.
Affiliations
  • Mayo Clinic (US)
  • Mayo Clinic Health System (US)
Corresponding author
Rushlow, David R.
ORCID
Venue
Mayo Clinic Proceedings
Publisher
Elsevier BV
Published
2022-11
Volume
97
Issue
11
Pages
2076-2085
ISSN
0025-6196 (ISSN-L)
DOI
10.1016/j.mayocp.2022.04.008
OpenAlex
W4308031451
Cited by
33
References
32
Citations by year
2026: 8 · 2025: 11 · 2024: 12 · 2023: 1 · 2022: 1
FWCI
1.36
Type
journal article
Language
en
Open access
Licence
Locations
  • Mayo Clinic Proceedings (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
Funders
Mayo Clinic
Topics
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Cardiology and Cardiovascular Medicine › ECG Monitoring and Analysis
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
artificial intelligence, left ventricular systolic dysfunction, reduced left ventricular ejection fraction, clinician adoption, primary care, clinical decision support alerts, advanced practice providers, patient complexity
MeSH terms
Artificial Intelligence, Electrocardiography, Humans, Primary Health Care, Stroke Volume, Ventricular Function, Left, Ventricular Dysfunction, Left

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-27.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-27.

Cited sources are checked by gpt-6-sol.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review