Cited by 42Healthcare Systems and TechnologyTelemedicine and Telehealth ImplementationElectronic Health Records Systems

Using the Quadruple Aim Framework to Measure Impact of Heath Technology Implementation: A Case Study of eConsult

Open access
Liddy, Clare, Keely, Erin·2018·The Journal of the American Board of Family Medicine

Abstract

BACKGROUND: Health technology solutions are too often implemented without a true understanding of the system-level problem they seek to address, resulting in excessive costs, poor adoption, ineffectiveness, and ultimately failure. Before implementing or adopting health care innovations, stakeholders should complete a thorough assessment to ensure effectiveness and value. In this article, we describe how to evaluate the impact of a health technology innovation through the 4 dimensions of care outlined by the Quadruple Aim Framework, using our experience with the Champlain Building Access to Specialists through eConsultation (BASE) eConsult service as a case example. METHODS: A descriptive overview of data was collected between April 1, 2011, and August 31, 2017, using 4 dimensions of care outlined by the Quadruple Aim Framework: patient experience, provider experience, costs, and population health. Findings were drawn from use data, primary care provider closeout surveys, surveys/interviews with patients and provider, and costing data. RESULTS: Overall, patients have received access to specialist advice within days and find the advice useful in 86% of cases. Provider experience is very positive, with satisfaction ratings of high/very high value in 94% of cases. The service cost a weighted average of $47.35/case, compared with $133.60/case for traditional referrals. In total, 1,299 primary care providers have enrolled in the service, completing 28,838 cases since 2011. Monthly case volumes have grown from an average of 13 cases/month in 2011 to 969 cases/month in 2016. CONCLUSIONS: The eConsult service has been widely adopted in our region and is currently expanding to new jurisdictions across Canada. However, although we successfully demonstrated eConsult's impact on patient experience, provider satisfaction, and reducing costs, we met several challenges in evaluating its impact on population health. More work is needed to evaluate eConsult's impact on key population health metrics (eg, mortality, morbidity, and system use). Efforts to conduct such evaluations are underway.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.3122/jabfm.2018.03.170397

Cites

37 of 37 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The Champlain BASE eConsult service has grown from a small proof-of-concept to a fully implemented regional service, with a solid body of evidence supporting its ability to improve access to specialist advice, reduce wait times, lower costs, and deliver high levels of patient and provider satisfaction.

    “First launched in 2009, this innovative service has grown from a small proof-of-concept to a fully implemented regional service, with a solid body of evidence supporting its ability to improve access to specialist advice, reduce wait times, lower costs, and deliver high levels of patient and provider satisfaction [8–10].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

1 of the 38 references couldn’t be matched to a known paper.

Research works printed but not linked: 5

  1. McAdams M, Cannavo L, Orlander JD. A medical specialty e-consult program in a VA health care sys- tem. Fed Pract 2014;31:26 -31.
  2. National Audit Office. Review of the final benefits statement for programs previously managed under the National Programme for IT in the NHS. Lon- don, United Kingdom: NAO; 2013. Available from: https://www.nao.org.uk/wp-content/uploads/2013/ 06/10171-001_NPfiT_Review.pdf. Accessed March 2, 2017.
  3. Champlain LHIN. Sub-Region Population Health Profiles. Available from: http://www.champlainlhin. on.ca/ϳ/media/sites/champlain/Goals_Achvmnts/ 454 JABFM May-June 2018 Vol. 31 No. 3 http://www.jabfm.org on 12 July 2023 by guest. Protected by copyright. http://www.jabfm.org/ J Am Board Fam Med: first published as 10.3122/jabfm.2018.03.170397 on 9 May 2018. Downloaded from IHSP/SubRegions/Data/201712SRHlthPrflsTech RptEN.pdf?laϭen. Accessed April 4, 2018.
  4. Rodriguez KL, Burkitt KH, Bayliss NK, et al. Vet- eran, primary care provider, and specialist satisfac- tion with electronic consultation. JMIR Med Inform 2015;3(1):e5.
  5. Sabesan S. Specialist cancer care through telehealth models. Aust J Rural Health 2015;23:19 -23. doi: 10.3122/jabfm.2018.03.170397 Impact of Heath Technology Implementation 455 on 12 July 2023 by guest. Protected by copyright. http://www.jabfm.org/ J Am Board Fam Med: first published as 10.3122/jabfm.2018.03.170397 on 9 May 2018. Downloaded from · DOI

Cited by

Sorted by citations

Authors
Liddy, C., & Keely, E.
Affiliations
University of Ottawa (CA)
Venue
The Journal of the American Board of Family Medicine
Published
2018-05
Volume
31
Issue
3
Pages
445-455
ISSN
  • 1557-2625 (ISSN-L)
  • 1558-7118
OpenAlex
W2800380926
Cited by
42
References
38
Citations by year
2026: 1 · 2024: 4 · 2023: 7 · 2022: 10 · 2021: 4 · 2020: 5 · 2019: 9 · 2018: 2
FWCI
8.27
Type
journal article
Language
en
Open access
Locations
  • The Journal of the American Board of Family Medicine (publishedVersion)
  • PubMed (publishedVersion)
Funders
Ontario Ministry of Health and Long-Term Care
Topics
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Healthcare Systems and Technology
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
Keywords
e-consultation, health technology implementation, specialist referral, access to specialist care, patient experience, provider satisfaction, health care costs, population health, primary care
MeSH terms
Implementation Science, Canada, Health Services Accessibility, Humans, Primary Health Care, Remote Consultation

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review