CoChat
Sign Up
Cited by 25Health Policy Implementation ScienceTelemedicine and Telehealth ImplementationHealthcare Systems and Technology

Key factors for national spread and scale-up of an eConsult innovation

Open access
Moroz, Isabella, Archibald, Douglas, Breton, Mylaine, Cote-Boileau, Elizabeth, Crowe, Lois, Horsley, Tanya, Hyseni, Lirjie, Johar, Gina +9 more·2020·Health Research Policy and Systems
DOI 10.1186/s12961-020-00574-0PMC PMC7268606
Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Expanding healthcare innovations from the local to national level is a complex pursuit requiring careful assessment of all relevant factors. In this study (a component of a larger eConsult programme of research), we aimed to identify the key factors involved in the spread and scale-up of a successful regional eConsult model across Canada. Methods: We conducted a constant comparative thematic analysis of stakeholder discussions captured during a full-day National eConsult Forum meeting held in Ottawa, Canada, on 11 December 2017. Sixty-four participants attended, representing provincial and territorial governments, national organisations, healthcare providers, researchers and patients. Proceedings were recorded, transcribed and underwent qualitative analysis using the Framework for Applied Policy Research. Results: This study identified four main themes that were critical to support the intentional efforts to spread and scale-up eConsult across Canada, namely (1) identifying population care needs and access problems, (2) engaging stakeholders who were willing to roll up their sleeves and take action, (3) building on current strategies and policies, and (4) measuring and communicating outcomes. Conclusions: Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation. Further research is needed to develop and strengthen the conceptual and applied foundations of the spread and scale-up of healthcare innovations, especially in the context of emergent learning health systems across Canada and beyond.

AI snapshot

Written from the paper’s full text.

TL;DR

This qualitative analysis of a 2017 National eConsult Forum identified four key factors for spreading and scaling up eConsult across Canada: population care needs/access, stakeholder engagement, building on current strategies/policies, and measuring/communicating outcomes. It presents a four-point checklist for healthcare innovators seeking to spread and scale innovations.

Key findings

  • Four main themes were critical: (1) identifying population care needs and access problems, (2) engaging stakeholders willing to take action, (3) building on current strategies and policies, and (4) measuring and communicating outcomes.
  • Of 103 invited participants, 64 attended (62%); attendees self-identified as 22% administrators, 16% general public including patient partners, 36% healthcare professionals and physicians including fellows, 14% non-healthcare, 7% researchers, and 5% sponsor representatives.
  • Participants emphasized eConsult as a necessity for remote regions, citing Nunavut’s 25 communities across three time zones, and for populations with complex circumstances such as rural, Indigenous, pediatric, mental health, LGBT, rehab, long-term care/dementia/frailty, and chronic pain.
  • The paper recommends a four-point checklist covering population needs, stakeholder engagement, existing policies and strategies, and measurement/communication.

Methods

  • Qualitative content analysis using the Framework for Applied Policy Research, based on a full-day National eConsult Forum held in Ottawa, Canada, on 11 December 2017.
  • Data came from 19 audio-recorded plenary and small-group sessions; after excluding 8 transcripts that provided no policy discussion, 11 transcripts were included.
  • Two reviewers independently coded transcripts and revised the codebook with a national working group of 14 self-selected Forum attendees; the framework by Charif et al. was adapted by adding “patients” and deleting “financing,” with final consensus on 21 August 2018.

Limitations

  • Stated in the paper: qualitative research limitations include participants’ personal biases and idiosyncrasies; attendees, working group, and analysts were supportive of eConsult’s national spread and scale-up; generalisability is limited by Canada’s 13 provincial/territorial jurisdictions; and quotation origins by participant type could not be determined in small group discussions.
  • Evident from the text: the findings rely on a single-day forum and a self-selected working group, which may underrepresent dissenting or less supportive perspectives.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s12961-020-00574-0

Cites

26 of 26 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Participants emphasised the role of engaging patients as partners in the design and promotion of eConsult.

    “In terms of patients, participants emphasised the role of engaging patients as partners in the design and promotion of eConsult:”
    Quote in the paperSubstantial claim
  • Findingown data

    When discussing strategies for eConsult's spread and scale-up, participants emphasised the importance of collaborating with stakeholders from a variety of disciplines and organisations.

    “When discussing strategies for eConsult’s spread and scale-up, participants emphasised the importance of collaborating with stakeholders from a variety of disciplines and organisations.”
    Quote in the paperSubstantial claim
  • Findingown data

    An important component of collaborating with stakeholders was establishing clear governance to decide what team members or groups would be responsible for making different decisions and oversee strategic and operational components of the service.

    “An important component of collaborating with stakeholders was establishing clear governance in order to decide what team members or groups would be responsible for making different decisions and oversee strategic and operational components of the service.”
    Quote in the paperSubstantial claim
  • Findingown data

    Participants emphasised the importance of discussing eConsult's benefits beyond improving access or reducing wait times.

    “participants emphasised the importance of discussing its benefits beyond improving access or reducing wait times.”
    Quote in the paperSubstantial claim
  • Findingown data

    Participants highlighted the importance of identifying and engaging individuals who can influence change, especially patient partners and those within organisations that will play a significant role in governance, development and implementation.

    “participants highlighted the importance of identifying and engaging individuals who can influence change, with specific emphasis on patient partners and those within organisations that will end up playing a significant role in the service’s governance, development and implementation.”
    Quote in the paperSubstantial claim
  • May combine two claims. 25
  • Classified as probably not a substantial claim. 43

  • May combine two claims.

    Findingown data

    Participants identified particular challenges of providing specialty care access to populations with complex circumstances and discussed the importance of reaching these populations with eConsult.

    “Participants identified the particular challenges of providing specialty care access to populations with complex circumstances and discussed the importance of reaching these populations with eConsult.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Many participants noted that patients with complex circumstances often face inequity in access and described eConsult's ability to address this inequity as one of its chief advantages.

    “Many participants noted that patients with complex circumstances often face inequity in access and described eConsult’s ability to address this inequity as one of its chief advantages.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Four main themes emerged from the analysis of the Policy Forum: (1) population care needs and access, (2) stakeholders and their roles, (3) building on current strategies and policies, and (4) measuring and communicating outcomes.

    “Four main themes emerged from our analysis, namely (1) population care needs and access, (2) stakeholders and their roles, (3) building on current strategies and policies, and (4) measuring and communicating outcomes.”
    • Also stated in: § Results · plain
      “This study identified four main themes that were critical to support the intentional efforts to spread and scale-up eConsult across Canada, namely (1) identifying population care needs and access problems, (2) engaging stakeholders who were willing to roll up their sleeves and take action, (3) building on current strategies and policies, and (4) measuring and communicating outcomes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing).

    Findingown data

    Participants suggested that the provinces' professional colleges could oversee licensing issues, as they already perform similar duties.

    “Additionally, participants suggested that the provinces’ professional colleges could oversee licensing issues, as they already perform similar duties.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants discussed the importance of keeping the needs of the target population central to any efforts to spread and scale-up eConsult.

    “Participants discussed the importance of keeping the needs of the target population central to any efforts to spread and scale-up eConsult.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Participants described excessive wait times as a significant barrier to accessing care, which eConsult had the potential to address.

    “Specifically, participants described excessive wait times as a significant barrier to accessing care, which eConsult had the potential to address:”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    In more remote regions of the Canadian north, eConsult was seen as a necessity.

    “In more remote regions of the Canadian north, eConsult was seen as a necessity:”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Participants identified a number of national organisations with the jurisdiction and resources to oversee accountability in privacy, licensing and education, including the Canadian Medical Protection Association, the Royal College of Physicians and Surgeons of Canada, and the College of Family Physicians of Canada.

    “Participants identified a number of national organisations with the jurisdiction and resources to oversee accountability in these issues, including the Canadian Medical Protection Association, the Royal College of Physicians and Surgeons of Canada, and the College of Family Physicians of Canada.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    Participants recognised eConsult's ability to provide support for PCPs caring for patients with complex circumstances and to enable interdisciplinary disease management.

    “Participants also recognised eConsult’s ability to provide support for PCPs caring for patients with complex circumstances and to enable interdisciplinary disease management.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Targeting rural areas was seen as particularly important.

    “Targeting rural areas was seen as particularly important.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    The original Charif et al. framework needed further adaptation following the addition of 'patients' to align with themes emerging from the data, and financing was deleted because it was not a theme from the data.

    “Upon careful consideration, it was determined that the original framework by Charif et al. [17] needed further adaptation following the addition of ‘patients’ to more appropriately align with the themes emerging from the data (e.g. financing was deleted as it was not coming out as a theme from the data).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites Charif et al. [17] Charif AB, Zomahoun HTV, LeBlanc A, Langlois L, Wolfenden L, Yoong SL, et al. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implement Sci. 2017;12:139. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Findingown data

    Governance also pertained to accountability, which affected such areas as privacy, licensing and education.

    “Governance also pertained to accountability, which affected such areas as privacy, licensing and education:”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants suggested aligning with existing government priorities as effective strategies to introduce new healthcare delivery programmes such as eConsult.

    “participants suggested aligning with existing government priorities (e.g. Health Links, opioid strategy) as effective strategies to introduce new healthcare delivery programmes such as eConsult”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Participants emphasised the importance of understanding the current context in which the service is being implemented, as existing policies or programmes could be adopted or adapted to eConsult, allowing innovators to avoid 'reinventing the wheel'.

    “Participants emphasised the importance of understanding the current context in which the service is being implemented, as in many cases there are already policies or programmes in place that could be adopted by or adapted to eConsult, allowing innovators to avoid ‘reinventing the wheel’.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingreasoning

    Participants noted that a system of evaluation should be an integral part of the service and include measures relevant to physicians, administrators and patients.

    “participants noted that a system of evaluation should be an integral part of the service and include measures that are relevant to physicians, administrators and patients.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Including eConsult in medical curriculum as a standard of practice was highlighted as important.

    “Final reflection highlighted the importance of including eConsult in medical curriculum as a standard of practice”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingreasoning

    The 'measuring and communicating outcomes' theme embodied the importance of systematically tracking eConsult's progress and growth across Canada and disseminating the results to support spread and scale-up.

    “The ‘measuring and communicating outcomes’ theme embodied the importance of systematically tracking eConsult’s progress and growth across Canada and disseminating the results to support spread and scale-up.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Using eConsult for strengthening education and communication among healthcare providers was viewed as an important learning opportunity, especially for chronic pain care and geriatric care.

    “using eConsult for strengthening education and communication among healthcare providers was viewed as an important learning opportunity, especially for chronic pain care and geriatric care”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants emphasised the importance of building upon existing eHealth strategies to enable the spread and scale-up of eConsult.

    “Participants emphasised the importance of building upon existing eHealth strategies to enable the spread and scale-up of eConsult.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    Other effective enablers included taking a multidisciplinary approach, including a research and education focus, and capitalising on physician leadership, patient engagement, and connecting people and organisations.

    “Other effective enablers included taking a multidisciplinary approach, including a research and education focus, and capitalising on physician leadership, patient engagement, and connecting people and organisations.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Communicating the benefits and impacts of eConsult with healthcare providers was raised as an important point.

    “Communicating the benefits and impacts of eConsult with healthcare providers was also raised as an important point”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    System-level enablers for broad scale eConsult implementation included developing strategies for quality assurance, improving technology, and adopting clinician and practice workflows.

    “System-level enablers included developing strategies for quality assurance, improving technology, and adopting clinician and practice workflows.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    This study identified four main themes critical to support spread and scale-up of eConsult across Canada: identifying population care needs and access problems; engaging stakeholders; building on current strategies and policies; measuring and communicating outcomes.

    “This study identified four main themes that were critical to support the intentional efforts to spread and scale-up eConsult across Canada, namely (1) identifying population care needs and access problems; (2) engaging stakeholders who were willing to roll up their sleeves and take action; (3) building on current strategies and policies; and (4) measuring and communicating outcomes.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: eConsult should be spread and scaled up across Canada.
    • Rests on an assumption: The views and experiences expressed by forum participants accurately reflect the factors that drive successful spread and scale-up of eConsult.

  • Probably not a substantial claim: may not pass S1 (asserted), S2 (checkable), S3 (load-bearing).

    Findingown data

    Existing regulatory and best practices, including templates for licensing in other jurisdictions, could be applied to eConsult.

    “Similar suggestions were raised regarding regulation, in that existing regulatory and best practices, including templates for licensing in other jurisdictions, could be applied to eConsult.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Findingown data

    Participants stressed the importance of making the service simple and reliable to encourage PCP and specialists to use it.

    “Participants stressed the importance of making the service simple and reliable in order to encourage PCP and specialists to use it.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    Routine evaluation at the patient level is currently not built into the system since it was developed primarily as a communication platform for providers.

    “Routine evaluation at the patient level is currently not built into the system since it was developed primarily as a communication platform for providers.”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    The Charif et al. framework identified four key components of scaling up evidence-based practices in primary care: human resources, healthcare infrastructure, policy/regulation, and financing.

    “Ultimately, we selected a thematic framework created by Charif et al. [17] since it was specific to primary care interventions and identified the following four key components of scaling up evidence-based practices in primary care: human resources (e.g. policy-makers/managers, providers, community healthcare workers), healthcare infrastructure, policy/regulation, and financing.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites Charif et al. [17] Charif AB, Zomahoun HTV, LeBlanc A, Langlois L, Wolfenden L, Yoong SL, et al. Effective strategies for scaling up evidence-based practices in primary care: a systematic review. Implement Sci. 2017;12:139. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    To ensure rigor and reduce the risk of selection bias, all excluded transcripts were thoroughly read by one reviewer to look for outliers and any disconfirming evidence.

    “To ensure rigor and reduce the risk of selection bias, all excluded transcripts were thoroughly read by one reviewer (AM) to look for outliers and any disconfirming evidence.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Engagement of Forum participants in the coding of the data helped to verify interpretation and increase validity of the data analysis.

    “Using a participatory approach, engagement of Forum participants in the coding of the data helped to verify interpretation and increase validity of the data analysis.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The analysis excluded eight transcripts that provided no discussion of policy, resulting in 11 transcripts included in the final sample.

    “Eight transcripts were excluded from the analysis for this reason, resulting in 11 transcripts included in the final sample.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Active participation of patient partners transitioned the discussions and policy recommendations for the national spread and scale-up of eConsult from provider-centred to patient-centred thinking.

    “We expanded the framework dimensions to include ‘patients’ based on our previous work, where active participation of patient partners transitioned the discussions and policy recommendations for the national spread and scale-up of eConsult from provider-centred to patient-centred thinking [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Liddy C, Moroz I, Joschko J, Horsley T, Kuziemsky C, Kovacs Burns K, et al. Using an integrated knowledge translation (IKT) approach to enable policy change for electronic consultations in Canada. Healthc Pol. 2018;14:19–29.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    A total of 64 of 103 invited participants (62%) attended the meeting.

    “A total of 64 of 103 invited participants (62% of those invited) attended the meeting.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation.

    “Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Think Tank yielded specific, actionable recommendations regarding provincial and national policy related to eConsult, particularly in the areas of interjurisdictional licensing, payment, privacy, quality assurance and regulation.

    “The Think Tank yielded specific, actionable recommendations regarding provincial and national policy related to eConsult, particularly in the areas of interjurisdictional licensing, payment, privacy, quality assurance and regulation [14].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [14] Liddy C, Moroz I, Joschko J, Horsley T, Kuziemsky C, Kovacs Burns K, et al. Using an integrated knowledge translation (IKT) approach to enable policy change for electronic consultations in Canada. Healthc Pol. 2018;14:19–29.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The Champlain BASE eConsult service specialists respond to cases in a median of 0.9 days and two-thirds of patients receive care without needing a face-to-face specialist visit.

    “Specialists respond to cases in a median of 0.9 days and two-thirds of patients receive care without needing a face-to-face specialist visit [7].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Liddy C, Moroz I, Afkham A, Keely E. Sustainability of a primary care driven eConsult service. Ann Fam Med. 2018;16:120–8. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    Data saturation was likely reached.

    “Members of the working group highlighted a few minor areas for adjustment, identified key quotations and confirmed that data saturation was likely reached.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Among attendees, 22% self-identified primarily as administrators, 16% as general public (including patient partners), 36% as healthcare professionals and physicians (including fellows), 14% as non-healthcare (e.g. government officials), 7% as researchers (including PhD students), and 5% as sponsor (partner organisation) representatives.

    “Among attendees, 22% self-identified primarily as administrators, 16% as general public, including patient partners, 36% as healthcare professionals and physicians (including fellows), 14% as non-healthcare (e.g. government officials), 7% as researchers (including PhD students), and 5% as sponsor (partner organisation) representatives.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    A study of non-adoption and abandonment of innovations found that limited success in attempts to scale-up, spread or sustain the programme was explained by the tendency to assume that issues to be addressed were simple rather than complex.

    “A study of non-adoption and abandonment of innovations found that limited success in attempts to scale-up, spread or sustain the programme was explained, in many instances, by the tendency to assume that the issues to be addressed were simple or uncomplicated (hence knowable, predictable and controllable) rather than complex (i.e. inherently unknowable, unpredictable, dynamic and emergent) [4].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [4] Greenhalgh T, Wherton J, Papoutsi C, Lynch J, Hughes G, A'Court C, et al. Beyond adoption: a new framework for theorizing and evaluating nonadoption, abandonment, and challenges to the scale-up, spread, and sustainability of health and care technologies. J Med Internet Res. 2017;19:e367. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The Champlain BASE eConsult service has grown from a small proof-of-concept to a fully implemented regional service, with a solid body of evidence supporting its ability to improve access to specialist advice, reduce wait times, lower costs, and deliver high levels of patient and provider satisfaction.

    “First launched in 2009, this innovative service has grown from a small proof-of-concept to a fully implemented regional service, with a solid body of evidence supporting its ability to improve access to specialist advice, reduce wait times, lower costs, and deliver high levels of patient and provider satisfaction [8–10].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [8] Liddy C, Keely E. Using the quadruple aim framework to measure impact of heath technology implementation: a case study of eConsult. J Am Board Fam Med. 2018;31:445–55. source

      Not checked in the free run.

    • cites [9] Keely E, Williams R, Epstein G, Afkham A, Liddy C. Specialist perspectives on Ontario provincial electronic consultation services. Telemed J E Health. 2019;25:3–10. source

      Not checked in the free run.

    • cites [10] Joschko J, Liddy C, Moroz I, Reiche M, Crowe L, Afkham A, et al. Just a click away: exploring patients’ perspectives on receiving care through the Champlain BASE eConsult service. Fam Pract. 2018;35:350–5. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Two reviewers independently coded three transcripts using the initial codebook, meeting regularly to discuss discrepancies and revise the codebook.

    “Two reviewers (AM and IM) independently coded three transcripts using the initial codebook, meeting regularly to discuss any discrepancies and revise the codebook accordingly.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    In the Canadian setting, disparity of access to care and inappropriately long wait times are the most concerning issues that threaten patient safety.

    “In our own Canadian setting, disparity of access to care and inappropriately long wait times are the most concerning issues that threaten patient safety [23–26].”
    Quote in the paperSubstantial claim0 of 4 sources checked
    • cites [23] Barua B, Esmail N, Jackson T. The Effect of Wait Times on Mortality in Canada. Vancouver: Fraser Institute; 2014.

      Not checked in the free run.

    • cites [24] Institute for Healthcare Improvement. Closing the Loop: A Guide to Safer Ambulatory Referrals in the EHR Era. http://www.ihi.org/resources/Pages/Publications/Closing-the-Loop-A-Guide-to-Safer-Ambulatory-Referrals.aspx. Accessed 12 Oct 2019.

      Not checked in the free run.

    • cites [25] Kulkarni GS, Urbach DR, Austin PC, Fleshner NE, Laupacis A. Longer wait times increase overall mortality in patients with bladder cancer. J Urolog. 2009;182(4):1318–24. source

      Not checked in the free run.

    • cites [26] Canadian Medical Protective Association. Wait Times — a medical liability perspective. https://www.cmpa-acpm.ca/en/research-policy/public-policy/wait-times-a-medical-liability-perspective. Accessed 12 Oct 2019.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    In small group discussions, we were not able to determine the origin of specific quotations by participant type, limiting ability to identify differences in opinion between groups.

    “From a methodological standpoint, in small group discussions, we were not able to determine the origin of specific quotations in terms of the type of participant (e.g. patient, researcher, administrator, manager), limiting our ability to identify any differences in opinion between groups.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    Simply implementing a 'cookie-cutter' approach that worked in another region is unlikely to yield the same results without identifying local needs.

    “Simply implementing a ‘cookie-cutter’ approach that worked in another region is unlikely to yield the same results without this step.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Study design and conductreasoning

    The study had broad representation of different stakeholders across Canada, including patient partners, involved in data analysis, which allowed integration of multiple perspectives and strengthened quality of analysis and interpretation.

    “We had a broad representation of different stakeholders across Canada, including patient partners, involved in the data analysis, which allowed for the integration of multiple perspectives and strengthened the quality of the analysis and interpretation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Change champions have been typically described as important for moving new innovations through initiation, development and implementation, and play a key role in successful scale-up of health service innovations.

    “The concept of change champions is not new; they have been typically described as important for moving new innovations through the phases of initiation, development and implementation [27], thus playing a key role in the successful scale-up of health service innovations [28].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [27] Shaw EK, Howard J, West DR, Crabtree BF, Nease DE, Tutt B, et al. The role of the champion in primary care change efforts: from the State Networks of Colorado Ambulatory Practices and Partners (SNOCAP). J Am Board Fam Med. 2012;25:676–85. source

      Not checked in the free run.

    • cites [28] Simmons R, Fajans P, Ghiron L. Scaling Up Health Service Delivery: From Pilot Innovations to Policies and Programmes. Geneva: World Health Organization; 2007.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Innovative solutions must often work around structural barriers unless integrated into health system transformation at the outset.

    “Innovative solutions must often work around structural barriers unless integrated into the health system transformation at the outset.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Widespread dissemination of eConsult-like programmes will occur only when their positive impact is obvious and the quality of care delivery is acceptable to all populations, including vulnerable ones.

    “These objectives are in line with previous research suggesting that a widespread dissemination of eConsult-like programmes will occur only when their positive impact is obvious and the quality of care delivery is acceptable to all populations, including the vulnerable ones [31].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [31] Tuot DS, Liddy C, Vimalananda VG, Pecina J, Murphy EJ, Keely E, et al. Evaluating diverse electronic consultation programs with a common framework. BMC Health Serv Res. 2018;18:814. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Evaluation provides vital feedback to implementers and lets them identify what is working and what is not working.

    “Evaluation provides vital feedback to the individuals implementing the service and lets them clearly identify “how we can do a better job of monitoring and evaluating what’s working, what’s not working”.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Participants' personal biases and idiosyncrasies are limitations characteristic of qualitative research.

    “The limitations characteristic to the conduct of qualitative research must also be acknowledged, especially participants’ personal biases and idiosyncrasies.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisereasoning

    eConsult's successful spread and scale-up will depend on the collection and publication of data to comprehensively demonstrate its value in terms of reach, effectiveness, and adoption, and should include the consumer/patient perspective.

    “The authors further propose that eConsult’s successful spread and scale-up will depend on the collection and publication of data to comprehensively demonstrate its value in terms of reach, effectiveness, and adoption and should include the consumer/patient perspective.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [31] Tuot DS, Liddy C, Vimalananda VG, Pecina J, Murphy EJ, Keely E, et al. Evaluating diverse electronic consultation programs with a common framework. BMC Health Serv Res. 2018;18:814. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    We have conducted a study where we interviewed patients for whom eConsult was completed by their PCP about their experience.

    “However, we have conducted a study where we interviewed patients for whom eConsult was completed by their PCP about their experience [10].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [10] Joschko J, Liddy C, Moroz I, Reiche M, Crowe L, Afkham A, et al. Just a click away: exploring patients’ perspectives on receiving care through the Champlain BASE eConsult service. Fam Pract. 2018;35:350–5. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Innovations are not self-spreading or self-replicating; diffusion is an atypical outcome.

    “These points are especially important since it is well known that innovations are not self-spreading or self-replicating, and the existing evidence indicates that “diffusion is an atypical outcome” [2].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [2] Weil AR. Diffusion of Innovation. Health Aff. 2018;37:175. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The meeting attendees, working group, and analysts consisted of individuals interested in and supportive of national spread and scale-up of eConsult.

    “The meeting attendees, our working group and those who conducted the analysis consisted of individuals interested in and hence very supportive of the national spread and scale-up of eConsult.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationown data

    Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive spread and scale-up of innovation.

    “Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation.”
    Quote in the paperNot a substantial claim
    • Rests on an assumption: eConsult should be spread and scaled up across Canada.
    • Rests on an assumption: The views and experiences expressed by forum participants accurately reflect the factors that drive successful spread and scale-up of eConsult.

  • May combine two claims.

    Premisecitation

    A store-and-forward telemedicine tool created by Médecins Sans Frontières has been shown to improve the primary-specialty care interface in low-resource settings, allowing field doctors to obtain an expert opinion within a few hours.

    “Similarly, a store-and-forward telemedicine tool created by Médecins Sans Frontières has been shown to improve the primary-specialty care interface in low-resource settings, allowing field doctors to obtain an expert opinion within a few hours, wherever they are located in the world [22].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [22] Wootton R, Bonnardot L. Telemedicine in low-resource settings. Front Pub Health. 2015;3:3.

      Not checked in the free run.

55 of 55

  • The paper relies on this without support.

    Research on the spread and scale-up of healthcare innovations has been quite limited.

    “Although research in the area of spread and scale-up of healthcare innovations has been quite limited”

  • The paper relies on this without support.

    Communication between Canadian provinces is limited and there is little to no capacity for sharing or distribution of innovations.

    “Communication between provinces is limited and there is little to no capacity for sharing or distribution of innovations.”

  • The paper relies on this without support.

    Expanding healthcare innovations from the local to national level is a complex pursuit.

    “Expanding healthcare innovations from the local to national level is a complex pursuit, particularly for projects pertaining to eHealth services.”

  • The paper relies on this without support.

    eConsult should be spread and scaled up across Canada.

    “Efforts to promote innovation in healthcare are more likely to succeed if they are based on an understanding of the forces that drive the spread and scale-up of innovation.”

  • The paper relies on this without support.

    The views and experiences expressed by forum participants accurately reflect the factors that drive successful spread and scale-up of eConsult.

    “Our in-depth analysis of the discussion surrounding eConsult’s spread and scale-up revealed four themes that reflect strategies for successful spread and scale-up”

  • The paper relies on this without support.

    The four-point checklist derived from the themes is applicable to other healthcare innovators.

    “We recommend that other healthcare innovators base their spread and scale efforts on consideration of these factors as outlined in the checklist in Table 2.”
6 of 6

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Working group count does not match the listed attendees

    Minor

    The paper says the working group had 14 attendees, but lists 15 sets of initials.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Edwards N. Scaling-up Health Innovations and Interventions in Public Health: A Brief Review of the Current State-Of-The-Science. Ottawa: Canadian Institutes of Health Research; 2014.
  2. Barua B, Esmail N, Jackson T. The Effect of Wait Times on Mortality in Canada. Vancouver: Fraser Institute; 2014.
  3. Naylor D, Girard F, Mintz JM, Fraser N, Jenkins T, Power C, et al. Unleashing Innovation: Excellent Healthcare for Canada: Report of the Advisory Panel on Healthcare Innovation. Ottawa: Government of Canada; 2015.
  4. Bourgeault I, Turnbull S, Mulvale G, Ries N. Challenging Health System Leaders to Align Optimal Scopes of Practice and Innovative Care Models to Enable Health System Transformation in Canada. Toronto: CAHSPR Conference; 2014. https://www.cahspr.ca/en/presentation/5384b42f37dee8a72fd5018e. Accessed 12 Oct 2019.
  5. Olsen L, Aisner D, McGinnis JM, Institute of Medicine (US). Roundtable on Evidence-Based Medicine. The Learning Healthcare System: Workshop Summary. Washington, DC: National Academies Press; 2007.

Cited by

Sorted by citations

Related

Authors
Moroz, I., Archibald, D., Breton, M., Cote-Boileau, E., Crowe, L., Horsley, T., Hyseni, L., Johar, G., Keely, E., Burns, K. K., Kuziemsky, C., Laplante, J., Mihan, A., Oppenheimer, L., Sturge, D., Tuot, D. S., & Liddy, C.
Affiliations
  • University of Ottawa (CA)
  • Université de Sherbrooke (CA)
  • Royal College of Physicians and Surgeons of Canada (CA)
  • Alberta Health Services (CA)
  • MacEwan University (CA)
  • University of Manitoba (CA)
  • University of California, San Francisco (US)
Corresponding author
Liddy, Clare
ORCID
Venue
Health Research Policy and Systems
Publisher
Springer Science and Business Media LLC
Published
2020-06-03
Volume
18
Issue
1
Pages
57-57
ISSN
1478-4505 (ISSN-L)
DOI
10.1186/s12961-020-00574-0
PMC
PMC7268606
OpenAlex
W3030393818
Semantic Scholar
b245621bd83b2eccd867fb83cdf0a26530555c89
Cited by
24
References
26
Citations by year
2026: 1 · 2025: 3 · 2024: 3 · 2023: 6 · 2022: 7 · 2021: 3 · 2020: 1
FWCI
4.44
Type
journal article
Language
en
Open access
Licence
Locations
  • Health Research Policy and Systems (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • eScholarship (California Digital Library) (submittedVersion)
  • eScholarship (California Digital Library) (submittedVersion, cc-by)
  • Mspace (University of Manitoba) (submittedVersion)
  • uO Research (University of Ottawa) (submittedVersion, public-domain)
Funders
Canadian Institutes of Health Research (Health System Improvement Grant)
Topics
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
  • Social Sciences › Business, Management and Accounting › Organizational Behavior and Human Resource Management › Healthcare Systems and Technology
Keywords
e-consultation, diffusion of innovations, scale-up, healthcare innovation, stakeholder engagement, health policy, health care access, outcome measurement, learning health systems, Canada
MeSH terms
Stakeholder Participation, Canada, Communication, Delivery of Health Care, Diffusion of Innovation, Health Policy, Health Services Accessibility, Health Services Needs and Demand, Humans, Primary Health Care, Referral and Consultation, Research, Telemedicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review