CoChat
Sign Up
Cited by 1,011Electronic Health Records SystemsHealth Policy Implementation ScienceTelemedicine and Telehealth Implementation

Factors that influence the implementation of e-health: a systematic review of systematic reviews (an update)

Open access
Ross, Jamie, Stevenson, Fiona, Lau, Rosa, Murray, Elizabeth·2016·Implementation Science
DOI 10.1186/s13012-016-0510-7PMC PMC5080780
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: There is a significant potential for e-health to deliver cost-effective, quality health care, and spending on e-health systems by governments and healthcare systems is increasing worldwide. However, there remains a tension between the use of e-health in this way and implementation. Furthermore, the large body of reviews in the e-health implementation field, often based on one particular technology, setting or health condition make it difficult to access a comprehensive and comprehensible summary of available evidence to help plan and undertake implementation. This review provides an update and re-analysis of a systematic review of the e-health implementation literature culminating in a set of accessible and usable recommendations for anyone involved or interested in the implementation of e-health. METHODS: MEDLINE, EMBASE, CINAHL, PsycINFO and The Cochrane Library were searched for studies published between 2009 and 2014. Studies were included if they were systematic reviews of the implementation of e-health. Data from included studies were synthesised using the principles of meta-ethnography, and categorisation of the data was informed by the Consolidated Framework for Implementation Research (CFIR). RESULTS: Forty-four reviews mainly from North America and Europe were included. A range of e-health technologies including electronic medical records and clinical decision support systems were represented. Healthcare settings included primary care, secondary care and home care. Factors important for implementation were identified at the levels of the following: the individual e-health technology, the outer setting, the inner setting and the individual health professionals as well as the process of implementation. CONCLUSION: This systematic review of reviews provides a synthesis of the literature that both acknowledges the multi-level complexity of e-health implementation and provides an accessible and useful guide for those planning implementation. New interpretations of a large amount of data across e-health systems and healthcare settings have been generated and synthesised into a set of useable recommendations for practice. This review provides a further empirical test of the CFIR and identifies areas where additional research is necessary. TRIAL REGISTRATION: PROSPERO, CRD42015017661.

AI snapshot

Written from the paper’s full text.

TL;DR

This updated systematic review of reviews synthesises 44 reviews on e-health implementation and finds that implementation is multi-level, involving the technology, outer setting, inner setting, individual health professionals, and implementation process. The findings are well described by the Consolidated Framework for Implementation Research (CFIR), with no data that could not be coded to its constructs.

Key findings

  • From 2,812 unique citations, 118 full papers were assessed and 44 reviews published between 2003 and 2014 were included; 28 focused on one e-health domain, 16 covered multiple domains, 12 focused on one healthcare setting, and 21 covered two or more settings.
  • Factors important for implementation were identified at the levels of the individual e-health technology, outer setting, inner setting, individual health professionals, and implementation process; no single factor was identified as the key barrier or facilitator.
  • Cost was reported as an important implementation factor by the majority of studies across all e-health domains, and workflow compatibility/fit was discussed by the majority of studies.
  • No data could not be coded to a CFIR construct; areas with no supporting data included peer pressure, tension for change, relative priority, learning climate, external change agents, and executing.

Methods

  • Systematic review of reviews; MEDLINE, EMBASE, CINAHL, PsycINFO, and The Cochrane Library were searched, replicating the original review’s strategy, covering literature published between 1995 and 2014.
  • Included studies were systematic reviews, narrative reviews, qualitative meta-syntheses, or meta-ethnographies of e-health implementation; data were extracted and synthesised using principles of meta-ethnography with CFIR as the organising framework.
  • Methodological quality was described using ENTREQ guidelines but studies were not excluded based on quality assessment.

Limitations

  • The paper states that systematic reviews and included studies may be subject to publication bias; reviews of reviews have a time-lag; the review depended on interpretations provided by authors of included reviews; it was often unclear whether data came from primary studies or review authors’ interpretations; many reviews did not specify whether data came from clinicians, nurses, other staff, or multidisciplinary teams; and synthesis was limited by what previous reviewers included, so unaddressed CFIR areas may be unimportant or merely unaddressed.
  • Evident from the text: many included reviews were methodologically poor against ENTREQ, including three that did not give databases searched, seven that searched only one database or source, sixteen that did not specify inclusion/exclusion criteria, and five that did not detail the number of primary studies included.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1186/s13012-016-0510-7

Cites

62 of 62 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 148 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    Sixteen of the 44 included studies did not specify the criteria for inclusion or exclusion.

    “Sixteen of the 44 studies did not specify the criteria for inclusion or exclusion [16, 17, 20, 22, 30, 32–36, 38–40, 48, 53, 56].”
    Quote in the paperNot a substantial claim
  • Findingcitation

    The cost of e-health systems and costs associated with their implementation were reported as important implementation factors by the majority of studies across all e-health domains.

    “The cost of e-health system and costs associated with their implementation were reported as important implementation factors by the majority of studies across all e-health domains [15–19, 22–25, 27, 28, 30–33, 36, 38, 39, 41, 43, 47, 49, 51–54, 57]”
    Quote in the paperSubstantial claim
  • Findingown data

    Factors important for implementation were identified at the levels of the individual e-health technology, the outer setting, the inner setting, the individual health professionals, and the process of implementation.

    “Factors important for implementation were identified at the levels of the following: the individual e-health technology, the outer setting, the inner setting and the individual health professionals as well as the process of implementation.”
    Quote in the paperSubstantial claim
  • Findingcitation

    Some studies cited cost-related factors as the main barrier to implementation.

    “with some studies citing cost-related factors as the main barrier to implementation [49, 54].”
    Quote in the paperNot a substantial claim
  • Findingown data

    Of the 118 full papers assessed, 44 met the criteria for inclusion.

    “Of the 118 full papers assessed, 44 [15–58] met the criteria for inclusion and were included in this review”
    Quote in the paperNot a substantial claim
  • Classified as probably not a substantial claim. 46
  • May combine two claims. 46

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingnone given

    Cost factors are related to start-up costs, ongoing costs, costs related to a loss of revenue, and potential savings to put against these costs.

    “Cost factors are related to start-up costs, ongoing costs, costs related to a loss of revenue and potential savings to put against these costs.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Sixteen included studies focused on e-health technologies from across a range of e-health domains.

    “Sixteen studies focussed on e-health technologies from across a range of e-health domains [15, 23, 25, 26, 30, 35, 37, 46, 47, 49, 50, 52, 53, 56–58].”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    All included studies were published between 2003 and 2014.

    “All studies were published between 2003 and 2014.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Twelve included studies were focused on one particular type of healthcare setting.

    “Twelve studies were focussed on one particular type of healthcare setting including primary care/ambulatory [15, 24, 27, 28, 36, 39], hospital/inpatient/acute care [22, 29, 40, 46] and home care [19, 53].”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Financial incentives for e-health adoption include initial funds to cover upfront costs, financial sponsorship, reimbursements for adoption, and pay-for-performance initiatives.

    “Financial incentives include the provision of initial funds to cover upfront costs [49], financial sponsorship [39], reimbursements for adoption [19, 30, 31, 36], and pay-for-performance initiatives [24, 36, 37, 49].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [49] Police RL, Foster T, Wong KS. Adoption and use of health information technology in physician practice organisations: systematic review. Inform Prim Care. 2010;18(4):245–58.

      Not checked in the free run.

    • cites [39] Ludwick DA, Doucette J. Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries. Int J Med Inform. 2009;78(1):22–31. source

      Not checked in the free run.

    • cites [19] Botsis T, Hartvigsen G. Current status and future perspectives in telecare for elderly people suffering from chronic diseases. J Telemed Telecare. 2008;14(4):195–203. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    All included papers were written in English.

    “All papers were written in English.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The remainder of the included studies did not define a specific healthcare setting.

    “the remainder of the studies did not define a specific healthcare setting [16, 18, 20, 28, 34, 38, 48, 57, 58].”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    The included reviews originated mainly from North America and Europe.

    “Forty-four reviews mainly from North America and Europe were included.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    Twenty-one included studies focused on the implementation of e-health into two or more types of healthcare setting.

    “Twenty-one studies focussed on the implementation of e-health in to two or more types of healthcare setting [21, 23, 25, 26, 30, 31, 33, 35, 37, 41–45, 47, 49, 50, 52, 54–56]; the remainder of the studies did not define a specific healthcare setting [16, 18, 20, 28, 34, 38, 48, 57, 58].”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Fifteen included studies originated from Canada; fourteen from the USA; three from the Netherlands; three from the UK; two from Australia; and one each from Germany, Sweden, Norway, Mexico, Malaysia, Kenya and Israel.

    “Fifteen studies originated from Canada [16, 25–29, 31–33, 36, 39, 41, 42, 44, 53], fourteen from the USA [17, 22–24, 30, 35, 38, 40, 49, 52, 54–57], three from the Netherlands [17, 20, 34], three from the UK [37, 46, 51], two from Australia [15, 43] and one each from Germany [47], Sweden [50], Norway [19], Mexico [21], Malaysia [58], Kenya [45] and Israel [48].”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Findingown data

    The relationships between studies seemed to be reciprocal, with many themes occurring across studies largely in agreement about the factors that influenced implementation, and from which a line of argument could be developed.

    “It became clear that the relationships between studies seemed to be reciprocal (where concepts of one study encompass another), with many themes occurring across studies which were largely in agreement about the factors that influenced implementation, and from which a line of argument (overarching narrative) could be developed.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    A total of 2812 unique citations were identified.

    “A total of 2812 unique citations were identified (see Fig. 1).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Complexity factors such as slow system performance, software and hardware that were difficult to use, the need for extensive software modifications, the work involved in transferring records between two systems, the inability to provide real-time access, data handling, reliability, slow speed, unplanned downtime, and connectivity issues influence implementation of systems in healthcare settings.

    “Complexity factors such as slow system performance [55], software and hardware that were difficult to use [27], the need for extensive software modifications [52], the work involved in transferring records between two systems [18], the inability to provide real-time access [38], data handling, reliability, slow speed, unplanned downtime [41] and connectivity issues [49] influence implementation of”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    There were no data that could not be coded to one of the CFIR constructs, meaning the categories of the CFIR were endorsed and no additional original theoretical insights developed.

    “There were no data that could not be coded to one of the CFIR constructs, meaning the categories of the CFIR were endorsed and no additional original theoretical insights developed.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingown data

    Of 2812 unique citations, 2694 were excluded on the basis of title or abstract, leaving 118 citations requiring full paper assessment.

    “Of these, 2694 could be excluded on the basis of the title or abstract, leaving 118 citations requiring the full paper before a decision could be made.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    Access to ongoing support to use systems was important for system use and a barrier to implementation when lacking.

    “Access to ongoing support to use systems was important for system use [21, 25, 28, 31, 37, 39, 43, 54] and a barrier to implementation when it was lacking [21].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [21] Castillo VH, Martinez-Garcia AI, Pulido JR. A knowledge-based taxonomy of critical factors for adopting electronic health record systems by physicians: a systematic literature review. BMC Med Inform Decis Mak. 2010;10:60. source

      Not checked in the free run.

    • cites [25] Gagnon MP, Desmartis M, Labrecque M, Car J, Pagliari C, Pluye P, et al. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. J Med Syst. 2012;36(1):241–77. source

      Not checked in the free run.

    • cites [28] Goldstein DH, Phelan R, Wilson R, Ross-White A, VanDenKerkhof EG, Penning JP, et al. Brief review: adoption of electronic medical records to enhance acute pain management. Can J Anesth. 2014;61(2):164–79. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Providing a period of transition in which end users can become familiar with and learn how to use new systems has been advocated.

    “Providing a period of transition in which end users can become familiar with and learn how to use new systems has been advocated [39, 50].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [39] Ludwick DA, Doucette J. Adopting electronic medical records in primary care: lessons learned from health information systems implementation experience in seven countries. Int J Med Inform. 2009;78(1):22–31. source

      Not checked in the free run.

    • cites [50] Rahimi B, Vimarlund V, Timpka T. Health information system implementation: a qualitative meta-analysis. J Med Syst. 2009;33(5):359–68. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    Positive staff attitudes were described as beliefs that new systems would benefit patients, interest in technologies, perceived usefulness, and motivation in working with the systems.

    “Positive staff attitudes were described as: beliefs that the new systems would benefit patients [56], interest in the technologies, perceived usefulness and motivation in working with the systems [21].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [56] Waneka R, Spetz J. Hospital information technology systems' impact on nurses and nursing care. J Nurs Adm. 2010;40(12):509–14. source

      Not checked in the free run.

    • cites [21] Castillo VH, Martinez-Garcia AI, Pulido JR. A knowledge-based taxonomy of critical factors for adopting electronic health record systems by physicians: a systematic literature review. BMC Med Inform Decis Mak. 2010;10:60. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Access to appropriate, high-quality, well-funded, and easily available training was reported as a facilitator to implementation, whereas non-existent or inadequate training was reported as a barrier.

    “Generally, access to appropriate, high-quality, well-funded, and easily available training was reported as a facilitator to implementation, whereas it was reported as a barrier when it was non-existent or existent but inadequate [15, 16, 18, 20, 23, 25, 28, 31, 37, 43, 45, 50, 51, 53–58].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [15] Adaji A, Schattner P, Jones K. The use of information technology to enhance diabetes management in primary care: a literature review. Inform Prim Care. 2008;16(3):229–37.

      Not checked in the free run.

    • cites [16] Archer N, Fevrier-Thomas U, Lokker C, McKibbon KA, Straus SE. Personal health records: a scoping review. J Am Med Inform Assoc. 2011;18(4):515–22. source

      Not checked in the free run.

    • cites [18] Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Findingcitation

    Quality of project management, careful study of downstream effects on workflow, additional training, adaptability of technologies to fit with roles/tasks/workflows, and dedicated technical support staff are suggested as strategies to reduce barriers related to disruptions to workflow, roles and responsibilities.

    “The quality of project management during the implementation period [18], careful study of the downstream effects of implementation on workflow [55], additional training [31, 55], the adaptability of technologies to fit with roles, tasks and workflows [37] and dedicated technical support staff [31] are suggested as strategies to reduce barriers related to disruptions to workflow, roles and respons”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [18] Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. source

      Not checked in the free run.

    • cites [55] Vreeman DJ, Taggard SL, Rhine MD, Worrell TW. Evidence for electronic health record systems in physical therapy. Phys Ther. 2006;86(3):434–46. source

      Not checked in the free run.

    • cites [31] Jarvis-Selinger S, Chan E, Payne R, Plohman K, Ho K. Clinical telehealth across the disciplines: lessons learned. Telemed e-Health. 2008;14(7):720–5. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    A good fit, or perceived fit, between e-health systems and workflows, and systems positively influencing workplace efficiency, facilitates use.

    “When there is a good fit, or perceived fit, between e-health systems and workflows, and when systems positively influence workplace efficiency, this facilitates use [15, 40, 41].”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [15] Adaji A, Schattner P, Jones K. The use of information technology to enhance diabetes management in primary care: a literature review. Inform Prim Care. 2008;16(3):229–37.

      Not checked in the free run.

    • cites [40] Mack EH, Wheeler DS, Embi PJ. Clinical decision support systems in the pediatric intensive care unit. Pediatric Crit Care Med. 2009;10(1):23–8. source

      Not checked in the free run.

    • cites [41] McGinn CA, Grenier S, Duplantie J, Shaw N, Sicotte C, Mathieu L, et al. Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med. 2011;9:46. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Incorporating workflow analysis into system design, integrating systems into the usual process of care, user-friendly systems, and minimising workflow interruptions during implementation may minimise disruptions to workflow.

    “Incorporating workflow analysis into system design [48, 55], the integration of systems into the usual process of care [15], user-friendly systems [40] and minimising workflow interruptions during implementation [17] may minimise disruptions to workflow.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [48] Peleg M, Tu S. Decision support, knowledge representation and management in medicine. Yearbook Med Inform. 2006:72–80.

      Not checked in the free run.

    • cites [55] Vreeman DJ, Taggard SL, Rhine MD, Worrell TW. Evidence for electronic health record systems in physical therapy. Phys Ther. 2006;86(3):434–46. source

      Not checked in the free run.

    • cites [15] Adaji A, Schattner P, Jones K. The use of information technology to enhance diabetes management in primary care: a literature review. Inform Prim Care. 2008;16(3):229–37.

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Infrastructure features important for implementation include electricity supply, available bandwidth, reliable internet connectivity, access to computers, electrical power, and access to phone lines and mobile phones.

    “Infrastructure features included electricity supply [51], available bandwidth [31, 51], access to reliable internet connectivity [45, 51], access to computers [34], electrical power [45] and access to phone lines and mobile phones [45].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [51] Saliba V, Legido-Quigley H, Hallik R, Aaviksoo A, Car J, McKee M. Telemedicine across borders: a systematic review of factors that hinder or support implementation. Int J Med Inform. 2012;81(12):793–809. source

      Not checked in the free run.

    • cites [31] Jarvis-Selinger S, Chan E, Payne R, Plohman K, Ho K. Clinical telehealth across the disciplines: lessons learned. Telemed e-Health. 2008;14(7):720–5. source

      Not checked in the free run.

    • cites [45] Oluoch T, Santas X, Kwaro D, Were M, Biondich P, Bailey C, et al. The effect of electronic medical record-based clinical decision support on HIV care in resource-constrained settings: a systematic review. Int J Med Inform. 2012;81(10):e83–92. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingcitation

    One review emphasised the need for ongoing effort after the initial 'go-live' phase, referring to the 'under-recognised maintenance phase of implementation'.

    “One review emphasised the need for ongoing effort after the initial ‘go-live’ phase, referring to the ‘under-recognised maintenance phase of implementation’ [29].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [29] Gruber D, Cummings GG, Leblanc L, Smith DL. Factors influencing outcomes of clinical information systems implementation: a systematic review. Comput Inform Nurs. 2009;27(3):151–63. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Negative perceptions included beliefs that electronic systems would disrupt care delivery, doubts that systems can improve patient care/clinical outcomes/quality of medical practices, distrust in systems, and general staff resistance to change.

    “Negative perceptions included beliefs that electronic systems would disrupt the delivery of care [49]; doubts that these systems can improve patient care, clinical outcomes or improve the quality of medical practices [43]; and distrust in the systems [37] as well as a more general staff resistance to change [28, 32, 39, 41, 49, 51–53].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [49] Police RL, Foster T, Wong KS. Adoption and use of health information technology in physician practice organisations: systematic review. Inform Prim Care. 2010;18(4):245–58.

      Not checked in the free run.

    • cites [43] Moxey A, Robertson J, Newby D, Hains I, Williamson M, Pearson SA. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc. 2010;17(1):25–33. source

      Not checked in the free run.

    • cites [37] Lluch M. Healthcare professionals' organisational barriers to health information technologies-a literature review. Int J Med Inform. 2011;80(12):849–62. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The synthesis showed that findings were remarkably consistent across different e-health domains and healthcare settings and well described by the CFIR framework, with no data that did not fit the CFIR.

    “The synthesis showed that findings were remarkably consistent across different e-health domains and healthcare settings and well described by the CFIR framework, with no data that did not fit the CFIR.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Findingcitation

    The work of planning includes delineation of roles and responsibilities, securing time for system selection and procurement, evaluating concomitant policy and process changes, needs assessment and analysis, development of a business plan, early identification and engagement of champions, involving end users, establishing a guiding philosophy, testing organisational readiness, development of incent

    “The work of planning includes the delineation of roles and responsibilities [43], securing time to invest system selection and procurement [18], evaluating other concomitant policy and process changes [40], needs assessment and analysis, development of a business plan [44], early identification and engagement of champions [31], involving end users [16, 27, 41, 51], establishing a guiding”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [43] Moxey A, Robertson J, Newby D, Hains I, Williamson M, Pearson SA. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc. 2010;17(1):25–33. source

      Not checked in the free run.

    • cites [18] Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. source

      Not checked in the free run.

    • cites [40] Mack EH, Wheeler DS, Embi PJ. Clinical decision support systems in the pediatric intensive care unit. Pediatric Crit Care Med. 2009;10(1):23–8. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Fears over loss of autonomy, concerns about liability, concerns over patient privacy and security being compromised, and perceived threats to patient and health professional relationships through the introduction of e-health systems were repeatedly reported as barriers to use.

    “Specifically, fears over a loss of autonomy [16, 18, 27, 34, 37, 39, 43], concerns about liability [16, 18, 22, 37, 43, 51], concerns over patient privacy and security being compromised [16, 18, 20, 23, 24, 27, 28, 30, 31, 33, 39, 41, 43, 45, 47, 49, 51–53, 56, 57], and perceived threats to patient and health professional relationships [18, 25, 27, 28, 31, 34, 37–39, 41, 43, 54, 57] through the”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [16] Archer N, Fevrier-Thomas U, Lokker C, McKibbon KA, Straus SE. Personal health records: a scoping review. J Am Med Inform Assoc. 2011;18(4):515–22. source

      Not checked in the free run.

    • cites [18] Boonstra A, Broekhuis M. Barriers to the acceptance of electronic medical records by physicians from systematic review to taxonomy and interventions. BMC Health Serv Res. 2010;10:231. source

      Not checked in the free run.

    • cites [27] Gagnon M-P, Nsangou E-R, Payne-Gagnon J, Grenier S, Sicotte C. Barriers and facilitators to implementing electronic prescription: a systematic review of user groups' perceptions. J Am Med Inform Assoc. 2014;21(3):535–41. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Strategies to challenge negative attitudes include fostering a culture of communication and cooperation, involving eventual users in development and implementation, leadership, friendly and context-aware user interfaces, better education, and clearly communicating intended benefits and realistic expectations.

    “Strategies to challenge negative attitudes included fostering a culture of communication and cooperation, involving the eventual users of systems in the development and implementation [20, 49], leadership [56], friendly and context-aware user interfaces which promote perceived ease of use and usefulness [21], better education [47], and clearly and prospectively communicating intended benefits and”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [20] Broens TH, Huis in't Veld RM, Vollenbroek-Hutten MM, Hermens HJ, van Halteren AT, Nieuwenhuis LJ. Determinants of successful telemedicine implementations: a literature study. J Telemed Telecare. 2007;13(6):303–9. source

      Not checked in the free run.

    • cites [49] Police RL, Foster T, Wong KS. Adoption and use of health information technology in physician practice organisations: systematic review. Inform Prim Care. 2010;18(4):245–58.

      Not checked in the free run.

    • cites [56] Waneka R, Spetz J. Hospital information technology systems' impact on nurses and nursing care. J Nurs Adm. 2010;40(12):509–14. source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    All included reviews reported multiple factors that were important for implementation, and no single factor was identified as a key barrier or facilitator.

    “All the included reviews reported multiple factors that were important for implementation, and no single factor was identified as a key barrier or facilitator.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    The attitudes of colleagues and patients were reported to influence staff attitudes with regard to e-health acceptance, as were staff demographic factors.

    “The attitudes of colleagues [25, 41] and patients [25, 27, 41] were also reported to influence staff attitudes with regard to e-health acceptance as were staff demographic factors [23, 46, 56].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [25] Gagnon MP, Desmartis M, Labrecque M, Car J, Pagliari C, Pluye P, et al. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. J Med Syst. 2012;36(1):241–77. source

      Not checked in the free run.

    • cites [41] McGinn CA, Grenier S, Duplantie J, Shaw N, Sicotte C, Mathieu L, et al. Comparison of user groups’ perspectives of barriers and facilitators to implementing electronic health records: a systematic review. BMC Med. 2011;9:46. source

      Not checked in the free run.

    • cites [27] Gagnon M-P, Nsangou E-R, Payne-Gagnon J, Grenier S, Sicotte C. Barriers and facilitators to implementing electronic prescription: a systematic review of user groups' perceptions. J Am Med Inform Assoc. 2014;21(3):535–41. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Training and financial incentives were cited as strategies to overcome skill-related barriers.

    “Training [25, 37, 43] and financial incentives [37] were cited as strategies to overcome skill-related barriers.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [25] Gagnon MP, Desmartis M, Labrecque M, Car J, Pagliari C, Pluye P, et al. Systematic review of factors influencing the adoption of information and communication technologies by healthcare professionals. J Med Syst. 2012;36(1):241–77. source

      Not checked in the free run.

    • cites [37] Lluch M. Healthcare professionals' organisational barriers to health information technologies-a literature review. Int J Med Inform. 2011;80(12):849–62. source

      Not checked in the free run.

    • cites [43] Moxey A, Robertson J, Newby D, Hains I, Williamson M, Pearson SA. Computerized clinical decision support for prescribing: provision does not guarantee uptake. J Am Med Inform Assoc. 2010;17(1):25–33. source

      Not checked in the free run.

  • May combine two claims.

    Findingcitation

    Evaluation was seen as important to ensure system benefits, to increase health professional acceptance through demonstration of benefits, and to secure ongoing funding.

    “Evaluation was seen as important to ensure system benefits [29, 48], to increase health professional acceptance through demonstration of benefits [31, 34, 41, 43] and to secure ongoing funding [53],”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [29] Gruber D, Cummings GG, Leblanc L, Smith DL. Factors influencing outcomes of clinical information systems implementation: a systematic review. Comput Inform Nurs. 2009;27(3):151–63. source

      Not checked in the free run.

    • cites [48] Peleg M, Tu S. Decision support, knowledge representation and management in medicine. Yearbook Med Inform. 2006:72–80.

      Not checked in the free run.

    • cites [31] Jarvis-Selinger S, Chan E, Payne R, Plohman K, Ho K. Clinical telehealth across the disciplines: lessons learned. Telemed e-Health. 2008;14(7):720–5. source

      Not checked in the free run.

  • May combine two claims.

    Findingreasoning

    Key factors for effective implementation included outer context, in particular, the need for supportive legislation, and recognised standards.

    “Key factors for effective implementation included outer context, in particular, the need for supportive legislation, and recognised standards.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Four reviews included data on the effects of the e-health systems implemented; these reported mixed effects.

    “Four reviews included data on the effects of the e-health systems implemented [24, 36, 49, 52]; these reported mixed effects.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [24] Fontaine P, Ross SE, Zink T, Schilling LM. Systematic review of health information exchange in primary care practices. J Am Board Fam Med. 2010;23(5):655–70. source

      Not checked in the free run.

    • cites [36] Lau F, Price M, Boyd J, Partridge C, Bell H, Raworth R. Impact of electronic medical record on physician practice in office settings: a systematic review. BMC Med Inform Decis Mak. 2012;12:10. source

      Not checked in the free run.

    • cites [49] Police RL, Foster T, Wong KS. Adoption and use of health information technology in physician practice organisations: systematic review. Inform Prim Care. 2010;18(4):245–58.

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    The use of the CFIR highlighted that the individual e-health technology, the outer setting, the inner setting, the individual health professionals, and the process of implementation are all important for implementation and should be considered carefully when attempts are made to implement e-health into health systems.

    “The use of the CFIR highlighted that the individual e-health technology, the outer setting, the inner setting, the individual health professionals, and the process of implementation are all important for implementation and should be considered carefully when attempts are made to implement e-health into health systems.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingcitation

    Both reviews highlighted the importance of policies and incentives; adequate infrastructure and resources; engagement of key personnel; organisational readiness; individuals' knowledge and beliefs; and the fit of innovations with workflows, processes and systems.

    “Both reviews highlighted the importance of policies and incentives; adequate infrastructure and resources; engagement of key personnel; organisational readiness; individuals’ knowledge and beliefs; and the fit of innovations with workflows, processes and systems.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [60] Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11(1):1–39.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    This systematic review is reported following the ENTREQ statement guidelines to enhance transparency in reporting qualitative evidence synthesis.

    “This systematic review is reported following the ENTREQ statement guidelines to enhance transparency in reporting qualitative evidence synthesis [11].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12:181. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The protocol for this systematic review has been published and registered with PROSPERO (CRD42015017661).

    “The protocol for this systematic review has been published [9] and registered with the Prospective Register of Systematic Reviews (registration number CRD42015017661).”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [9] Ross J, Stevenson F, Lau R, Murray E. Exploring the challenges of implementing e-health: a protocol for an update of a systematic review of reviews. BMJ Open. 2015;5(4):e006773. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The CFIR provides a systematic way of identifying factors important for implementation and allows identification of areas where evidence is insufficient and further research is required.

    “The CFIR provides a systematic way of identifying the factors that are important for implementation, and its use also allows identification of areas where there is insufficient evidence and further research is required.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. source

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    High-profile e-health implementation failures continue to be reported, such as the failure of implementation of an e-health system in a major UK teaching hospital, leading to reduced performance, demoralised staff, costs of £200 million and the trust being put into special measures.

    “High-profile implementation failures continue to be reported, such as the failure of implementation of an e-health system in a major UK teaching hospital, leading to reduced performance, demoralised staff, costs of £200 million and the trust being put into special measures [6].”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [6] Computing. Addenbrooke's Hospital £200m IT system proves an epic fail. 2015.  http://www.computing.co.uk/ctg/news/2427100/addenbrookes-hospital-gbp200m-it-system-proves-an-epic-fail . Accessed 17 Jun 2016.

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Although there is widespread agreement about the importance and potential benefits of e-health, realisation of these benefits has often been slower than anticipated, often because of difficulties with implementation.

    “However, although there is widespread agreement about the importance and potential benefits of e-health, realisation of these benefits has often been slower than anticipated, often because of difficulties with implementation [4].”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [4] Wachter RM. Making IT Work: Harnessing the power of health information technology to improve care in England. Report of the National Advisory Group on Health Information Technology in England. 2016.  https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/550866/Wachter_Review_Accessible.pdf . Accessed 27 Sept 2016.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    With many reviews on the implementation of different e-health technologies available, it may be difficult for clinicians, managers or policymakers to locate and apply an appropriate body of evidence for their specific circumstances.

    “With a plethora of reviews on the implementation of different e-health technologies available, it may be difficult for clinicians, managers or policymakers to locate and apply an appropriate body of evidence for their specific circumstances.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    A systematic review of reviews provides a summary of evidence from a variety of different levels, including the combination of different interventions, different populations and different settings, in a coherent and economical way.

    “A systematic review of reviews provides a summary of evidence from a variety of different levels, including the combination of different interventions, different populations and different settings [10] in a coherent and economical way [7].”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [10] Smith V, Devane D, Begley C, Clarke M. Methodology in conducting a systematic review of systematic reviews of healthcare interventions. BMC Med Res Methodol. 2011;11(1):15. source

      Not checked in the free run.

    • cites [7] Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012;90(5):357–64. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    There was no limitation of language in the search.

    “There was no limitation of language.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    The original review was based on 37 papers published between 1995 and 31 July 2009.

    “The original review [7] was based on 37 papers published between 1995 and 31 July 2009.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012;90(5):357–64. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    Studies were not excluded based on methodological quality assessment because the aim was to describe and synthesise qualitative literature and not determine an effect size.

    “Because the aim was to describe and synthesise a body of qualitative literature and not determine an effect size, studies were not excluded based on this assessment.”
    Quote in the paperSubstantial claim
    • Rests on an assumption: The methodological limitations of the included reviews do not prevent a valid synthesis of their findings.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    The CFIR consolidates implementation factors from a broad array of implementation theories and is composed of five major constructs made up of components that influence the implementation of innovations into practice.

    “The CFIR [8], consolidates implementation factors from a broad array of implementation theories and is composed of five major constructs made up of components that influence the implementation of innovations into practice (see Table 2).”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [8] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Any discrepancies between reviewers were resolved through discussion.

    “Any discrepancies between reviewers were resolved through discussion.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Full-text papers of potentially eligible references were assessed for eligibility against pre-specified selection criteria.

    “Full-text papers of references that were deemed potentially eligible were obtained and assessed for eligibility against the pre-specified selection criteria.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Data were re-categorised from one construct to another and discussions between JR, EM and FS were held until the reviewers were confident that all data were coded into appropriate constructs or categories.

    “Data were re-categorised from one construct to another and discussions between JR, EM and FS were held until the reviewers were confident that all data were coded into appropriate constructs or categories.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    Meta-ethnography focusses on interpretation to synthesise understanding, unlike other approaches to qualitative synthesis such as thematic analysis, which seek to summarise data.

    “Meta-ethnography, as described in depth by Noblit and Hare [13], focusses on interpretation to ‘synthesise understanding’, unlike other approaches to qualitative synthesis, such as thematic analysis, which seek to summarise data [14].”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [13] Noblit GW, Hare RD. Meta-ethnography: synthesizing qualitative studies. Newbury Park: Sage;1988. source

      Not checked in the free run.

    • cites [14] Bearman M, Dawson P. Qualitative synthesis and systematic review in health professions education. Med Educ. 2013;47(3):252–60. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Reference lists of all included studies were screened for additional literature.

    “Reference lists of all included studies were also screened for additional literature.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Interpretationreasoning

    Data from discussions were included because they often contained further interpretations from the authors, which offered important insights and enhanced the richness of the findings.

    “Data from discussions were included as they often contained further interpretations from the authors, which offered important insights and enhanced the richness of the findings.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Titles and abstracts of all identified records were independently assessed by JR and RL.

    “Titles and abstracts of all identified records were independently assessed by JR and RL.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Data were extracted from included studies by JR, and data extraction and coding was checked by EM.

    “Data were extracted from the included studies by JR, and data extraction and coding was checked by EM.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    Data on the methodological quality of the included reviews was extracted based on ENTREQ statement guidelines and used to describe the quality of the included reviews.

    “Data on the methodological quality of the included reviews was extracted based on ENTREQ statement guidelines [11] and was used to describe the quality of the included reviews.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [11] Tong A, Flemming K, McInnes E, Oliver S, Craig J. Enhancing transparency in reporting the synthesis of qualitative research: ENTREQ. BMC Med Res Methodol. 2012;12:181. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Comprehensive electronic searches of MEDLINE, EMBASE, CINAHL, PsycINFO and The Cochrane Library were conducted.

    “Comprehensive electronic searches of MEDLINE, EMBASE, CINAHL, PsycINFO and The Cochrane Library were conducted.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Premisecitation

    The use of a framework like the CFIR aids the transferability and comparability of findings from this review to other implementation studies and allows those undertaking implementation to access the parts of the review that are of most interest to them.

    “The use of a framework like the CFIR aids the transferability and comparability of findings from this review to other implementation studies and allows those undertaking implementation to access the parts of this review that are of most interest to them.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [8] Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50. source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Implementation climate includes the compatibility or general fit between the e-health intervention and the organisation.

    “Implementation climate includes the compatibility or general fit between the e-health intervention and the organisation [16, 23, 31, 33, 37, 46, 49, 59].”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [16] Archer N, Fevrier-Thomas U, Lokker C, McKibbon KA, Straus SE. Personal health records: a scoping review. J Am Med Inform Assoc. 2011;18(4):515–22. source

      Not checked in the free run.

    • cites [23] Fitzpatrick L, Melnikas A, Weathers M, Kachnowski S. Understanding communication capacity. Communication patterns and ICT usage in clinical settings. J Healthc Inf Manag. 2007;22(3):34–41.

      Not checked in the free run.

    • cites [31] Jarvis-Selinger S, Chan E, Payne R, Plohman K, Ho K. Clinical telehealth across the disciplines: lessons learned. Telemed e-Health. 2008;14(7):720–5. source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Interpretationreasoning

    Separate reviews are brought together, compared and contrasted, which allows for new insights to be generated across the literature and synthesised into a simple overview of a large body of work.

    “Separate reviews are brought together, compared and contrasted, which allows for new insights to be generated across the literature and synthesised into a simple overview of a large body of work.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    Issues around implementation are multi-level and complex.

    “Findings suggest that issues around implementation are multi-level and complex.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Lau et al.'s review found that the perceived benefits or harm of implementation such as expectations of more efficient workflow or lower productivity were only an important factor for the implementation of e-health interventions and was not present in the data for other types of interventions.

    “Lau et al.’s review found that the perceived benefits or harm of implementation such as expectations of more efficient workflow or lower productivity were only an important factor for the implementation of e-health interventions and was not present in the data for other types of interventions (guidelines or evidence-based practice, management of care, public health or preventative medicine, integ”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [60] Lau R, Stevenson F, Ong BN, Dziedzic K, Treweek S, Eldridge S, et al. Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews. Implement Sci. 2016;11(1):1–39.

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    This review was rigorous, carefully executed and employed a robust methodological approach.

    “Although this review was rigorous, carefully executed and employed a robust methodological approach, it has several limitations.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    The data available for the synthesis was limited by what previous reviewers included in their reviews, and therefore, it is not possible to tell whether the areas of the CFIR which are not addressed are not important or just not addressed.

    “Finally, the data available for the synthesis was limited by what previous reviewers included in their reviews, and therefore, it is not possible to tell whether the areas of the CFIR which are not addressed are not important or just not addressed.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The original review reported a concentration on organisational issues within the literature.

    “The original review reported a concentration on organisational issues within the literature.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012;90(5):357–64. source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    The use of the CFIR sensitised the focus of analysis to other factors as well including factors related to the innovation, outer context, individuals and the process of implementation.

    “Although also strongly present in this review, the use of the CFIR sensitised the focus of analysis to other factors as well including factors related to the innovation, outer context, individuals and the process of implementation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    It was often not clear whether the data came from the primary studies or were subsequent interpretations by the authors of included reviews.

    “It was often not clear whether the data came from the primary studies or were subsequent interpretations by the authors of included reviews.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductreasoning

    The use of a meta-ethnographic approach provided a rigorous and transparent approach to the data analysis and the translation of the studies into one another, which allowed the development of an overarching narrative which endorsed the value of the CFIR framework.

    “The use of a meta-ethnographic approach provided a rigorous and transparent approach to the data analysis and the translation of the studies into one another, which allowed the development of an overarching narrative which endorsed the value of the CFIR framework.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    In this review, there was a focus given to the role of reflecting and evaluating which may represent an increased awareness of their importance in implementation.

    “In this review, there was a focus given to the role of reflecting and evaluating which may represent an increased awareness of their importance in implementation.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Interpretationreasoning

    Reviews of reviews inevitably result in a time-lag, as new primary data must first be published, then included in a review and then into a review of reviews.

    “Reviews of reviews inevitably result in a time-lag, as new primary data must first be published, then included in a review and then into a review of reviews.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductown data

    Many reviews did not specify whether the data came from clinicians, nurses, other primary care staff or multidisciplinary teams; therefore, it was not possible to differentiate the perspectives of specific roles.

    “Many reviews did not specify whether the data came from clinicians, nurses, other primary care staff or multidisciplinary teams; therefore, it was not possible to differentiate the perspectives of specific roles (e.g. nurses).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The original authors reported that very little attention had been paid in ensuring that the potential benefits of new technologies are made transparent through ongoing evaluation and feedback.

    “The original authors reported that very little attention had been paid in ensuring that the potential benefits of new technologies are made transparent through ongoing evaluation and feedback.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7] Mair FS, May C, O'Donnell C, Finch T, Sullivan F, Murray E. Factors that promote or inhibit the implementation of e-health systems: an explanatory systematic review. Bull World Health Organ. 2012;90(5):357–64. source

      Not checked in the free run.

73 of 73

  • The paper relies on this without support.

    The methodological limitations of the included reviews do not prevent a valid synthesis of their findings.

    “When judged against the ENTREQ statement [11], many studies were methodologically poor.”
    “Because the aim was to describe and synthesise a body of qualitative literature and not determine an effect size, studies were not excluded based on this assessment.”
1 of 1

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Study [28] is assigned conflicting healthcare-setting categories

    Minor

    Study [28] is listed among reviews focused on primary care, but it is also listed among reviews that did not define a specific healthcare setting.

  • Findings table cites studies outside the included reviews

    Minor

    The compatibility row cites references [59] and [61] as sources, although the paper says the 44 included reviews are references [15-58].

2 of 2

Cites

Flagged first, then by citations

2 of the 64 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Ross, J., Stevenson, F., Lau, R., & Murray, E.
Affiliations
University College London (GB)
Corresponding author
Ross, Jamie
ORCID
Venue
Implementation Science
Publisher
Springer Science and Business Media LLC
Published
2016-10-26
Volume
11
Issue
1
Pages
146-146
ISSN
1748-5908 (ISSN-L)
DOI
10.1186/s13012-016-0510-7
PMC
PMC5080780
OpenAlex
W2537576125
Cited by
1,011
References
64
Citations by year
2026: 67 · 2025: 116 · 2024: 167 · 2023: 127 · 2022: 134 · 2021: 147 · 2020: 131 · 2019: 61 · 2018: 49 · 2017: 11 · 2015: 1
FWCI
8.10
Type
journal article
Language
en
Open access
Licence
Locations
  • Implementation Science (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
  • UCL Discovery (University College London) (submittedVersion)
Funders
Programme Grants for Applied Research (RP-PG-0609-10135)
Topics
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Health Professions › General Health Professions › Health Policy Implementation Science
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
Keywords
eHealth implementation, implementation barriers and facilitators, Consolidated Framework for Implementation Research, electronic health records, clinical decision support systems, healthcare workers, primary care, secondary care, home care, implementation guidelines
MeSH terms
Clinical Decision-Making, Aptitude, Costs and Cost Analysis, Diffusion of Innovation, Health Personnel, Health Planning, Health Policy, Humans, Health Knowledge, Attitudes, Practice, Review Literature as Topic, Systems Analysis, Telemedicine, Biomedical Technology, Translational Research, Biomedical, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-06.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-06.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review