The DOI matches its Crossref record; 165 of the 168 references it cites were found in a registry or an index; 4 of the 32 sources its claims cite were checked for support.
The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Open accessAbstract
BACKGROUND: There is considerable international interest in exploiting the potential of digital solutions to enhance the quality and safety of health care. Implementations of transformative eHealth technologies are underway globally, often at very considerable cost. In order to assess the impact of eHealth solutions on the quality and safety of health care, and to inform policy decisions on eHealth deployments, we undertook a systematic review of systematic reviews assessing the effectiveness and consequences of various eHealth technologies on the quality and safety of care. METHODS AND FINDINGS: We developed novel search strategies, conceptual maps of health care quality, safety, and eHealth interventions, and then systematically identified, scrutinised, and synthesised the systematic review literature. Major biomedical databases were searched to identify systematic reviews published between 1997 and 2010. Related theoretical, methodological, and technical material was also reviewed. We identified 53 systematic reviews that focused on assessing the impact of eHealth interventions on the quality and/or safety of health care and 55 supplementary systematic reviews providing relevant supportive information. This systematic review literature was found to be generally of substandard quality with regards to methodology, reporting, and utility. We thematically categorised eHealth technologies into three main areas: (1) storing, managing, and transmission of data; (2) clinical decision support; and (3) facilitating care from a distance. We found that despite support from policymakers, there was relatively little empirical evidence to substantiate many of the claims made in relation to these technologies. Whether the success of those relatively few solutions identified to improve quality and safety would continue if these were deployed beyond the contexts in which they were originally developed, has yet to be established. Importantly, best practice guidelines in effective development and deployment strategies are lacking. CONCLUSIONS: There is a large gap between the postulated and empirically demonstrated benefits of eHealth technologies. In addition, there is a lack of robust research on the risks of implementing these technologies and their cost-effectiveness has yet to be demonstrated, despite being frequently promoted by policymakers and "techno-enthusiasts" as if this was a given. In the light of the paucity of evidence in relation to improvements in patient outcomes, as well as the lack of evidence on their cost-effectiveness, it is vital that future eHealth technologies are evaluated against a comprehensive set of measures, ideally throughout all stages of the technology's life cycle. Such evaluation should be characterised by careful attention to socio-technical factors to maximise the likelihood of successful implementation and adoption.
Overview
CoChat Summary
TL;DR
This systematic overview of 108 reviews found that the empirical evidence for most common eHealth technologies’ benefits on quality and safety is weak, inconsistent, or absent. The paper argues that large-scale eHealth investments should not be justified by assumed gains alone, because patient-level and cost-effectiveness benefits were rarely demonstrated.
Key findings
- The searches retrieved 46,349 references, and the authors included 108 reviews: 53 as the main empirical evidence base and 55 supplementary reviews for context.
- For EHRs, the paper found only weak evidence for benefits in time efficiency and data quality (legibility, completeness, comprehensiveness), while risks such as time costs, increased costs, and reduced patient-centeredness were mostly anecdotal.
- For PACS, there were only 3 reviews; evidence for improved organizational efficiency was weak, and overall cost-effectiveness could not be determined because economic findings were often contradictory and of poor quality.
- For CPOE and ePrescribing, the review reports moderate evidence for some organizational/practitioner benefits, but weak or inconsistent evidence for patient-level outcomes; for ePrescribing, there was no clear evidence of patient outcome benefit outside prescribing, and medication-error reductions were not clearly linked to patient harm reduction.
Methods
- Systematic overview of systematic reviews focused on clinician-oriented eHealth technologies related to storage/retrieval of data and support for clinical decision making; patient-focused interventions were excluded.
- Searched PubMed/MEDLINE, EMBASE, and the Cochrane Library for secondary research reports published from 1997 to 2007, with no language restrictions; searches were updated through end of April 2010.
- Performed dual-reviewer screening and critical appraisal (with arbitration by a third reviewer), then used narrative synthesis because the literature was too diverse for quantitative pooling; evidence was classified using a modified WHO Health Evidence Network approach as strong, moderate, or weak.
Limitations
- The paper states that poor indexing, reliance on English-language databases, and a novel multimethod approach meant some reviews may have been missed and the synthesis was “less rigorous” than a conventional systematic review because individual primary studies were not appraised.
- The authors note that they had to adapt the critical appraisal instrument, and that overlap in reviews plus inconsistent terminology required judgment calls about which reviews and primary studies mapped to which interventions.
- Evident from the text: the overview covers mostly economically developed country settings and clinician-orientated information systems, so the findings may not generalize well to patient-oriented technologies or other contexts.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
4 Valid · 0 Not valid · 24 Not checked · 67 Own results and interpretations
Every source the paper cites for this claim was checked and supports it.
In the paper
“NHS Connecting for Health's difficulties with the implementation of EHRs into hospitals in England is a prime example of the challenges that can ensue if such socio-technical factors are given insufficient attention [148].”
§ The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview > Discussion
What the sources say
- Supportedcites [148] AR Robertson, 2010, Prospective evaluation of the implementation and adoption of NHS Connecting for Health's electronic health record in secondary care in England: interim results from a national evaluation., BMJ source
The Methods (Data analysis; page not specified) explicitly frame implementation as a socio-technical process in which organisational, human, and technical factors shape one another. The Results (Trusts want systems that are tailored; page not specified) describe implementation difficulties caused by a centrally conceived standardised system that did not meet end users’ needs. This supports the citing sentence’s use of the programme as an example of challenges arising when those factors receive insufficient attention. (LLM, conf=0.90)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Additionally, it is not unusual for insufficient time and effort to be devoted to the all-important customisation process [147].”
§ The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview > Discussion
What the sources say
- Partly supportedcites [147] N Pollock, 2003, Fitting standard software packages to non-standard organizations: the “biography' of an enterprise-wide system., Tech Anal Strat Manag sourceExperimental — accuracy not yet measured
In “Building a Global Product” (pp. 16–17), the paper reports that the supplier had more requests for changes than resources and was pruning what it would deliver for a pilot site. This supports a resource constraint on adaptation in that case, but the paper does not establish that insufficient time and effort are often devoted to customisation across commercial systems. (LLM, conf=0.82)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Also of note is that we found virtually no evidence in support of the cost-effectiveness claims (Tables 2– 11) that are frequently being made by policy makers when constructing business cases to raise funding for the large-scale eHealth deployments that are now taking place in many parts of the world [1].”
§ The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview > Discussion
What the sources say
- Partly supportedcites [1] L Catwell, 2009, Evaluating eHealth interventions., PLoS Med sourceExperimental — accuracy not yet measured
The ‘Recognising the Risk of Harm’ section says governments around the world are undertaking large eHealth rollouts, and the Conclusions call for scientific evidence to inform policy decisions. The paper does not specifically document frequent cost-effectiveness claims in policymakers’ funding business cases. The citing authors’ statement that ‘we found virtually no evidence’ refers to their own findings, not a finding attributed to this source. (LLM, conf=0.79)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We found only anecdotal evidence of the fundamental expected benefits and risks relating to the organisational efficiency resulting from the storage and management facilities within the EHR and thus the potential for secondary uses (Table 2).”
§ The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview > Results > Data Storage, Management, and Retrieval Systems
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Our final selection of 53 reviews provided the main empirical evidence base in relation to assessing the impact of the selected eHealth technologies (see Table 1 for our critical appraisal of these studies) [15]–[67], full details of which can be found in Table S2.”
§ The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview > Results
What the sources say
- Not checked yetcites [15]–[67]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
41 more claims
Create a free account to see all 95 claims and to verify the 28 references still open.
- Probably not a substantial claim. 27
- It may state two separate claims as one. 28
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While such technologies can be effective [1,2], one of the main problems is that users’ adoption and long-term use remains lower than expected [3-5].”
From Opening the Black Box of Electronic Health: Collecting, Analyzing, and Interpreting Log Data (2017) · cites this paper as [5]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“overall effectiveness in clinical practice is unclear.14”
From ‘Too much, too late’: mixed methods multi-channel video recording study of computerized decision support systems and GP prescribing (2013) · cites this paper as [14]
The cited source says what the claim attributes to it.
In the citing paper
“Several authors [1,2,4,6,62,76-87] note that a broader view is needed to assess the overall impact on health care.”
From A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies (2011) · cites this paper as [2]
The cited source says what the claim attributes to it.
In the citing paper
“The foundation of the framework is based on the aforementioned principles, reviews in the field of eHealth [2,3,22,96-99], progressive insights from eHealth research (see Multimedia Appendix 4), and multidisciplinary theories from psychology, communication, and human–computer interaction design.”
From A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies (2011) · cites this paper as [2]
The check wasn’t confident either way, so it gives no verdict — never disputed.
In the citing paper
“The scale, volume of data obtained, and real time nature of this evaluation of the implementation of electronic health records in English secondary healthcare settings are strengths of our work.1”
From Implementation and adoption of nationwide electronic health records in secondary care in England: final qualitative results from prospective national evaluation in early adopter hospitals (2011) · cites this paper as [1]
Experimental — accuracy not yet measured
5 more claims about this paper
Create a free account to see every claim other papers make about this one.
Citations
Cites
Older Adults' Reasons for Using Technology while Aging in Place
Peek, S. T. M., Luijkx, K. G., Rijnaard, M. D., Nieboer, M. E., van der Voort, C. S., Aarts, S., van Hoof, J., Vrijhoef, H. J. M., & Wouters, E. J. M. (2016) · Gerontology · doi:10.1159/000430949 · Cited by 19,690
To Err Is Human: Building a Safer Health System
(2000) · National Academies Press eBooks · doi:10.17226/9728 · Cited by 14,044
Crossing the Quality Chasm: A New Health System for the 21st Century
Baker, A. (2001) · BMJ · doi:10.1136/bmj.323.7322.1192 · Cited by 10,736
Wall, A. (2000) · British Journal of Healthcare Management · doi:10.12968/bjhc.2000.6.9.19311 · Cited by 5,903
Crossing the Quality Chasm: A New Health System for the 21st Century
Bloom, B. S. (2002) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.287.5.646-a · Cited by 3,726
Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S. C., & Shekelle, P. G. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-10-200605160-00125 · Cited by 3,159
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
The Technology Acceptance Model: Its past and its future in health care
Holden, R. J., & Karsh, B.-T. (2010) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2009.07.002 · Cited by 2,669
Critical appraisal skills programme
Singh, J. (2013) · Journal of Pharmacology and Pharmacotherapeutics · doi:10.4103/0976-500x.107697 · Cited by 2,556
Research works printed but not linked: 3
- C Pagliari, 2004, EH1 E-Health scoping exercise. Review of wider Web-based information sources.
- C Pagliari, 2004, EH1 E-Health scoping exercise. Review of the traditional research literature.
- PG Shekelle, 2009, Costs and benefits of health technology information: an updated systematic review.
Cited by
Allocation of Physician Time in Ambulatory Practice: A Time and Motion Study in 4 Specialties
Sinsky, C., Colligan, L., Li, L., Prgomet, M., Reynolds, S., Goeders, L., Westbrook, J., Tutty, M., & Blike, G. (2016) · Annals of Internal Medicine · doi:10.7326/m16-0961 · Cited by 1,494
Buntin, M. B., Burke, M. F., Hoaglin, M. C., & Blumenthal, D. (2011) · Health Affairs · doi:10.1377/hlthaff.2011.0178 · Cited by 1,311
Persuasive System Design Does Matter: a Systematic Review of Adherence to Web-based Interventions
Kelders, S. M., Kok, R. N., Ossebaard, H. C., & Van Gemert-Pijnen, J. E. (2012) · Journal of Medical Internet Research · doi:10.2196/jmir.2104 · Cited by 1,308
“This approach has been recommended in recent literature [10, 11, 13, 23] and has been the key point in the field of persuasive technology [24], where there are examples of studies on the persuasive capacities of technology to support web-based interventions in the health care domain [25-28].”
BackgroundGlobal health 2035: a world converging within a generation
Jamison, D. T., Summers, L. H., Alleyne, G., Arrow, K. J., Berkley, S., Binagwaho, A., Bustreo, F., Evans, D., Feachem, R. G. A., Frenk, J., et al. (2013) · The Lancet · doi:10.1016/s0140-6736(13)62105-4 · Cited by 1,220
A Holistic Framework to Improve the Uptake and Impact of eHealth Technologies
van Gemert-Pijnen, J. E., Nijland, N., van Limburg, M., Ossebaard, H. C., Kelders, S. M., Eysenbach, G., & Seydel, E. R. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.1672 · Cited by 1,126
Metadata
- Authors
- Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A.
- Affiliations
- Imperial College London (GB)
- University of Edinburgh (GB)
- University of Manchester (GB)
- Corresponding author
- Sheikh, Aziz
- ORCID
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-01-18
- Volume
- 8
- Issue
- 1
- Pages
- e1000387
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- PMC
- PMC3022523
- OpenAlex
- W2002144979
- Semantic Scholar
- 8989e6727eab5d5fc760d0c7a63fb3864fe49a4f
- Cited by
- 1,442
- References
- 168
- Citations by year
- 2026: 23 · 2025: 26 · 2024: 44 · 2023: 46 · 2022: 55 · 2021: 81 · 2020: 94 · 2019: 102 · 2018: 109 · 2017: 110 · 2016: 129 · 2015: 109 · 2014: 126 · 2013: 207 · 2012: 111
- FWCI
- 24.19
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (publishedVersion)
- Warwick Research Archive Portal (University of Warwick) (acceptedVersion)
- Research Publications (Maastricht University) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Research Explorer (The University of Manchester) (submittedVersion)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Spiral (Imperial College London) (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion, other-oa)
- Funders
- Economic and Social Research Council
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Telemedicine and Telehealth Implementation
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- e-health, quality of care, patient safety, clinical decision support, telemedicine, health data management, digital health technologies, cost-effectiveness, evidence gaps, health policy
- MeSH terms
- Cost-Benefit Analysis, Delivery of Health Care, Health Policy, Medical Informatics, Quality of Health Care, Radiology Information Systems, Review Literature as Topic, Efficiency, Organizational, Electronic Prescribing, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview. https://app.cochat.ai/papers/10.1371/journal.pmed.1000387
- The paper
- Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011). The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview. PLoS Medicine. https://doi.org/10.1371/journal.pmed.1000387
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
