The DOI matches its Crossref record; 62 of the 62 references it cites were found in a registry or an index.
Effectiveness of Computerized Decision Support Systems Linked to Electronic Health Records: A Systematic Review and Meta-Analysis
Open accessAbstract
We systematically reviewed randomized controlled trials (RCTs) assessing the effectiveness of computerized decision support systems (CDSSs) featuring rule- or algorithm-based software integrated with electronic health records (EHRs) and evidence-based knowledge. We searched MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, and Cochrane Database of Abstracts of Reviews of Effects. Information on system design, capabilities, acquisition, implementation context, and effects on mortality, morbidity, and economic outcomes were extracted. Twenty-eight RCTs were included. CDSS use did not affect mortality (16 trials, 37395 patients; 2282 deaths; risk ratio [RR] = 0.96; 95% confidence interval [CI] = 0.85, 1.08; I 2 = 41%). A statistically significant effect was evident in the prevention of morbidity, any disease (9 RCTs; 13868 patients; RR = 0.82; 95% CI = 0.68, 0.99; I 2 = 64%), but selective outcome reporting or publication bias cannot be excluded. We observed differences for costs and health service utilization, although these were often small in magnitude. Across clinical settings, new generation CDSSs integrated with EHRs do not affect mortality and might moderately improve morbidity outcomes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139
Bias in meta-analysis detected by a simple, graphical test
Egger, M., Smith, G. D., Schneider, M., & Minder, C. (1997) · BMJ · doi:10.1136/bmj.315.7109.629 · Cited by 56,722
Meta-analysis in clinical trials
DerSimonian, R., & Laird, N. (1986) · Controlled Clinical Trials · doi:10.1016/0197-2456(86)90046-2 · Cited by 39,552
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
Operating characteristics of a rank correlation test for publication bias
Begg, C. B., & Mazumdar, M. (1994) · Biometrics · doi:10.2307/2533446 · Cited by 16,955
Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease
(1959) · JNCI: Journal of the National Cancer Institute · doi:10.1093/jnci/22.4.719 · Cited by 14,689
The combination of estimates from different experiments
Cochran, W. G. (1954) · Biometrics · doi:10.2307/3001666 · Cited by 5,380 · cited 1×
“10 level of significance.(34) We also calculated the I(2) statistic(35) that describes the percentage variation across studies that is attributed to heterogeneity rather than chance.”
BackgroundCrossing the Quality Chasm: A New Health System for the 21st Century
Bloom, B. S. (2002) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.287.5.646-a · Cited by 3,726
Chaudhry, B., Wang, J., Wu, S., Maglione, M., Mojica, W., Roth, E., Morton, S. C., & Shekelle, P. G. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-10-200605160-00125 · Cited by 3,159
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
Bates, D. W., Kuperman, G. J., Wang, S., Gandhi, T., Kittler, A., Volk, L., Spurr, C., Khorasani, R., Tanasijevic, M., & Middleton, B. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1370 · Cited by 1,346
Effect of Clinical Decision-Support Systems: A Systematic Review
Bright, T. J., Wong, A., Dhurjati, R., Bristow, E., Bastian, L., Coeytaux, R. R., Samsa, G., Hasselblad, V., Williams, J. W., Musty, M. D., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-1-201207030-00450 · Cited by 1,200
Electronic Alerts to Prevent Venous Thromboembolism among Hospitalized Patients
Kucher, N., Koo, S., Quiroz, R., Cooper, J. M., Paterno, M. D., Soukonnikov, B., & Goldhaber, S. Z. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa041533 · Cited by 942
Clinical decision support systems for the practice of evidence-based medicine
Sim, I., Gorman, P., Greenes, R. A., Haynes, R. B., Kaplan, B., Lehmann, H., & Tang, P. C. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080527 · Cited by 732
Roshanov, P. S., Fernandes, N., Wilczynski, J. M., Hemens, B. J., You, J. J., Handler, S. M., Nieuwlaat, R., Souza, N. M., Beyene, J., Spall, H. G. C. V., et al. (2013) · BMJ · doi:10.1136/bmj.f657 · Cited by 456
Gunter, T. D., & Terry, N. P. (2005) · Journal of Medical Internet Research · doi:10.2196/jmir.7.1.e3 · Cited by 439
Accessing pre-appraised evidence: fine-tuning the 5S model into a 6S model
DiCenso, A., Bayley, L., & Haynes, R. B. (2009) · Evidence Based Nursing · doi:10.1136/ebn.12.4.99-b · Cited by 370
Romano, M. J., & Stafford, R. S. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2010.527 · Cited by 297
Statistical Methods in Meta-Analysis
Petitti, D. B. (1999) · Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis · doi:10.1093/acprof:oso/9780195133646.003.07 · Cited by 287
Montgomery, A. A. (2000) · BMJ · doi:10.1136/bmj.320.7236.686 · Cited by 279
Impact of Electronic Health Record Clinical Decision Support on Diabetes Care: A Randomized Trial
O'Connor, P. J., Sperl-Hillen, J. M., Rush, W. A., Johnson, P. E., Amundson, G. H., Asche, S. E., Ekstrom, H. L., & Gilmer, T. P. (2011) · The Annals of Family Medicine · doi:10.1370/afm.1196 · Cited by 263
Roshanov, P. S., Misra, S., Gerstein, H. C., Garg, A. X., Sebaldt, R. J., Mackay, J. A., Weise-Kelly, L., Navarro, T., Wilczynski, N. L., Haynes, R. B., et al. (2011) · Implementation Science · doi:10.1186/1748-5908-6-92 · Cited by 236
Holbrook, A., Thabane, L., Keshavjee, K., Dolovich, L., Bernstein, B., Chan, D., Troyan, S., Foster, G., Gerstein, H., & for the COMPETE II Investigators (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.081272 · Cited by 231
Paul, M., Andreassen, S., Tacconelli, E., Nielsen, A. D., Almanasreh, N., Frank, U., Cauda, R., & Leibovici, L. (2006) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkl372 · Cited by 214
Tierney, W. M., Overhage, J. M., Murray, M. D., Harris, L. E., Zhou, X.-H., Eckert, G. J., Smith, F. E., Nienaber, N., McDonald, C. J., & Wolinsky, F. D. (2003) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2003.30635.x · Cited by 199
Gonzales, R., Anderer, T., McCulloch, C. E., Maselli, J. H., Bloom, F. J., Graf, T. R., Stahl, M., Yefko, M., Molecavage, J., & Metlay, J. P. (2013) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2013.1589 · Cited by 186
Souza, N. M., Sebaldt, R. J., Mackay, J. A., Prorok, J. C., Weise-Kelly, L., Navarro, T., Wilczynski, N. L., Haynes, R. B., & the CCDSS Systematic Review Team (2011) · Implementation Science · doi:10.1186/1748-5908-6-87 · Cited by 178
McGregor, J. C., Weekes, E., Forrest, G. N., Standiford, H. C., Perencevich, E. N., Furuno, J. P., & Harris, A. D. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2049 · Cited by 175
Miriovsky, B. J., Shulman, L. N., & Abernethy, A. P. (2012) · Journal of Clinical Oncology · doi:10.1200/jco.2012.42.8011 · Cited by 159
Haynes, R. B., & Wilczynski, N. L. (2010) · Implementation Science · doi:10.1186/1748-5908-5-12 · Cited by 150
Electronic Health Record–Based Decision Support to Improve Asthma Care: A Cluster-Randomized Trial
Bell, L. M., Grundmeier, R. W., Localio, R., Zorc, J. J., Fiks, A. G., Zhang, X., Stephens, T. B., Swietlik, M., & Guevara, J. P. (2010) · PEDIATRICS · doi:10.1542/peds.2009-1385 · Cited by 146
Computer Decision Support Systems
Payne, T. H. (2000) · CHEST Journal · doi:10.1378/chest.118.2_suppl.47s · Cited by 145
Trauma Resuscitation Errors and Computer-Assisted Decision Support
Fitzgerald, M. (2011) · Archives of Surgery · doi:10.1001/archsurg.2010.333 · Cited by 139
Roy, P., Durieux, P., Gillaizeau, F., Legall, C., Armand-Perroux, A., Martino, L., Hachelaf, M., Dubart, A.-E., Schmidt, J., Cristiano, M., et al. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-10-200911170-00003 · Cited by 131
Rothschild, J. M., McGurk, S., Honour, M. M., Lu, L. K., McClendon, A. A., Srivastava, P., Churchill, W. H., Kaufman, R. M., Avorn, J., Cook, E. F., et al. (2006) · Transfusion · doi:10.1111/j.1537-2995.2007.01093.x · Cited by 124
Bonnabry, P., Despont-Gros, C., Grauser, D., Casez, P., Despond, M., Pugin, D., Rivara-Mangeat, C., Koch, M., Vial, M., Iten, A., et al. (2008) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2677 · Cited by 120
Bosworth, H. B., Olsen, M. K., Dudley, T. K., Orr, M., Goldstein, M. K., Datta, S., McCant, F., Gentry, P., Simel, D. L., & Oddone, E. Ž. (2009) · American Heart Journal · doi:10.1016/j.ahj.2008.11.003 · Cited by 106
Gurwitz, J. H., Field, T. S., Rochon, P., Judge, J., Harrold, L. R., Bell, C. M., Lee, M., White, K., LaPrino, J., Erramuspe‐Mainard, J., et al. (2008) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2008.02004.x · Cited by 106
IMPLEMENTING CLINICAL GUIDELINES IN THE TREATMENT OF DIABETES MELLITUS IN GENERAL PRACTICE
Hetlevik, I., Holmen, J., Krüger, Ø., Kristensen, P., Iversen, H., & Furuseth, K. (2000) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300161185 · Cited by 91
Jeffery, R. A., Iserman, E. C., Haynes, R. B., & the CDSS Systematic Review Team (2012) · Diabetic Medicine · doi:10.1111/dme.12087 · Cited by 91
A Review of Online Evidence-based Practice Point-of-Care Information Summary Providers
Banzi, R., Liberati, A., Moschetti, I., Tagliabue, L., & Moja, L. (2010) · Journal of Medical Internet Research · doi:10.2196/jmir.1288 · Cited by 91
Self-citationMcCowan, C. (2001) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230110067890 · Cited by 90
Hetlevik, I., Jostein, H., & Øystein, K. (1999) · Scandinavian Journal of Primary Health Care · doi:10.1080/028134399750002872 · Cited by 88
Tamblyn, R. M., Eguale, T., Buckeridge, D. L., Huang, A. R., Hanley, J., Reidel, K. E., Shi, S., & Winslade, N. E. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000609 · Cited by 83
Abdel-Kader, K. F. M., Fischer, G. S., Li, J., Moore, C. G., Hess, R., & Unruh, M. L. (2011) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2011.08.028 · Cited by 78
Cost-effectiveness of an electronic medical record based clinical decision support system
Gilmer, T. P., O'Connor, P. J., Sperl‐Hillen, J. M., Rush, W. A., Johnson, P. E., Amundson, G. H., Asche, S. E., & Ekstrom, H. L. (2012) · Health Services Research · doi:10.1111/j.1475-6773.2012.01427.x · Cited by 73
Shared Electronic Vascular Risk Decision Support in Primary Care
Holbrook, A. M. (2011) · Archives of Internal Medicine · doi:10.1001/archinternmed.2011.471 · Cited by 57
MacLean, C. D., Gagnon, M., CALLAS, P. W., & Littenberg, B. (2009) · Journal of General Internal Medicine · doi:10.1007/s11606-009-1147-x · Cited by 56
A Randomized Outpatient Trial of a Decision-Support Information Technology Tool
Apkon, M., Mattera, J. A., Lin, Z., Herrin, J., Bradley, E. H., Carbone, M., Holmboe, E. S., Gross, C. P., Selter, J. G., Rich, A. S., et al. (2005) · Archives of Internal Medicine · doi:10.1001/archinte.165.20.2388 · Cited by 56
The Development of Primary Care Information Technology in the United Kingdom
de Lusignan, S., & Chan, T. (2008) · Journal of Ambulatory Care Management · doi:10.1097/01.jac.0000324664.88131.d2 · Cited by 52
Efficacy of a Clinical Decision-Support System in an HIV Practice: A Randomized Trial
Robbins, G. K., Lester, W., Johnson, K. L., Chang, Y., Estey, G. A., Surrao, D., Zachary, K. C., Lammert, S. M., Chueh, H. C., Meigs, J. B., et al. (2012) · Annals of Internal Medicine · doi:10.7326/0003-4819-157-11-201212040-00003 · Cited by 50
Clinical decision support systems: Potential with pitfalls
Eberhardt, J. S., Bilchik, A. J., & Stojadinovic, A. (2012) · Journal of Surgical Oncology · doi:10.1002/jso.23053 · Cited by 43
Ash, J. S., McCormack, J. L., Sittig, D. F., Wright, A., McMullen, C., & Bates, D. W. (2012) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000705 · Cited by 41
Information Technology and Medical Missteps: Evidence from a Randomized Trial
Javitt, J. C., Rebitzer, J. B., & Reisman, L. (2007) · Journal of Health Economics · doi:10.1016/j.jhealeco.2007.10.008 · Cited by 41
Graumlich, J. F., Novotny, N. L., Nace, G. S., & Aldag, J. C. (2009) · Journal of Hospital Medicine · doi:10.1002/jhm.565 · Cited by 39
O’Reilly, D. J., Holbrook, A. M., Blackhouse, G., Troyan, S., & Goeree, R. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000371 · Cited by 35
Cobos, A., Vilaseca, J., Asenjo, C. S., Pedro‐Botet, J., nchez, E. S., Jiménez, A. V., Torremad, E., Espinosa, C., & n, S. B. (2005) · Disease Management & Health Outcomes · doi:10.2165/00115677-200513060-00007 · Cited by 34
Cleveringa, F. G. W., Welsing, P. M. J., van den Donk, M., Gorter, K. J., Niessen, L. W., Rutten, G. E. H. M., & Redekop, K. (2009) · Diabetes Care · doi:10.2337/dc09-1232 · Cited by 33
Kaushal, R., Shojania, K. G., & Bates, D. W. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.12.1409 · Cited by 30
Navigators for medicine: evolution of online point-of-care evidence-based services
Moja, L., & Banzi, R. (2010) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2010.02441.x · Cited by 25
Self-citationKhan, S., MacLean, C. D., & Littenberg, B. (2010) · Health Outcomes Research in Medicine · doi:10.1016/j.ehrm.2010.03.002 · Cited by 24
Subramanian, S., Hoover, S., Wagner, J. L., Donovan, J. L., Kanaan, A. O., Rochon, P. A., Gurwitz, J. H., & Field, T. S. (2011) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2011-000179 · Cited by 15
Cited by
Kwan, J. L., Lo, L., Ferguson, J., Goldberg, H., Diaz-Martinez, J. P., Tomlinson, G., Grimshaw, J. M., & Shojania, K. G. (2020) · BMJ · doi:10.1136/bmj.m3216 · Cited by 461
Liberati, E. G., Ruggiero, F., Galuppo, L., Gorli, M., González-Lorenzo, M., Maraldi, M., Ruggieri, P., Polo Friz, H., Scaratti, G., Kwag, K. H., et al. (2017) · Implementation Science · doi:10.1186/s13012-017-0644-2 · Cited by 301
Self-citationMachine learning in clinical decision making
Adlung, L., Cohen, Y., Mor, U., & Elinav, E. (2021) · Med · doi:10.1016/j.medj.2021.04.006 · Cited by 282
Harnessing the Power of Clinical Decision Support Systems: Challenges and Opportunities
Chen, Z., Liang, N., Zhang, H., Li, H., Yang, Y., Zong, X., Chen, Y., Wang, Y., & Shi, N. (2023) · Open Heart · doi:10.1136/openhrt-2023-002432 · Cited by 244
The impact of machine learning on patient care: A systematic review
Ben-Israel, D., Jacobs, W. B., Casha, S., Lang, S., Ryu, W. H. A., de Lotbiniere-Bassett, M., & Cadotte, D. W. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2019.101785 · Cited by 207
Metadata
- Authors
- Moja, L., Kwag, K. H., Lytras, T., Bertizzolo, L., Brandt, L., Pecoraro, V., Rigon, G., Vaona, A., Ruggiero, F., Mangia, M., Iorio, A., Kunnamo, I., & Bonovas, S.
- Venue
- American Journal of Public Health
- Publisher
- American Public Health Association
- Published
- 2014-12
- Volume
- 104
- Issue
- 12
- Pages
- e12-e22
- ISSN
- 0090-0036 (ISSN-L)
- 1541-0048
- OpenAlex
- W2059459859
- Cited by
- 339
- References
- 62
- FWCI
- 6.78
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.