The DOI matches its Crossref record; 68 of the 69 references it cites were found in a registry or an index, and 1 are not checked yet.
Features predicting the success of computerized decision support for prescribing: a systematic review of randomized controlled trials
Open accessAbstract
BACKGROUND: Computerized decision support systems (CDSS) are believed to have the potential to improve the quality of health care delivery, although results from high quality studies have been mixed. We conducted a systematic review to evaluate whether certain features of prescribing decision support systems (RxCDSS) predict successful implementation, change in provider behaviour, and change in patient outcomes. METHODS: A literature search of Medline, EMBASE, CINAHL and INSPEC databases (earliest entry to June 2008) was conducted to identify randomized controlled trials involving RxCDSS. Each citation was independently assessed by two reviewers for outcomes and 28 predefined system features. Statistical analysis of associations between system features and success of outcomes was planned. RESULTS: Of 4534 citations returned by the search, 41 met the inclusion criteria. Of these, 37 reported successful system implementations, 25 reported success at changing health care provider behaviour, and 5 noted improvements in patient outcomes. A mean of 17 features per study were mentioned. The statistical analysis could not be completed due primarily to the small number of studies and lack of diversity of outcomes. Descriptive analysis did not confirm any feature to be more prevalent in successful trials relative to unsuccessful ones for implementation, provider behaviour or patient outcomes. CONCLUSION: While RxCDSSs have the potential to change health care provider behaviour, very few high quality studies show improvement in patient outcomes. Furthermore, the features of the RxCDSS associated with success (or failure) are poorly described, thus making it difficult for system design and implementation to improve.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“interventions that demonstrated clear and consistent clinical evidence of benefit [3, 16, 34, 35, 51, 55, 56, 68, 74, 89] or good applicability relevant to setting [3, 35, 68] were shown to facilitate implementation.”
From Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews (2015) · cites this paper as [74]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“When the fit was good [14, 73, 74], for example, when e-prescribing was sufficiently integrated as part of clinician workflow, work process was improved [14].”
From Achieving change in primary care—causes of the evidence to practice gap: systematic reviews of reviews (2015) · cites this paper as [74]
Experimental — accuracy not yet measuredIn the citing paper
“However, insufficient attention to the reporting of these features in intervention studies has hampered these approaches.14”
From Computerized clinical decision support for prescribing: provision does not guarantee uptake (2010) · cites this paper as [14]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Framework for design and evaluation of complex interventions to improve health
Campbell, M. (2000) · BMJ · doi:10.1136/bmj.321.7262.694 · Cited by 3,474 · cited 1×
“All CDSS are by definition, complex interventions meaning mutifactorial, multidisciplinary and usually multi-staged [20,63].”
Garg, A. X., Adhikari, N. K. J., McDonald, H., Rosas-Arellano, M. P., Devereaux, P. J., Beyene, J., Sam, J., & Haynes, R. B. (2005) · JAMA · doi:10.1001/jama.293.10.1223 · Cited by 2,936
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors
Koppel, R. (2005) · JAMA · doi:10.1001/jama.293.10.1197 · Cited by 2,126
Improving Safety with Information Technology
Bates, D. W., & Gawande, A. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmsa020847 · Cited by 1,544
Electronic Alerts to Prevent Venous Thromboembolism among Hospitalized Patients
Kucher, N., Koo, S., Quiroz, R., Cooper, J. M., Paterno, M. D., Soukonnikov, B., & Goldhaber, S. Z. (2005) · New England Journal of Medicine · doi:10.1056/nejmoa041533 · Cited by 944
Clinical Decision-Support Systems
Musen, M. A., Middleton, B., & Greenes, R. A. (2013) · Springer eBooks · doi:10.1007/978-1-4471-4474-8_22 · Cited by 588
Physician Inpatient Order Writing on Microcomputer Workstations
Tierney, W. M. (1993) · JAMA · doi:10.1001/jama.269.3.379 · Cited by 549
A Computerized Reminder System to Increase the Use of Preventive Care for Hospitalized Patients
Dexter, P. R., Perkins, S., Overhage, J. M., Maharry, K., Kohler, R. B., & McDonald, C. J. (2001) · New England Journal of Medicine · doi:10.1056/nejmsa010181 · Cited by 519
Detecting Adverse Events Using Information Technology
Bates, D. W., Evans, R. S., Murff, H. J., Stetson, P. D., Pizziferri, L., & Hripcsak, G. M. (2003) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1074 · Cited by 467
Developing and implementing clinical practice guidelines.
Grimshaw, J., Freemantle, N., Wallace, S. A., Russell, I. T., Hurwitz, B., Watt, I., Long, A., & Sheldon, T. (1995) · BMJ Quality & Safety · doi:10.1136/qshc.4.1.55 · Cited by 467
Evaluation Methods in Medical Informatics
Friedman, C. P., & Wyatt, J. C. (1997) · Computers and medicine · doi:10.1007/978-1-4757-2685-5 · Cited by 450
Computer Technology and Clinical Work: Still Waiting for Godot
Wears, R. L., & Berg, M. (2005) · JAMA · doi:10.1001/jama.293.10.1261 · Cited by 448
The Regenstrief Medical Record System: a quarter century experience
McDonald, C. J., Overhage, J. M., Tierney, W. M., Dexter, P. R., Martin, D., Suico, J. G., Zafar, A., Schadow, G., Blevins, L., Glazener, T. T., et al. (1999) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(99)00009-x · Cited by 395
A Randomized Trial of "Corollary Orders" to Prevent Errors of Omission
Overhage, J. M., Tierney, W. M., Zhou, X., & McDonald, C. J. (1997) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1997.0040364 · Cited by 375
Eccles, M. (2002) · BMJ · doi:10.1136/bmj.325.7370.941 · Cited by 354 · cited 2×
“CDSS provide patient-specific advice by using algorithms to compare patient characteristics against a knowledge base [7-9].”
Tamblyn, R. M., Huang, A. R., Perreault, R. B., Jacques, A., Roy, D., Hanley, J. A., McLeod, P. J., & Laprise, R. (2003) · Europe PMC (PubMed Central) · Cited by 298
Sequist, T. D., Gandhi, T. K., Karson, A. S., Fiskio, J. M., Bugbee, D., Sperling, M., Cook, E. F., Orav, E. J., Fairchild, D. G., & Bates, D. W. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1788 · Cited by 250
VistA—U.S. Department of Veterans Affairs national-scale HIS
Brown, S. (2002) · International Journal of Medical Informatics · doi:10.1016/s1386-5056(02)00131-4 · Cited by 238
Improving Blood Pressure Control through Provider Education, Provider Alerts, and Patient Education
Roumie, C. L., Elasy, T. A., Greevy, R. A., Griffin, M. R., Liu, X., Stone, W. J., Wallston, K. A., Dittus, R. S., Alvarez, V., Cobb, J., et al. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-145-3-200608010-00004 · Cited by 233
Paul, M., Andreassen, S., Tacconelli, E., Nielsen, A. D., Almanasreh, N., Frank, U., Cauda, R., & Leibovici, L. (2006) · Journal of Antimicrobial Chemotherapy · doi:10.1093/jac/dkl372 · Cited by 214
Clinical Decision Support and Appropriateness of Antimicrobial Prescribing
Samore, M. H., Bateman, K. A., Alder, S. C., Hannah, E., Donnelly, S. M., Stoddard, G. J., Haddadin, B., Rubin, M. A., Williamson, J., Stults, B. M., et al. (2005) · JAMA · doi:10.1001/jama.294.18.2305 · Cited by 212
Effects of computerized guidelines for managing heart disease in primary care
Tierney, W. M., Overhage, J. M., Murray, M. D., Harris, L. E., Zhou, X., Eckert, G. J., Smith, F. E., Nienaber, N. A., McDonald, C. J., & Wolinsky, F. D. (2003) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2003.30635.x · Cited by 195
Physician Response to Computer Reminders
McDonald, C. J. (1980) · JAMA · doi:10.1001/jama.1980.03310140037026 · Cited by 184
Feldstein, A. C., Elmer, P. J., Smith, D. H., Herson, M., ORWOLL, E. S., Chen, C., Aickin, M. G., & Swain, M. C. (2006) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.2005.00618.x · Cited by 182
Ansari, M. N., Shlipak, M. G., Heidenreich, P. A., Ostaeyen, D. V., Pohl, E. C., Browner, W. S., & Massie, B. M. (2003) · Circulation · doi:10.1161/01.cir.0000070952.08969.5b · Cited by 182
Clinical Decision Support in Electronic Prescribing: Recommendations and an Action Plan
Teich, J. M., Osheroff, J. A., Pifer, E. A., & Sittig, D. F. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1822 · Cited by 180
Use of a Computer to Detect and Respond to Clinical Events: Its Effect on Clinician Behavior
McDonald, C. J. (1976) · Annals of Internal Medicine · doi:10.7326/0003-4819-84-2-162 · Cited by 169
Guide to Health Informatics, 2Ed
Coiera, E. W. (2003) · CRC Press eBooks · doi:10.1201/b13618 · Cited by 168
Tierney, W. M., Overhage, J. M., Murray, M. D., Harris, L. E., Zhou, X., Eckert, G. J., Smith, F. E., Nienaber, N. A., McDonald, C. J., & Wolinsky, F. D. (2005) · Health Services Research · doi:10.1111/j.1475-6773.2005.0t369.x · Cited by 137
Murray, M. D., Harris, L. E., Overhage, J. M., Zhou, X., Eckert, G. J., Smith, F. E., Buchanan, N. N., Wolinsky, F. D., McDonald, C. J., & Tierney, W. M. (2004) · Pharmacotherapy The Journal of Human Pharmacology and Drug Therapy · doi:10.1592/phco.24.4.324.33173 · Cited by 131
Rollman, B. L., Hanusa, B. H., Lowe, H., Gilbert, T., Kapoor, W. N., & Schulberg, H. C. (2002) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2002.10421.x · Cited by 125
Tierney, W. M., Overhage, J. M., Murray, M. D., et al. (2005) · PubMed · doi:10.1111/j.1475-6773.2005.00368.x · Cited by 124
Christakis, D., Zimmerman, F. J., Wright, J. A., Garrison, M. M., Rivara, F. P., & Davis, R. L. (2001) · PEDIATRICS · doi:10.1542/peds.107.2.e15 · Cited by 118
Prescribers' Responses to Alerts During Medication Ordering in the Long Term Care Setting
Judge, J., Field, T. S., DeFlorio, M., LaPrino, J., Auger, J., Rochon, P. A., Bates, D. W., & Gurwitz, J. H. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1945 · Cited by 116
The prognostic value of the components of the Glasgow Coma Scale following acute stroke
Weir, C. J. (2002) · QJM · doi:10.1093/qjmed/hcg008 · Cited by 109
The epidemiology of preventable adverse drug events: A review of the literature
von Laue, N. C., Schwappach, D. L. B., & Koeck, C. M. (2003) · Wiener klinische Wochenschrift · doi:10.1007/bf03040432 · Cited by 96
Hicks, L. S., Sequist, T. D., Ayanian, J. Z., Shaykevich, S., Fairchild, D. G., Orav, E. J., & Bates, D. W. (2008) · Journal of General Internal Medicine · doi:10.1007/s11606-007-0403-1 · Cited by 93
Berner, E. S., Houston, T. K., Ray, M. N., Allison, J. J., Heudebert, G. R., Chatham, W. W., Kennedy, J. I., Glandon, G. L., Norton, P. A., Crawford, M. A., et al. (2005) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1961 · Cited by 91
Filippi, A., Sabatini, A., Badioli, L., Samani, F., Mazzaglia, G., Catapano, A. L., & Cricelli, C. A. (2003) · Diabetes Care · doi:10.2337/diacare.26.5.1497 · Cited by 91
IMPLEMENTING CLINICAL GUIDELINES IN THE TREATMENT OF DIABETES MELLITUS IN GENERAL PRACTICE
Hetlevik, I., Holmen, J., Krüger, Ø., Kristensen, P., Iversen, H., & Furuseth, K. (2000) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300161185 · Cited by 91
McCowan, C. (2001) · Medical Informatics and the Internet in Medicine · doi:10.1080/14639230110067890 · Cited by 90
Randomised controlled trial of computer assisted management of hypertension in primary care.
McAlister, N. H., Covvey, H. D., Tong, C., Lee, A., & Wigle, E. D. (1986) · BMJ · doi:10.1136/bmj.293.6548.670 · Cited by 84
A computerized intervention to decrease the use of calcium channel blockers in hypertension
Rossi, R. A., & Every, N. R. (1997) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.1997.07140.x · Cited by 74
Cost-Benefit Analysis of the Detection of Prescribing Errors by Hospital Pharmacy Staff
van den Bemt, P. M. L. A., Postma, M. J., van Roon, E. N., Chow, M., Fijn, R., & Brouwers, J. R. B. J. (2002) · Drug Safety · doi:10.2165/00002018-200225020-00006 · Cited by 73
Lester, W. T., Grant, R. W., Barnett, G. O., & Chueh, H. C. (2005) · Journal of General Internal Medicine · doi:10.1111/j.1525-1497.2005.00268.x · Cited by 70
Kawamoto, K., & Lobach, D. F. (2003) · PubMed · Cited by 69
Influence of Simple Computerized Feedback on Prescription Charges in an Ambulatory Clinic
Hershey, C. O., Porter, D. K., Breslau, D., & Cohen, D. I. (1986) · Medical Care · doi:10.1097/00005650-198606000-00002 · Cited by 68
Can computerised decision support systems deliver improved quality in primary care?
Delaney, B. C., Fitzmaurice, D. A., Riaz, A. A., & Hobbs, R. F. (1999) · BMJ · doi:10.1136/bmj.319.7220.1281 · Cited by 66
The effect of computer reminders on GPs’ prescribing behaviour: A cluster-randomised trial
Martens, J. D., van der Weijden, T., Severens, J. L., de Clercq, P. A., Bruijn‐Geraets, D. P. D., Kester, A. D. M., & Winkens, R. A. G. (2007) · International Journal of Medical Informatics · doi:10.1016/j.ijmedinf.2007.04.005 · Cited by 63
Selker, H. P., Beshansky, J. R., Griffith, J. L., & for the TPI Trial Investigators* (2002) · Annals of Internal Medicine · doi:10.7326/0003-4819-137-2-200207160-00006 · Cited by 63
Rotman, B. L., Sullivan, A. N., McDonald, T. W., Brown, B. W., DeSmedt, P., Goodnature, D., Higgins, M. C., Suermondt, H. J., Young, C., & Owens, D. K. (1996) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.1996.97035025 · Cited by 61
Computerized physician order entry and medication errors: Finding a balance
Bates, D. W. (2005) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2005.05.003 · Cited by 59
Clinical decision support systems
Coiera, E. (2015) · CRC Press eBooks · doi:10.1201/b13617-31 · Cited by 59
Subramanian, U., Fihn, S. D., Weinberger, M., Plue, L. D., Smith, F. E., Udris, E. M., McDonell, M. B., Eckert, G. J., Temkit, M., Zhou, X., et al. (2004) · The American Journal of Medicine · doi:10.1016/j.amjmed.2003.11.021 · Cited by 53
Randomized Trial to Improve Prescribing Safety During Pregnancy
Raebel, M. A., Carroll, N. M., Kelleher, J. A., Chester, E. A., Berga, S. L., & Magid, D. J. (2007) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m2412 · Cited by 46
A Cluster Randomized Clinical Trial to Improve Prescribing Patterns in Ambulatory Pediatrics
Davis, R. L., Wright, J. A., Chalmers, F., Levenson, L. M., Brown, J. C. S., Lozano, P., & Christakis, D. (2007) · PLoS Clinical Trials · doi:10.1371/journal.pctr.0020025 · Cited by 41
A prospective controlled trial of computerized decision support for lipid management in primary care
Hobbs, R. F., Delaney, B. C., Carson, A., & Kenkre, J. E. (1996) · Family Practice · doi:10.1093/fampra/13.2.133 · Cited by 39
Kuilboer, M. M., van Wijk, M. A. M., Mosseveld, M., van der Does, E., de Jongste, J. C., Overbeek, S. E., Ponsioen, B., & van der Lei, J. (2006) · PubMed · Cited by 38
Niès, J., Colombet, I., Patrice Degoulet, A., & Durieux, P. (2006) · PubMed Central · Cited by 35
Javitt, J. C., Steinberg, G., Locke, T. P., Couch, J. B., JACQUES, J. J., Juster, I. A., & Reisman, L. (2005) · PubMed · Cited by 35
Computed Critiquing Integrated into Daily Clinical Practice Affects Physicians’ Behavior
Kuilboer, M. M., Mosseveld, M., van der Does, E., de Jongste, J. C., Overbeek, S. E., Ponsioen, B., van der Lei, J., & van Wijk, M. A. M. (2006) · Methods of Information in Medicine · doi:10.1055/s-0038-1634103 · Cited by 35
Prescription in Ischaemic Stroke Management (PRISM) Study Group (2003) · QJM · doi:10.1093/qjmed/hcg019 · Cited by 34
Cobos, A., Vilaseca, J., Asenjo, C. S., Pedro‐Botet, J., nchez, E. S., Jiménez, A. V., Torremad, E., Espinosa, C., & n, S. B. (2005) · Disease Management & Health Outcomes · doi:10.2165/00115677-200513060-00007 · Cited by 33
Physician response to computer reminders
McDonald, C. J. (1980) · JAMA · doi:10.1001/jama.244.14.1579 · Cited by 32
Kaushal, R., Shojania, K. G., & Bates, D. W. (2003) · Archives of Internal Medicine · doi:10.1001/archinte.163.12.1409 · Cited by 30
Murray, M. D., Loos, B., Tu, W., Eckert, G. J., Zhou, X., & Tierney, W. M. (1999) · American Journal of Health-System Pharmacy · doi:10.1093/ajhp/56.3.225 · Cited by 30
What factors determine the success of clinical decision support systems?
Holbrook, A. M., Xu, S., & Banting, J. (2003) · PubMed · Cited by 21
Self-citationE-health interventions as complex interventions: improving the quality of methods of assessment.
Holbrook, A. M., Thabane, L., Shcherbatykh, I. Y., & O’Reilly, D. J. (2006) · PubMed Central · Cited by 8
Self-citation
Research works printed but not linked: 2
- Canadian Institute for Health Information: Drug Expenditure in Canada 1985 to 2004. 2005, [http://dsp-psd.pwgsc.gc.ca/Collection/H115-27-2004E.pdf]
- Baker M, Robson B, Shears J: Clinical decision support in the NHS – the clinical element. The Journal of Clinical Governance. 2002, 10: 77-82.
Cited by
Sorted by citations
The Impact of eHealth on the Quality and Safety of Health Care: A Systematic Overview
Black, A. D., Car, J., Pagliari, C., Anandan, C., Cresswell, K., Bokun, T., McKinstry, B., Procter, R., Majeed, A., & Sheikh, A. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1000387 · Cited by 1,442
Ross, J., Stevenson, F., Lau, R., & Murray, E. (2016) · Implementation Science · doi:10.1186/s13012-016-0510-7 · Cited by 1,011
Mair, F. S., May, C., O’Donnell, C., Finch, T., Sullivan, F., & Murray, E. (2012) · Bulletin of the World Health Organization · doi:10.2471/blt.11.099424 · Cited by 553
Lau, R., Stevenson, F., Ong, B. N., Dziedzic, K., Treweek, S., Eldridge, S., Everitt, H., Kennedy, A., Qureshi, N., Rogers, A., et al. (2015) · Implementation Science · doi:10.1186/s13012-016-0396-4 · Cited by 533
Roshanov, P. S., Fernandes, N., Wilczynski, J. M., Hemens, B. J., You, J. J., Handler, S. M., Nieuwlaat, R., Souza, N. M., Beyene, J., Spall, H. G. C. V., et al. (2013) · BMJ · doi:10.1136/bmj.f657 · Cited by 470
Metadata
- Authors
- Mollon, B., Chong, J. J., Holbrook, A. M., Sung, M., Thabane, L., & Foster, G.
- Affiliations
- McMaster University (CA)
- Corresponding author
- Holbrook, Anne M
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2009-02-11
- Volume
- 9
- Issue
- 1
- Pages
- 11-11
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC2667396
- OpenAlex
- W2017372316
- Cited by
- 124
- References
- 67
- Citations by year
- 2025: 4 · 2023: 8 · 2021: 1 · 2020: 7 · 2019: 3 · 2018: 5 · 2017: 9 · 2016: 8 · 2015: 8 · 2014: 11 · 2013: 12 · 2012: 12
- FWCI
- 2.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- Topics
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Technology and Human Factors in Education and Health
- Keywords
- computerized decision support systems, system characteristics, implementation success, patient outcomes, drug safety, system design
- MeSH terms
- Attitude of Health Personnel, Drug Therapy, Computer-Assisted, Humans, Quality Assurance, Health Care, Randomized Controlled Trials as Topic, Decision Support Systems, Clinical, Medical Order Entry Systems
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
