The DOI matches its Crossref record; 13 of the 13 references it cites were found in a registry or an index.
Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials
Open accessAbstract
OBJECTIVES: To identify factors that differentiate between effective and ineffective computerised clinical decision support systems in terms of improvements in the process of care or in patient outcomes. DESIGN: Meta-regression analysis of randomised controlled trials. DATA SOURCES: A database of features and effects of these support systems derived from 162 randomised controlled trials identified in a recent systematic review. Trialists were contacted to confirm the accuracy of data and to help prioritise features for testing. MAIN OUTCOME MEASURES: "Effective" systems were defined as those systems that improved primary (or 50% of secondary) reported outcomes of process of care or patient health. Simple and multiple logistic regression models were used to test characteristics for association with system effectiveness with several sensitivity analyses. RESULTS: Systems that presented advice in electronic charting or order entry system interfaces were less likely to be effective (odds ratio 0.37, 95% confidence interval 0.17 to 0.80). Systems more likely to succeed provided advice for patients in addition to practitioners (2.77, 1.07 to 7.17), required practitioners to supply a reason for over-riding advice (11.23, 1.98 to 63.72), or were evaluated by their developers (4.35, 1.66 to 11.44). These findings were robust across different statistical methods, in internal validation, and after adjustment for other potentially important factors. CONCLUSIONS: We identified several factors that could partially explain why some systems succeed and others fail. Presenting decision support within electronic charting or order entry systems are associated with failure compared with other ways of delivering advice. Odds of success were greater for systems that required practitioners to provide reasons when over-riding advice than for systems that did not. Odds of success were also better for systems that provided advice concurrently to patients and practitioners. Finally, most systems were evaluated by their own developers and such evaluations were more likely to show benefit than those conducted by a third party.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
A solution to the problem of separation in logistic regression
Heinze, G., & Schemper, M. (2002) · Statistics in Medicine · doi:10.1002/sim.1047 · Cited by 2,711
The “Meaningful Use” Regulation for Electronic Health Records
Blumenthal, D., & Tavenner, M. (2010) · New England Journal of Medicine · doi:10.1056/nejmp1006114 · Cited by 2,059
Overriding of Drug Safety Alerts in Computerized Physician Order Entry
van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006) · Journal of the American Medical Informatics Association · doi:10.1197/jamia.m1809 · Cited by 1,117
Quinn, C. C., Clough, S. S., Minor, J. M., Lender, D., Okafor, M. C., & Gruber-Baldini, A. (2008) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2008.0283 · Cited by 406
Improving Preventive Care by Prompting Physicians
Balas, E. A. (2000) · Archives of Internal Medicine · doi:10.1001/archinte.160.3.301 · Cited by 385
Overrides of Medication Alerts in Ambulatory Care
Isaac, T., Weissman, J. S., Davis, R. B., Massagli, M., Cyrulik, A., Sands, D. Z., & Weingart, S. N. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2008.551 · Cited by 309
Lessons from Experienced Guideline Implementers: Attend to Many Factors and Use Multiple Strategies
Solberg, L. I., Brekke, M. L., Fazio, C. J., Fowles, J., Jacobsen, D. N., Kottke, T. E., Mosser, G., O’Connor, P. J., Ohnsorg, K. A., & Rolnick, S. J. (2000) · The Joint Commission Journal on Quality Improvement · doi:10.1016/s1070-3241(00)26013-6 · Cited by 259
Phansalkar, S., Edworthy, J., Hellier, E., Seger, D. L., Schedlbauer, A., Avery, A. J., & Bates, D. W. (2010) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2010.005264 · Cited by 226
Cited by
Understanding and Promoting Effective Engagement With Digital Behavior Change Interventions
Yardley, L., Spring, B. J., Riper, H., Morrison, L. G., Crane, D. H., Curtis, K., Merchant, G. C., Naughton, F., & Blandford, A. (2016) · American Journal of Preventive Medicine · doi:10.1016/j.amepre.2016.06.015 · Cited by 1,118
Interventions for improving the adoption of shared decision making by healthcare professionals
Légaré, F., Ratté, S., Stacey, D., Kryworuchko, J., Gravel, K., Graham, I. D., & Turcotte, S. (2010) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.cd006732.pub2 · Cited by 823
Corp, N., Mansell, G., Stynes, S., Wynne‐Jones, G., Morsø, L., Hill, J. C., & van der Windt, D. A. (2021) · European Journal of Pain · doi:10.1002/ejp.1679 · Cited by 501
The impact of health information technology on patient safety
Alotaibi, Y. K., & Federico, F. (2017) · Saudi Medical Journal · doi:10.15537/smj.2017.12.20631 · Cited by 467
Kwan, J. L., Lo, L., Ferguson, J., Goldberg, H., Diaz-Martinez, J. P., Tomlinson, G., Grimshaw, J. M., & Shojania, K. G. (2020) · BMJ · doi:10.1136/bmj.m3216 · Cited by 461
Metadata
- Authors
- Roshanov, P. S., Fernandes, N., Wilczynski, J. M., Hemens, B. J., You, J. J., Handler, S. M., Nieuwlaat, R., Souza, N. M., Beyene, J., Spall, H. G. C. V., Garg, A. X., & Haynes, R. B.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2013-02-14
- Volume
- 346
- Issue
- feb14 1
- Pages
- f657-f657
- ISSN
- 1756-1833 (ISSN-L)
- OpenAlex
- W2155170531
- Cited by
- 456
- References
- 13
- FWCI
- 73.93
- Type
- journal article
- Language
- en
- Open access
- hybrid
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. https://app.cochat.ai/papers/10.1136/bmj.f657
- The paper
- Roshanov, P. S., Fernandes, N., Wilczynski, J. M., Hemens, B. J., You, J. J., Handler, S. M., Nieuwlaat, R., Souza, N. M., Beyene, J., Spall, H. G. C. V., Garg, A. X., & Haynes, R. B. (2013). Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. https://doi.org/10.1136/bmj.f657
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.