Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions
The DOI matches its Crossref record; 79 of the 80 references it cites were found in a registry or an index.
Abstract
Clinical prediction rules, sometimes called clinical decision rules, have proliferated in recent years. However, very few have undergone formal impact analysis, the standard of evidence to assess their impact on patient care. Without impact analysis, clinicians cannot know whether using a prediction rule will be beneficial or harmful. This paper reviews standards of evidence for developing and evaluating prediction rules; important differences between prediction rules and decision rules; how to assess the potential clinical impact of a prediction rule before translating it into a decision rule; methodologic issues critical to successful impact analysis, including defining outcome measures and estimating sample size; the importance of close collaboration between clinical investigators and practicing clinicians before, during, and after impact analysis; and the need to measure both efficacy and effectiveness when analyzing a decision rule's clinical impact. These considerations should inform future development, evaluation, and use of all clinical prediction or decision rules.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Overfitting, optimism, and miscalibration may also be addressed and accounted for during the model development by applying shrinkage (for example, heuristic or based on bootstrapping techniques) or penalization procedures (for example, ridge regression or lasso) (20).”
From Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement (2015) · cites this paper as [20]
In the citing paper
“Quantifying the impact of a prediction model on participant outcome requires a comparative study design [9],[48].”
From Reporting and Methods in Clinical Prediction Research: A Systematic Review (2012) · cites this paper as [48]
Citations
Cites
Why Don't Physicians Follow Clinical Practice Guidelines?: A Framework for Improvement
Cabana, M. D., Rand, C. S., Powe, N. R., Wu, A. W., Wilson, M. H., Abboud, P.-A. C., & Rubin, H. R. (1999) · JAMA · doi:10.1001/jama.282.15.1458 · Cited by 6,677
Gage, B. F., Waterman, A. D., Shannon, W., Boechler, M., Rich, M. W., & Radford, M. J. (2001) · JAMA · doi:10.1001/jama.285.22.2864 · Cited by 4,832
Antman, E. M., Cohen, M., Bernink, P. J. L. M., McCabe, C. H., Horacek, T., Papuchis, G., Mautner, B., Corbalan, R., Radley, D., & Braunwald, E. (2000) · JAMA · doi:10.1001/jama.284.7.835 · Cited by 3,242
Serial Evaluation of the SOFA Score to Predict Outcome in Critically Ill Patients
Ferreira, F. L. (2001) · JAMA · doi:10.1001/jama.286.14.1754 · Cited by 2,862
Kawamoto, K., Houlihan, C. A., Balas, E. A., & Lobach, D. F. (2005) · BMJ · doi:10.1136/bmj.38398.500764.8f · Cited by 2,703
Foundations of Cost-Effectiveness Analysis for Health and Medical Practices
Weinstein, M. C., & Stason, W. B. (1977) · New England Journal of Medicine · doi:10.1056/nejm197703312961304 · Cited by 2,119
D'Agostino, R. B., Grundy, S., Sullivan, L. M., Wilson, P., & for the CHD Risk Prediction Group (2001) · JAMA · doi:10.1001/jama.286.2.180 · Cited by 2,108
(2001) · PubMed · Cited by 1,709
Evaluation of D-Dimer in the Diagnosis of Suspected Deep-Vein Thrombosis
Wells, P. S., Anderson, D. R., Rodger, M., Forgie, M., Kearon, C., Dreyer, J., Kovacs, G., Mitchell, M., Lewandowski, B., & Kovacs, M. J. (2003) · New England Journal of Medicine · doi:10.1056/nejmoa023153 · Cited by 1,552
Wells, P. S., Anderson, D. R., Rodger, M., Stiell, I., Dreyer, J. F., Barnes, D., Forgie, M., Kovacs, G., Ward, J., & Kovacs, M. J. (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-135-2-200107170-00010 · Cited by 1,438
Research works printed but not linked: 1
- Cameron C, 1999, CMAJ, 160, 1165
Cited by
Harrell (2015) · Springer Series in Statistics · doi:10.1007/978-3-319-19425-7 · Cited by 6,878
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388
“External validation may use participant data collected by the same investigators, typically using the same predictor and outcome definitions and measurements, but sampled from a later period (temporal or narrow validation); by other investigators in another hospital or country, sometimes using different definitions an…”
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Metadata
- Authors
- Reilly, B. M., & Evans, A. T.
- Affiliations
- Rush University Medical Center (US)
- ORCID
- Reilly, Brendan M.
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2006-02-07
- Volume
- 144
- Issue
- 3
- Pages
- 201-209
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2124704085
- Cited by
- 794
- References
- 80
- Citations by year
- 2026: 15 · 2025: 21 · 2024: 28 · 2023: 40 · 2022: 32 · 2021: 54 · 2020: 42 · 2019: 36 · 2018: 40 · 2017: 51 · 2016: 41 · 2015: 61 · 2014: 41 · 2013: 58 · 2012: 50
- FWCI
- 25.18
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- UCLA - Biblioteca de Ciencias de la Salud (submittedVersion)
- Topics
- Health Sciences › Medicine › Family Practice › Clinical Reasoning and Diagnostic Skills
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- clinical prediction rules, clinical decision rules, impact analysis, evidence-based medicine, evidence standards, outcome measures, sample size determination, efficacy versus effectiveness, clinical translation
- MeSH terms
- Decision Support Techniques, Patient Care, Humans, Predictive Value of Tests, Evidence-Based Medicine, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions. https://app.cochat.ai/papers/10.7326/0003-4819-144-3-200602070-00009
- The paper
- Reilly, B. M., & Evans, A. T. (2006). Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions. Annals of Internal Medicine. https://doi.org/10.7326/0003-4819-144-3-200602070-00009
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.