PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration
Open accessThe DOI matches its Crossref record; 194 of the 201 references it cites were found in a registry or an index.
Abstract
Prediction models in health care use predictors to estimate for an individual the probability that a condition or disease is already present (diagnostic model) or will occur in the future (prognostic model). Publications on prediction models have become more common in recent years, and competing prediction models frequently exist for the same outcome or target population. Health care providers, guideline developers, and policymakers are often unsure which model to use or recommend, and in which persons or settings. Hence, systematic reviews of these studies are increasingly demanded, required, and performed. A key part of a systematic review of prediction models is examination of risk of bias and applicability to the intended population and setting. To help reviewers with this process, the authors developed PROBAST (Prediction model Risk Of Bias ASsessment Tool) for studies developing, validating, or updating (for example, extending) prediction models, both diagnostic and prognostic. PROBAST was developed through a consensus process involving a group of experts in the field. It includes 20 signaling questions across 4 domains (participants, predictors, outcome, and analysis). This explanation and elaboration document describes the rationale for including each domain and signaling question and guides researchers, reviewers, readers, and guideline developers in how to use them to assess risk of bias and applicability concerns. All concepts are illustrated with published examples across different topics. The latest version of the PROBAST checklist, accompanying documents, and filled-in examples can be downloaded from www.probast.org.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: the PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & for the PRISMA Group (2009) · BMJ · doi:10.1136/bmj.b2535 · Cited by 83,823
Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & the PRISMA Group* (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00135 · Cited by 37,554
The Cochrane Collaboration's tool for assessing risk of bias in randomised trials
Higgins, J. P. T., Altman, D. G., Gotzsche, P. C., Juni, P., Moher, D., Oxman, A. D., Savovic, J., Schulz, K. F., Weeks, L., Sterne, J. A. C., et al. (2011) · BMJ · doi:10.1136/bmj.d5928 · Cited by 33,817
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000100 · Cited by 28,164
Self-citationROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
Self-citationQUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Self-citationLiberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349
Self-citationMeasuring the Accuracy of Diagnostic Systems
Swets, J. A. (1988) · Science · doi:10.1126/science.3287615 · Cited by 10,058
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792
A simulation study of the number of events per variable in logistic regression analysis
Peduzzi, P., Concato, J., Kemper, E., Holford, T. R., & Feinstein, A. R. (1996) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(96)00236-3 · Cited by 9,655
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299
Decision curve analysis: a novel method for evaluating prediction models
Vickers, A. J., & Elkin, E. B. (2006) · Medical Decision Making · doi:10.1177/0272989x06295361 · Cited by 6,341
Cochrane Handbook for Systematic Reviews of Interventions
Tarsilla, M. (2010) · Journal of MultiDisciplinary Evaluation · doi:10.56645/jmde.v6i14.284 · Cited by 5,178
Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures
Steyerberg, E. W., Vickers, A. J., Cook, N. R., Gerds, T., Gonen, M., Obuchowski, N., Pencina, M. J., & Kattan, M. W. (2010) · Epidemiology · doi:10.1097/ede.0b013e3181c30fb2 · Cited by 5,170
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147
Self-citationRelaxing the Rule of Ten Events per Variable in Logistic and Cox Regression
Vittinghoff, E., & McCulloch, C. E. (2007) · American Journal of Epidemiology · doi:10.1093/aje/kwk052 · Cited by 4,359
Schafer, J. L. (1999) · Statistical Methods in Medical Research · doi:10.1177/096228029900800102 · Cited by 3,949
On the practice of dichotomization of quantitative variables.
MacCallum, R. C., Zhang, S., Preacher, K. J., & Rucker, D. D. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.1.19 · Cited by 3,538
Reitsma, J. B., Glas, A. S., Rutjes, A. W. S., Scholten, R. J. P. M., Bossuyt, P. M., & Zwinderman, A. H. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.02.022 · Cited by 3,369
Self-citationCollins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,293
Self-citationAssessing Bias in Studies of Prognostic Factors
Hayden, J. A., van der Windt, D. A., Cartwright, J. L., Côté, P., & Bombardier, C. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-4-201302190-00009 · Cited by 3,287
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267
Self-citationMcInnes, M. D. F., Moher, D., Thombs, B. D., McGrath, T. A., Bossuyt, P. M., and the PRISMA-DTA Group, Clifford, T., Cohen, J. F., Deeks, J. J., Gatsonis, C., et al. (2018) · JAMA · doi:10.1001/jama.2017.19163 · Cited by 3,210
Self-citationPROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citationInternal validation of predictive models
Steyerberg, E. W., Harrell, F. E., Borsboom, G. J. J. M., Eijkemans, M. J. C., Vergouwe, Y., & Habbema, J. D. F. (2001) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(01)00341-9 · Cited by 2,746
The cost of dichotomising continuous variables
Altman, D. G., & Royston, P. (2006) · BMJ · doi:10.1136/bmj.332.7549.1080 · Cited by 2,688
Data Resource Profile: Clinical Practice Research Datalink (CPRD)
Herrett, E., Gallagher, A. M., Bhaskaran, K., Forbes, H., Mathur, R., van Staa, T., & Smeeth, L. (2015) · International Journal of Epidemiology · doi:10.1093/ije/dyv098 · Cited by 2,667
Review: a gentle introduction to imputation of missing values.
Donders, A. R. T., van der Heijden, G. J. M. G., Stijnen, T., & Moons, K. G. M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.014 · Cited by 2,582
Self-citationBias in error estimation when using cross-validation for model selection
Varma, S., & Simon, R. (2006) · BMC Bioinformatics · doi:10.1186/1471-2105-7-91 · Cited by 2,485
Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers
Pencina, M. J., D'Agostino, R. B., & Steyerberg, E. W. (2011) · Statistics in Medicine · doi:10.1002/sim.4085 · Cited by 2,453
Dichotomizing continuous predictors in multiple regression: a bad idea
Royston, P., Altman, D. G., & Sauerbrei, W. (2006) · Statistics in Medicine · doi:10.1002/sim.2331 · Cited by 2,312
ROBIS: A new tool to assess risk of bias in systematic reviews was developed
Whiting, P., Savović, J., Higgins, J. P. T., Caldwell, D. M., Reeves, B. C., Shea, B., Davies, P., Kleijnen, J., & Churchill, R. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.06.005 · Cited by 2,272
Self-citationPeduzzi, P., Concato, J., Feinstein, A. R., & Holford, T. R. (1995) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(95)00048-8 · Cited by 2,154
Moons, K. G. M., de Groot, J. A. H., Bouwmeester, W., Vergouwe, Y., Mallett, S., Altman, D. G., Reitsma, J. B., & Collins, G. S. (2014) · PLoS Medicine · doi:10.1371/journal.pmed.1001744 · Cited by 2,018
Self-citationThe Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981
Empirical evidence of design-related bias in studies of diagnostic tests.
Lijmer, J. G. (1999) · JAMA · doi:10.1001/jama.282.11.1061 · Cited by 1,861
Problems of Spectrum and Bias in Evaluating the Efficacy of Diagnostic Tests
Ransohoff, D. F., & Feinstein, A. R. (1978) · New England Journal of Medicine · doi:10.1056/nejm197810262991705 · Cited by 1,734
Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research
Steyerberg, E. W., Moons, K. G. M., van der Windt, D. A., Hayden, J. A., Perel, P., Schroter, S., Riley, R. D., Hemingway, H., Altman, D. G., & for the PROGRESS Group (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001381 · Cited by 1,508
Self-citationA hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations
Rutter, C. M., & Gatsonis, C. A. (2001) · Statistics in Medicine · doi:10.1002/sim.942 · Cited by 1,505
Prognosis and prognostic research: validating a prognostic model
Altman, D. G., Vergouwe, Y., Royston, P., & Moons, K. G. M. (2009) · BMJ · doi:10.1136/bmj.b605 · Cited by 1,436
Self-citationRiley, R. D., Snell, K. I., Ensor, J., Burke, D. L., Harrell Jr, F. E., Moons, K. G., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7992 · Cited by 1,409
Self-citationClinical Prediction Rules: Applications and Methodological Standards
Wasson, J. H., Sox, H. C., Neff, R. K., & Goldman, L. (1985) · New England Journal of Medicine · doi:10.1056/nejm198509263131306 · Cited by 1,357
Prognosis and prognostic research: what, why, and how?
Moons, K. G. M., Royston, P., Vergouwe, Y., Grobbee, D. E., & Altman, D. G. (2009) · BMJ · doi:10.1136/bmj.b375 · Cited by 1,337
Self-citationHiggins (2002) · doi:10.1002/(issn)1097-0258 · Cited by 1,248
Risk prediction models: II. External validation, model updating, and impact assessment
Moons, K. G. M., Kengne, A. P., Grobbee, D. E., Royston, P., Vergouwe, Y., Altman, D. G., & Woodward, M. (2012) · Heart · doi:10.1136/heartjnl-2011-301247 · Cited by 1,195
Self-citationPrognosis and prognostic research: Developing a prognostic model
Royston, P., Moons, K. G. M., Altman, D. G., & Vergouwe, Y. (2009) · BMJ · doi:10.1136/bmj.b604 · Cited by 1,172
Self-citationAssessing the Generalizability of Prognostic Information
Justice, A. C., Covinsky, K. E., & Berlin, J. A. (1999) · Annals of Internal Medicine · doi:10.7326/0003-4819-130-6-199903160-00016 · Cited by 1,160
Dangers of Using "Optimal" Cutpoints in the Evaluation of Prognostic Factors
Altman, D. G., Lausen, B., Sauerbrei, W., & Schumacher, M. (1994) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/86.11.829 · Cited by 1,143
Jaeschke, R. (1994) · JAMA · doi:10.1001/jama.1994.03510290071040 · Cited by 1,049
Using the outcome for imputation of missing predictor values was preferred
Moons, K. G. M., Donders, R. A. R. T., Stijnen, T., & Harrell, F. E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.009 · Cited by 994
Self-citationRegression Models in Clinical Studies: Determining Relationships Between Predictors and Response
Harre, F. E., Lee, K. L., & Pollock, B. G. (1988) · JNCI Journal of the National Cancer Institute · doi:10.1093/jnci/80.15.1198 · Cited by 957
Prognosis and prognostic research: application and impact of prognostic models in clinical practice
Moons, K. G. M., Altman, D. G., Vergouwe, Y., & Royston, P. (2009) · BMJ · doi:10.1136/bmj.b606 · Cited by 934
Self-citationConfounding and Collapsibility in Causal Inference
Greenland, S., Pearl, J., & Robins, J. M. (1999) · Statistical Science · doi:10.1214/ss/1009211805 · Cited by 933
Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis
Sun, G.-W., Shook, T. L., & Kay, G. L. (1996) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(96)00025-x · Cited by 933
A calibration hierarchy for risk models was defined: from utopia to empirical data
Van Calster, B., Nieboer, D., Vergouwe, Y., De Cock, B., Pencina, M. J., & Steyerberg, E. W. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.12.005 · Cited by 910
Moons, K. G. M., Kengne, A. P., Woodward, M., Royston, P., Vergouwe, Y., Altman, D. G., & Grobbee, D. E. (2012) · Heart · doi:10.1136/heartjnl-2011-301246 · Cited by 889
Self-citationMarshall, A., Altman, D. G., Holder, R. L., & Royston, P. (2009) · BMC Medical Research Methodology · doi:10.1186/1471-2288-9-57 · Cited by 874
Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research
Riley, R. D., Hayden, J. A., Steyerberg, E. W., Moons, K. G. M., Abrams, K., Kyzas, P. A., Malats, N., Briggs, A., Schroter, S., Altman, D. G., et al. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001380 · Cited by 823
Self-citationModern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints
van der Ploeg, T., Austin, P. C., & Steyerberg, E. W. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-137 · Cited by 809
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8<1081::aid-sim406>3.0.co;2-a · Cited by 802
Reilly, B. M., & Evans, A. T. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-3-200602070-00009 · Cited by 794
Laupacis, A. (1997) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1997.03540300056034 · Cited by 793
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2000) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(20000430)19:8<1059::aid-sim412>3.0.co;2-0 · Cited by 785
A unification of models for meta-analysis of diagnostic accuracy studies
Harbord, R. M., Deeks, J. J., Egger, M., Whiting, P., & Sterne, J. A. C. (2007) · Biostatistics · doi:10.1093/biostatistics/kxl004 · Cited by 751
Self-citationCollins, G. S., de Groot, J. A., Dutton, S., Omar, O., Shanyinde, M., Tajar, A., Voysey, M., Wharton, R., Yu, L.-M., Moons, K. G., et al. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-40 · Cited by 729
Self-citationAssessment of diagnostic tests when disease verification is subject to selection bias
Begg, C. B., & Greenes, R. A. (1983) · Biometrics · doi:10.2307/2530820 · Cited by 708
GAIL, M. H., WIEAND, S., & PIANTADOSI, S. (1984) · Biometrika · doi:10.1093/biomet/71.3.431 · Cited by 698
Collins, G. S., Ogundimu, E. O., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6787 · Cited by 665
How to develop a more accurate risk prediction model when there are few events
Pavlou, M., Ambler, G., Seaman, S. R., Guttmann, O., Elliott, P., King, M., & Omar, R. Z. (2015) · BMJ · doi:10.1136/bmj.h3868 · Cited by 664
Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes
Hemingway, H., Croft, P., Perel, P., Hayden, J. A., Abrams, K., Timmis, A., Briggs, A., Udumyan, R., Moons, K. G. M., Steyerberg, E. W., et al. (2013) · BMJ · doi:10.1136/bmj.e5595 · Cited by 663
Self-citationA guide to systematic review and meta-analysis of prediction model performance
Debray, T. P. A., Damen, J. A. A. G., Snell, K. I. E., Ensor, J., Hooft, L., Reitsma, J. B., Riley, R. D., & Moons, K. G. M. (2017) · BMJ · doi:10.1136/bmj.i6460 · Cited by 657
Self-citationVergouwe, Y., Steyerberg, E. W., Eijkemans, M. J. C., & Habbema, J. D. F. (2005) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.06.017 · Cited by 636
Prognostic Models With Competing Risks: Methods and Application to Coronary Risk Prediction
Wolbers, M., Koller, M. T., Witteman, J. C. M., & Steyerberg, E. W. (2009) · Epidemiology · doi:10.1097/ede.0b013e3181a39056 · Cited by 625
Steyerberg, E. W., Bleeker, S. E., Moll, H. A., Grobbee, D. E., & Moons, K. G. M. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00047-7 · Cited by 611
Self-citationEvidence of bias and variation in diagnostic accuracy studies
Rutjes, A. W. S., Reitsma, J. B., Di Nisio, M., Smidt, N., van Rijn, J. C., & Bossuyt, P. M. M. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050090 · Cited by 611
Self-citationReporting and Methods in Clinical Prediction Research: A Systematic Review
Bouwmeester, W., Zuithoff, N. P. A., Mallett, S., Geerlings, M. I., Vergouwe, Y., Steyerberg, E. W., Altman, D. G., & Moons, K. G. M. (2012) · PLoS Medicine · doi:10.1371/journal.pmed.1001221 · Cited by 600
Self-citationDebray, T. P. A., Vergouwe, Y., Koffijberg, H., Nieboer, D., Steyerberg, E. W., & Moons, K. G. M. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2014.06.018 · Cited by 597
Self-citationNo rationale for 1 variable per 10 events criterion for binary logistic regression analysis
van Smeden, M., de Groot, J. A. H., Moons, K. G. M., Collins, G. S., Altman, D. G., Eijkemans, M. J. C., & Reitsma, J. B. (2016) · BMC Medical Research Methodology · doi:10.1186/s12874-016-0267-3 · Cited by 595
Self-citationMissing covariate data in medical research: To impute is better than to ignore
Janssen, K. J. M., Donders, A. R. T., Harrell, F. E., Vergouwe, Y., Chen, Q., Grobbee, D. E., & Moons, K. G. M. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.12.008 · Cited by 592
Self-citationRiley, R. D., Ensor, J., Snell, K. I. E., Debray, T. P. A., Altman, D. G., Moons, K. G. M., & Collins, G. S. (2016) · BMJ · doi:10.1136/bmj.i3140 · Cited by 585
Self-citationStatistical aspects of prognostic factor studies in oncology
Simon, R., & Altman, D. (1994) · British Journal of Cancer · doi:10.1038/bjc.1994.192 · Cited by 582
Steyerberg, E. W., Eijkemans, M. J. C., Harrell, F. E., & Habbema, J. D. F. (2001) · Medical Decision Making · doi:10.1177/0272989x0102100106 · Cited by 550
Adequate sample size for developing prediction models is not simply related to events per variable
Ogundimu, E. O., Altman, D. G., & Collins, G. S. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.02.031 · Cited by 546
Groenwold, R. H. H., White, I. R., Donders, A. R. T., Carpenter, J. R., Altman, D. G., & Moons, K. G. M. (2012) · Canadian Medical Association Journal · doi:10.1503/cmaj.110977 · Cited by 538
Self-citationEarly Prediction of Outcome of Activities of Daily Living After Stroke: A Systematic Review
Veerbeek, J. M., Kwakkel, G., van Wegen, E. E. H., Ket, J. C. F., & Heymans, M. W. (2011) · Stroke · doi:10.1161/strokeaha.110.604090 · Cited by 537
Evaluation of diagnostic tests when there is no gold standard. A review of methods
Rutjes, A., Reitsma, J., Coomarasamy, A., Khan, K., & Bossuyt, P. (2007) · Health Technology Assessment · doi:10.3310/hta11500 · Cited by 524
Self-citationAustin, P. C., & Steyerberg, E. W. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280214558972 · Cited by 516
Prognosis research strategy (PROGRESS) 4: Stratified medicine research
Hingorani, A. D., Windt, D. A. V. D., Riley, R. D., Abrams, K., Moons, K. G. M., Steyerberg, E. W., Schroter, S., Sauerbrei, W., Altman, D. G., Hemingway, H., et al. (2013) · BMJ · doi:10.1136/bmj.e5793 · Cited by 510
Self-citationCase–Control and Two-Gate Designs in Diagnostic Accuracy Studies
Rutjes, A. W., Reitsma, J. B., Vandenbroucke, J. P., Glas, A. S., & Bossuyt, P. M. (2005) · Clinical Chemistry · doi:10.1373/clinchem.2005.048595 · Cited by 501
Self-citationBiases in the assessment of diagnostic tests
Begg, C. B. (1987) · Statistics in Medicine · doi:10.1002/sim.4780060402 · Cited by 496
Collins, G. S., Mallett, S., Omar, O., & Yu, L.-M. (2011) · BMC Medicine · doi:10.1186/1741-7015-9-103 · Cited by 484
Self-citationPeters, S. A. E., den Ruijter, H. M., Bots, M. L., & Moons, K. G. M. (2012) · Heart · doi:10.1136/heartjnl-2011-300747 · Cited by 415
Self-citationReitsma, J. B., Rutjes, A. W. S., Khan, K. S., Coomarasamy, A., & Bossuyt, P. M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.02.005 · Cited by 405
Self-citationSOX, H. C. (1986) · Annals of Internal Medicine · doi:10.7326/0003-4819-104-1-60 · Cited by 364
Austin, P. C., & Steyerberg, E. W. (2014) · Statistics in Medicine · doi:10.1002/sim.5941 · Cited by 362
Hippisley-Cox, J., & Coupland, C. (2012) · BMJ · doi:10.1136/bmj.e3427 · Cited by 351
Minimum sample size for developing a multivariable prediction model: Part I – Continuous outcomes
Riley, R. D., Snell, K. I. E., Ensor, J., Burke, D. L., Harrell, F. E., Moons, K. G. M., & Collins, G. S. (2019) · Statistics in Medicine · doi:10.1002/sim.7993 · Cited by 351
Self-citationGeersing, G.-J., Bouwmeester, W., Zuithoff, P., Spijker, R., Leeflang, M., & Moons, K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0032844 · Cited by 350
Self-citationLeeflang, M. M. G., Moons, K. G. M., Reitsma, J. B., & Zwinderman, A. H. (2008) · Clinical Chemistry · doi:10.1373/clinchem.2007.096032 · Cited by 346
Self-citationSystematic review of prognostic models in traumatic brain injury
Perel, P., Edwards, P., Wentz, R., & Roberts, I. (2006) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-6-38 · Cited by 344
No role for quality scores in systematic reviews of diagnostic accuracy studies
Whiting, P., Harbord, R., & Kleijnen, J. (2005) · BMC Medical Research Methodology · doi:10.1186/1471-2288-5-19 · Cited by 337
Self-citationCourvoisier, D. S., Combescure, C., Agoritsas, T., Gayet-Ageron, A., & Perneger, T. V. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.012 · Cited by 320
Assessment of the accuracy of diagnostic tests: the cross-sectional study
Knottnerus, J. A., & Muris, J. W. (2003) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(03)00206-3 · Cited by 312
Comparisons of established risk prediction models for cardiovascular disease: systematic review
Siontis, G. C. M., Tzoulaki, I., Siontis, K. C., & Ioannidis, J. P. A. (2012) · BMJ · doi:10.1136/bmj.e3318 · Cited by 300
Assessing calibration of prognostic risk scores
Crowson, C. S., Atkinson, E. J., & Therneau, T. M. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280213497434 · Cited by 289
Development and validation of a prediction model with missing predictor data: a practical approach
Vergouwe, Y., Royston, P., Moons, K. G. M., & Altman, D. G. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.03.017 · Cited by 287
Self-citationA systematic review classifies sources of bias and variation in diagnostic test accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., & Mallett, S. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.05.014 · Cited by 283
Self-citationRagland, D. R. (1992) · Epidemiology · doi:10.1097/00001648-199209000-00009 · Cited by 280
A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes
Debray, T. P., Damen, J. A., Riley, R. D., Snell, K., Reitsma, J. B., Hooft, L., Collins, G. S., & Moons, K. G. (2019) · Statistical Methods in Medical Research · doi:10.1177/0962280218785504 · Cited by 278
Self-citationAssessment of Claims of Improved Prediction Beyond the Framingham Risk Score
Tzoulaki, I. (2009) · JAMA · doi:10.1001/jama.2009.1757 · Cited by 276
Performance of methods for meta-analysis of diagnostic test accuracy with few studies or sparse data
Takwoingi, Y., Guo, B., Riley, R. D., & Deeks, J. J. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280215592269 · Cited by 257
Self-citationEvaluation of diagnostic tests without gold standards
Hui, S. L., & Zhou, X. H. (1998) · Statistical Methods in Medical Research · doi:10.1177/096228029800700404 · Cited by 252
Systematic Review of Prognostic Models in Patients with Acute Stroke
Counsell, C., & Dennis, M. (2001) · Cerebrovascular Diseases · doi:10.1159/000047699 · Cited by 250
Marshall, A., Altman, D. G., Royston, P., & Holder, R. L. (2010) · BMC Medical Research Methodology · doi:10.1186/1471-2288-10-7 · Cited by 247
Vergouwe, Y., Moons, K. G. M., & Steyerberg, E. W. (2010) · American Journal of Epidemiology · doi:10.1093/aje/kwq223 · Cited by 243
Self-citationUsing a combination of reference tests to assess the accuracy of a new diagnostic test
Alonzo, T. A., & Pepe, M. S. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19991130)18:22<2987::aid-sim205>3.0.co;2-b · Cited by 240
Latent Class Models in Diagnostic Studies When There is No Reference Standard--A Systematic Review
van Smeden, M., Naaktgeboren, C. A., Reitsma, J. B., Moons, K. G. M., & de Groot, J. A. H. (2014) · American Journal of Epidemiology · doi:10.1093/aje/kwt286 · Cited by 217
Self-citationAssessment of radiologic tests: control of bias and other design considerations.
Begg, C. B., & McNeil, B. J. (1988) · Radiology · doi:10.1148/radiology.167.2.3357976 · Cited by 213
A method for checking regression models in survival analysis based on the risk score
Gr�nnesby, J. K., & Borgan, �. (1996) · Lifetime Data Analysis · doi:10.1007/bf00127305 · Cited by 209
Reporting methods in studies developing prognostic models in cancer: a review
Mallett, S., Royston, P., Dutton, S., Waters, R., & Altman, D. G. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-20 · Cited by 206
Self-citationPrognostic models in obstetrics: available, but far from applicable
Kleinrouweler, C. E., Cheong-See, F. M., Collins, G. S., Kwee, A., Thangaratinam, S., Khan, K. S., Mol, B. W. J., Pajkrt, E., Moons, K. G. M., & Schuit, E. (2016) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2015.06.013 · Cited by 193
Self-citationJolani, S., Debray, T. P. A., Koffijberg, H., van Buuren, S., & Moons, K. G. M. (2015) · Statistics in Medicine · doi:10.1002/sim.6451 · Cited by 192
Self-citationEmpirical evidence of the importance of comparative studies of diagnostic test accuracy
Takwoingi, Y., Leeflang, M. M. G., & Deeks, J. J. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-158-7-201304020-00006 · Cited by 187
Collins, G. S., Ogundimu, E. O., Cook, J. A., Manach, Y. L., & Altman, D. G. (2016) · Statistics in Medicine · doi:10.1002/sim.6986 · Cited by 185
Searching for Clinical Prediction Rules in MEDLINE
Ingui, B. J., & Rogers, M. A. M. (2001) · Journal of the American Medical Informatics Association · doi:10.1136/jamia.2001.0080391 · Cited by 184
Risk Prediction Models for Mortality in Ambulatory Patients With Heart Failure: A Systematic Review
Alba, A. C., Agoritsas, T., Jankowski, M., Courvoisier, D., Walter, S. D., Guyatt, G. H., & Ross, H. J. (2013) · Circulation: Heart Failure · doi:10.1161/circheartfailure.112.000043 · Cited by 181
Meads, C., Ahmed, I., & Riley, R. D. (2012) · Breast Cancer Research and Treatment · doi:10.1007/s10549-011-1818-2 · Cited by 177
Self-citationPrediction models in reproductive medicine: a critical appraisal†
Leushuis, E., van der Steeg, J. W., Steures, P., Bossuyt, P. M. M., Eijkemans, M. J. C., van der Veen, F., Mol, B. W. J., & Hompes, P. G. A. (2009) · Human Reproduction Update · doi:10.1093/humupd/dmp013 · Cited by 173
Kengne, A. P., Beulens, J. W., Peelen, L. M., Moons, K. G., van der Schouw, Y. T., Schulze, M. B., Spijkerman, A. M., Griffin, S. J., Grobbee, D. E., Palla, L., et al. (2014) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(13)70103-7 · Cited by 170
Self-citationPrediction of Mortality in Very Premature Infants: A Systematic Review of Prediction Models
Medlock, S. K., Ravelli, A. C. J., Tamminga, P., Mol, B. W., & Abu–Hanna, A. (2011) · PLoS ONE · doi:10.1371/journal.pone.0023441 · Cited by 168
Dealing with Missing Predictor Values When Applying Clinical Prediction Models
Janssen, K. J. M., Vergouwe, Y., Donders, A. R. T., Harrell, F. E., Chen, Q., Grobbee, D. E., & Moons, K. G. M. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.115345 · Cited by 168
Self-citationRietveld, R. P., Riet, G. T., Bindels, P. J. E., Sloos, J. H., & van Weert, H. C. P. M. (2004) · BMJ · doi:10.1136/bmj.38128.631319.ae · Cited by 168
Bertens, L. C. M., Broekhuizen, B. D. L., Naaktgeboren, C. A., Rutten, F. H., Hoes, A. W., van Mourik, Y., Moons, K. G. M., & Reitsma, J. B. (2013) · PLoS Medicine · doi:10.1371/journal.pmed.1001531 · Cited by 158
Self-citationSnell, K. I. E., Ensor, J., Debray, T. P. A., Moons, K. G. M., & Riley, R. D. (2017) · Statistical Methods in Medical Research · doi:10.1177/0962280217705678 · Cited by 152
Self-citationThe Wells Rule Does Not Adequately Rule Out Deep Venous Thrombosis in Primary Care Patients
Oudega, R., Hoes, A. W., & Moons, K. G. M. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-2-200507190-00008 · Cited by 148
Self-citationDiagnostic accuracy studies: how to report and analyse inconclusive test results
Shinkins, B., Thompson, M., Mallett, S., & Perera, R. (2013) · BMJ · doi:10.1136/bmj.f2778 · Cited by 146
Self-citationPrognostic Models: A Methodological Framework and Review of Models for Breast Cancer
Altman, D. G. (2009) · Cancer Investigation · doi:10.1080/07357900802572110 · Cited by 142
Case-cohort design in practice – experiences from the MORGAM Project
Kulathinal, S., Karvanen, J., Saarela, O., Kuulasmaa, K., & the MORGAM Project (2007) · Epidemiologic Perspectives & Innovations · doi:10.1186/1742-5573-4-15 · Cited by 141
Measuring the effects of imaging: An evaluative framework
Mackenzie, R., & Dixon, A. K. (1995) · Clinical Radiology · doi:10.1016/s0009-9260(05)83184-8 · Cited by 140
Advantages of the nested case-control design in diagnostic research
Biesheuvel, C. J., Vergouwe, Y., Oudega, R., Hoes, A. W., Grobbee, D. E., & Moons, K. G. M. (2008) · BMC Medical Research Methodology · doi:10.1186/1471-2288-8-48 · Cited by 139
Self-citationPredicting length of stay in thermal burns: A systematic review of prognostic factors
Hussain, A. K., & Dunn, K. (2013) · Burns · doi:10.1016/j.burns.2013.04.026 · Cited by 137
Performance of the original EuroSCORE
Siregar, S., Groenwold, R. H. H., de Heer, F., Bots, M. L., van der Graaf, Y., & van Herwerden, L. A. (2012) · European Journal of Cardio-Thoracic Surgery · doi:10.1093/ejcts/ezr285 · Cited by 135
Developing and validating risk prediction models in an individual participant data meta-analysis
Ahmed, I., Debray, T. P. A., Moons, K. G. M., & Riley, R. D. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-3 · Cited by 124
Self-citationLamain-de Ruiter, M., Kwee, A., Naaktgeboren, C. A., de Groot, I., Evers, I. M., Groenendaal, F., Hering, Y. R., Huisjes, A. J. M., Kirpestein, C., Monincx, W. M., et al. (2016) · BMJ · doi:10.1136/bmj.i4338 · Cited by 123
Self-citationWessler, B. S., Lana Lai, Y. H., Kramer, W., Cangelosi, M. J., Raman, G., Lutz, J. S., & Kent, D. M. (2015) · Circulation Cardiovascular Quality and Outcomes · doi:10.1161/circoutcomes.115.001693 · Cited by 121
Verification problems in diagnostic accuracy studies: consequences and solutions
de Groot, J. A. H., Bossuyt, P. M., Reitsma, J. B., Rutjes, A. W., Dendukuri, N., Janssen, K. J. M., & Moons, K. G. M. (2011) · BMJ · doi:10.1136/bmj.d4770 · Cited by 121
Self-citationDeveloping optimal search strategies for detecting sound clinical prediction studies in MEDLINE.
Wong, S. S.-L., Wilczynski, N. L., Haynes, R. B., & Ramkissoonsingh, R. (2003) · PubMed · Cited by 115
Framework for the impact analysis and implementation of Clinical Prediction Rules (CPRs)
(Members of the International Diagnostic and Prognosis Prediction (IDAPP) group), Wallace, E., Smith, S. M., Perera, R., Vaucher, P., McCowan, C., Collins, P. G. S., Verbakel, J. Y. J., Lakhanpaul, M., & Fahey, T. P. (2011) · BMC Medical Informatics and Decision Making · doi:10.1186/1472-6947-11-62 · Cited by 113
External Validation of the CRASH and IMPACT Prognostic Models in Severe Traumatic Brain Injury
Han, J. X., King, N. K. K., Neilson, S. J., Gandhi, M., & Ng, I. (2014) · Journal of Neurotrauma · doi:10.1089/neu.2013.3003 · Cited by 108
Value of composite reference standards in diagnostic research
Naaktgeboren, C. A., Bertens, L. C. M., Smeden, M. V., Groot, J. A. H. D., Moons, K. G. M., & Reitsma, J. B. (2013) · BMJ · doi:10.1136/bmj.f5605 · Cited by 105
Self-citationSynthesizing Evidence: Shifting the Focus From Individual Studies to the Body of Evidence
Murad, M. H., & Montori, V. M. (2013) · JAMA · doi:10.1001/jama.2013.5616 · Cited by 104
Perel, P., Prieto‐Merino, D., Shakur‐Still, H., Clayton, T. C., Lecky, F. E., Bouamra, O., Russell, R. J., Faulkner, M., Steyerberg, E. W., & Roberts, I. (2012) · BMJ · doi:10.1136/bmj.e5166 · Cited by 103
An empirical assessment of validation practices for molecular classifiers
Castaldi, P. J., Dahabreh, I. J., & Ioannidis, J. P. A. (2011) · Briefings in Bioinformatics · doi:10.1093/bib/bbq073 · Cited by 102
Tamariz, L. J., Eng, J., Segal, J. B., Krishnan, J. A., Bolger, D. T., Streiff, M. B., Jenckes, M. W., & Bass, E. B. (2004) · The American Journal of Medicine · doi:10.1016/j.amjmed.2004.04.021 · Cited by 96
Crowther, M. J., Look, M. P., & Riley, R. D. (2014) · Statistics in Medicine · doi:10.1002/sim.6191 · Cited by 95
Self-citationModeling conditional dependence between diagnostic tests: A multiple latent variable model
Dendukuri, N., Hadgu, A., & Wang, L. (2008) · Statistics in Medicine · doi:10.1002/sim.3470 · Cited by 94
Post hoc choice of cut points introduced bias to diagnostic research
Ewald, B. D. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2005.11.025 · Cited by 94
Schwartz, W. B., Wolfe, H. J., & Pauker, S. G. (1981) · New England Journal of Medicine · doi:10.1056/nejm198110153051604 · Cited by 93
Bivariate Random Effects Meta-Analysis of Diagnostic Studies Using Generalized Linear Mixed Models
Chu, H., Guo, H., & Zhou, Y. (2009) · Medical Decision Making · doi:10.1177/0272989x09353452 · Cited by 93
Ganna, A., Reilly, M., Fairé, U. D., Pedersen, N., Magnusson, P. K. E., & Ingelsson, E. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr374 · Cited by 93
The Impact of Clinical History on Mammographic Interpretations
Elmore, J. G. (1997) · JAMA · doi:10.1001/jama.1997.03540250057032 · Cited by 89
Risk Prediction Models for Incident Primary Cutaneous Melanoma: A Systematic Review
Vuong, K., McGeechan, K., Armstrong, B. K., & Cust, A. E. (2014) · JAMA Dermatology · doi:10.1001/jamadermatol.2013.8890 · Cited by 87
Snell, K. I. E., Hua, H., Debray, T. P. A., Ensor, J., Look, M. P., Moons, K. G. M., & Riley, R. D. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.05.009 · Cited by 84
Self-citationWalter, S. D. (1999) · Epidemiology · doi:10.1097/00001648-199901000-00012 · Cited by 83
Multidimensional prognostic indices for use in COPD patient care. A systematic review
van Dijk, W. D., van den Bemt, L., van den Haak-Rongen, S., Bischoff, E. W. M. A., van Weel, C., Veen, J. C. I. ’., & Schermer, T. R. (2011) · Respiratory Research · doi:10.1186/1465-9921-12-151 · Cited by 79
The influence of uninterpretability on the assessment of diagnostic tests
Begg, C. B., Greenes, R. A., & Iglewicz, B. (1986) · Journal of Chronic Diseases · doi:10.1016/0021-9681(86)90182-7 · Cited by 76
Riley, R. D., Ridley, G. F., Williams, K. J., Altman, D. G., Hayden, J. A., & de Vet, H. C. W. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.02.004 · Cited by 75
Self-citationAdaptation of Clinical Prediction Models for Application in Local Settings
Kappen, T. H., Vergouwe, Y., van Klei, W. A., van Wolfswinkel, L., Kalkman, C. J., & Moons, K. G. M. (2012) · Medical Decision Making · doi:10.1177/0272989x12439755 · Cited by 75
Self-citationWhen should we remain blind and when should our eyes remain open in diagnostic studies?
Moons, K. G. M., & Grobbee, D. E. (2002) · Journal of Clinical Epidemiology · doi:10.1016/s0895-4356(02)00408-0 · Cited by 75
Self-citationMeta‐analysis and aggregation of multiple published prediction models
Debray, T. P. A., Koffijberg, H., Nieboer, D., Vergouwe, Y., Steyerberg, E. W., & Moons, K. G. M. (2014) · Statistics in Medicine · doi:10.1002/sim.6080 · Cited by 73
Self-citationMultiple imputation for correcting verification bias
Harel, O., & Zhou, X. (2006) · Statistics in Medicine · doi:10.1002/sim.2494 · Cited by 72
The validation of cardiovascular risk scores for patients with type 2 diabetes mellitus
van der Leeuw, J., van Dieren, S., Beulens, J. W. J., Boeing, H., Spijkerman, A. M. W., van der Graaf, Y., van der A, D. L., Nöthlings, U., Visseren, F. L. J., Rutten, G. E. H. M., et al. (2014) · Heart · doi:10.1136/heartjnl-2014-306068 · Cited by 70
Self-citationGroenwold, R., Moons, K., Pajouheshnia, R., Altman, D., Collins, G., Debray, T. P. A., Reitsma, J., Riley, R., & Peelen, L. (2016) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2016.03.017 · Cited by 70
Self-citationDebray, T. P. A., Koffijberg, H., Vergouwe, Y., Moons, K. G. M., & Steyerberg, E. W. (2012) · Statistics in Medicine · doi:10.1002/sim.5412 · Cited by 67
Self-citationOverinterpretation of Clinical Applicability in Molecular Diagnostic Research
Lumbreras, B., Parker, L. A., Porta, M., Pollán, M., Ioannidis, J. P. A., & Hernández-Aguado, I. (2009) · Clinical Chemistry · doi:10.1373/clinchem.2008.121517 · Cited by 65
Prognosis research: Why is Dr. Lydgate still waiting?
Hemingway, H. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.005 · Cited by 62
Effects of Glasgow Outcome Scale Misclassification on Traumatic Brain Injury Clinical Trials
Lü, J., Murray, G. D., Steyerberg, E. W., Butcher, I., McHugh, G. S., Lingsma, H. F., Mushkudiani, N. A., Choi, S. C., Maas, A. I. R., & Marmarou, A. (2008) · Journal of Neurotrauma · doi:10.1089/neu.2007.0510 · Cited by 60
Multiple imputation to correct for partial verification bias revisited
de Groot, J. A. H., Janssen, K. J. M., Zwinderman, A. H., Moons, K. G. M., & Reitsma, J. B. (2008) · Statistics in Medicine · doi:10.1002/sim.3410 · Cited by 52
Self-citationDerivation and assessment of risk prediction models using case-cohort data
Sanderson, J., Thompson, S. G., White, I. R., Aspelund, T., & Pennells, L. A. M. (2013) · BMC Medical Research Methodology · doi:10.1186/1471-2288-13-113 · Cited by 51
Wynants, L., Vergouwe, Y., Huffel, S. V., Timmerman, D., & CALSTER, B. V. (2016) · Statistical Methods in Medical Research · doi:10.1177/0962280216668555 · Cited by 50
Unexpected predictor–outcome associations in clinical prediction research: causes and solutions
Schuit, E., Groenwold, R. H. H., Harrell, F. E., de Kort, W. L. A. M., Kwee, A., Mol, B. W., Riley, R. D., & Moons, K. G. M. (2013) · Canadian Medical Association Journal · doi:10.1503/cmaj.120812 · Cited by 49
Self-citationA new concordance measure for risk prediction models in external validation settings
van Klaveren, D., Gönen, M., Steyerberg, E. W., & Vergouwe, Y. (2016) · Statistics in Medicine · doi:10.1002/sim.6997 · Cited by 48
Evaluating Diagnostic Accuracy in the Face of Multiple Reference Standards
Naaktgeboren, C. A., de Groot, J. A. H., van Smeden, M., Moons, K. G. M., & Reitsma, J. B. (2013) · Annals of Internal Medicine · doi:10.7326/0003-4819-159-3-201308060-00009 · Cited by 42
Self-citationKeogh, C., Wallace, E., O’Brien, K. K., Murphy, P., Teljeur, C., McGrath, B., Smith, S. M., Doherty, N., Dimitrov, B. D., & Fahey, T. P. (2011) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.11.011 · Cited by 35
Adjusting for Partial Verification or Workup Bias in Meta-Analyses of Diagnostic Accuracy Studies
de Groot, J. A. H., Dendukuri, N., Janssen, K. J. M., Reitsma, J. B., Brophy, J. M., Joseph, L., Bossuyt, P. M., & Moons, K. G. M. (2012) · American Journal of Epidemiology · doi:10.1093/aje/kwr383 · Cited by 25
Self-citationvan Zaane, B., Vergouwe, Y., Donders, A. R. T., & Moons, K. G. M. (2012) · BMC Medical Research Methodology · doi:10.1186/1471-2288-12-166 · Cited by 21
Self-citationSteurer, J., Haller, C., Häuselmann, H., Brunner, F., & Bachmann, L. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0019994 · Cited by 21
Janssen, K. J. M., Siccama, I., Vergouwe, Y., Koffijberg, H., Debray, T. P. A., Keijzer, M., Grobbee, D. E., & Moons, K. G. M. (2012) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2011.08.011 · Cited by 18
Self-citationLü, J., Marmarou, A., & Lapane, O. B. O. T. I. I. K. L. (2011) · Journal of Neurotrauma · doi:10.1089/neu.2010.1746 · Cited by 18
Development of a Cardiovascular Risk Score for Use in Low- and Middle-Income Countries
Aslibekyan, S. W., Campos, H., Loucks, E. B., Linkletter, C. D., Ordovás, J. M., & Baylín, A. (2011) · Journal of Nutrition · doi:10.3945/jn.110.133140 · Cited by 10
Evaluation of diagnostic tests
Hlatky, M. A. (1986) · Journal of Chronic Diseases · doi:10.1016/0021-9681(86)90120-7 · Cited by 3
Multiple imputation of missing blood pressure covariates in survival analysis
van Buuren, S., Boshuizen, H. C., & Knook, D. L. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990330)18:6<681::aid-sim71>3.0.co;2-r
What do we mean by validating a prognostic model?
(2000) · doi:10.1002/(sici)1097-0258(20000229)19:4<453::aid-sim350>3.0.co;2-5
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4<361::aid-sim168>3.0.co;2-4
Research works printed but not linked: 2
- 10.1177/0962280218785504.30032705 · doi:10.1177/0962280218785504.30032705
- Wong SS, 2003, AMIA Annu Symp Proc, 728
Cited by
Collins, G. S., Moons, K. G. M., Dhiman, P., Riley, R. D., Beam, A. L., Van Calster, B., Ghassemi, M., Liu, X., Reitsma, J. B., van Smeden, M., et al. (2024) · BMJ · doi:10.1136/bmj-2023-078378 · Cited by 4,718
“Readers are referred to PROBAST 90 91 and the forthcoming PROBAST+AI 79 to assess the quality and risk of bias of prediction models (https://www.probast.org/).”
BackgroundSelf-citationWynants, L., Van Calster, B., Bonten, M. M., Collins, G. S., Debray, T. P., De Vos, M., Haller, M. C., Heinze, G., Moons, K. G., Riley, R. D., et al. (2020) · doi:10.1101/2020.03.24.20041020 · Cited by 3,070
Self-citationPROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies
Wolff, R. F., Moons, K. G. M., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., Mallett, S., & for the PROBAST Group† (2019) · Annals of Internal Medicine · doi:10.7326/m18-1376 · Cited by 2,984
Self-citationCalculating the sample size required for developing a clinical prediction model
Riley, R. D., Ensor, J., Snell, K. I. E., Harrell, F. E., Martin, G. P., Reitsma, J. B., Moons, K. G. M., Collins, G., & van Smeden, M. (2020) · BMJ · doi:10.1136/bmj.m441 · Cited by 2,917
Self-citationNagendran, M., Chen, Y., Lovejoy, C. A., Gordon, A. C., Komorowski, M., Harvey, H., Topol, E. J., Ioannidis, J. P. A., Collins, G. S., & Maruthappu, M. (2020) · BMJ · doi:10.1136/bmj.m689 · Cited by 1,152
Metadata
- Authors
- Moons, K. G. M., Wolff, R. F., Riley, R. D., Whiting, P. F., Westwood, M., Collins, G. S., Reitsma, J. B., Kleijnen, J., & Mallett, S.
- Affiliations
- Utrecht University (NL)
- Kleijnen Systematic Reviews (United Kingdom) (GB)
- Keele University (GB)
- University Hospitals Bristol NHS Foundation Trust (GB)
- Centre for Statistics in Medicine, University of Oxford, Oxford, United Kingdom (G.S.C.)
- Maastricht University (NL)
- Institute of Applied Health Research, National Institute for Health Research Birmingham Biomedical Research Centre, College of Medical and Dental Sciences, University of Birmingham, Birmingham, United Kingdom (S.M.)
- ORCID
- Venue
- Annals of Internal Medicine
- Publisher
- American College of Physicians
- Published
- 2019-01-01
- Volume
- 170
- Issue
- 1
- Pages
- W1-W33
- ISSN
- 0003-4819 (ISSN-L)
- 1539-3704
- OpenAlex
- W2907638671
- Cited by
- 1,669
- References
- 201
- Citations by year
- 2026: 359 · 2025: 330 · 2024: 253 · 2023: 236 · 2022: 193 · 2021: 175 · 2020: 96 · 2019: 24
- FWCI
- 56.12
- Type
- journal article
- Language
- en
- Open access
- Licence
- other-oa
- www.acpjournals.org/journal/aim/text-and-data-mining (tdm)
- Locations
- Annals of Internal Medicine (publishedVersion)
- PubMed (publishedVersion)
- Research Publications (Maastricht University) (publishedVersion, other-oa)
- Utrecht University Repository (Utrecht University) (publishedVersion, other-oa)
- University of Birmingham Research Portal (University of Birmingham) (publishedVersion)
- Bristol Research (University of Bristol) (publishedVersion, other-oa)
- Keele Research Repository (Keele University) (submittedVersion, cc-by)
- UCL Discovery (University College London) (submittedVersion)
- University of Birmingham Research Portal (University of Birmingham) (submittedVersion)
- Explore Bristol Research (submittedVersion, other-oa)
- Research Publications (Maastricht University) (submittedVersion)
- Utrecht University Repository (Utrecht University) (submittedVersion, other-oa)
- Funders
- National Institute for Health Research Birmingham Biomedical Research Centre at the University Hospitals Birmingham NHS Foundation Trust
- Nederlandse Organisatie voor Wetenschappelijk Onderzoek (ZONMW 918.10.615, ZONMW 912.08.004)
- National Institute for Health Research School for Primary Care Research (390)
- National Institute for Health Research Biomedical Research Centre, Oxford
- National Institute for Health Research (NIHR) Collaboration for Leadership in Applied Health Research and Care (CLAHRC) West at University Hospitals Bristol NHS Foundation Trust
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
- Keywords
- PROBAST, risk of bias assessment, applicability, prediction model, diagnostic model, prognostic model, model validation
- MeSH terms
- Systematic Reviews as Topic, Decision Support Techniques, Diagnosis, Humans, Prognosis, Research Design, Models, Statistical, Bias
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-01 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.