Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study
Open accessAbstract
OBJECTIVE: To identify existing prediction models for the risk of development of type 2 diabetes and to externally validate them in a large independent cohort. DATA SOURCES: Systematic search of English, German, and Dutch literature in PubMed until February 2011 to identify prediction models for diabetes. DESIGN: Performance of the models was assessed in terms of discrimination (C statistic) and calibration (calibration plots and Hosmer-Lemeshow test).The validation study was a prospective cohort study, with a case cohort study in a random subcohort. SETTING: Models were applied to the Dutch cohort of the European Prospective Investigation into Cancer and Nutrition cohort study (EPIC-NL). PARTICIPANTS: 38,379 people aged 20-70 with no diabetes at baseline, 2506 of whom made up the random subcohort. OUTCOME MEASURE: Incident type 2 diabetes. RESULTS: The review identified 16 studies containing 25 prediction models. We considered 12 models as basic because they were based on variables that can be assessed non-invasively and 13 models as extended because they additionally included conventional biomarkers such as glucose concentration. During a median follow-up of 10.2 years there were 924 cases in the full EPIC-NL cohort and 79 in the random subcohort. The C statistic for the basic models ranged from 0.74 (95% confidence interval 0.73 to 0.75) to 0.84 (0.82 to 0.85) for risk at 7.5 years. For prediction models including biomarkers the C statistic ranged from 0.81 (0.80 to 0.83) to 0.93 (0.92 to 0.94). Most prediction models overestimated the observed risk of diabetes, particularly at higher observed risks. After adjustment for differences in incidence of diabetes, calibration improved considerably. CONCLUSIONS: Most basic prediction models can identify people at high risk of developing diabetes in a time frame of five to 10 years. Models including biomarkers classified cases slightly better than basic ones. Most models overestimated the actual risk of diabetes. Existing prediction models therefore perform well to identify those at high risk, but cannot sufficiently quantify actual risk of future diabetes.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Most professional organizations recommended a two-stage strategy, starting with first-stage selection of people at higher risk by a questionnaire or risk score and followed by second-stage testing for conventional biomarkers such as FPG, HbA1c, or OGTT.
“Most of them recommended a two-stage strategy, starting with the first-stage of selection of people at higher risk by a questionnaire or risk score and followed by the second-stage of testing for conventional biomarkers such as FPG, HbA1c, or OGTT [12,39].”
From A Novel Testing Model for Opportunistic Screening of Pre-Diabetes and Diabetes among U.S. Adults (2015) · cites this paper as [39]
The Introduction (page not supplied) says European guidelines and the International Diabetes Federation recommend a questionnaire or risk score to identify people at high risk; it describes a different screening recommendation from the American Diabetes Association. The paper discusses models incorporating biomarkers, but our copy does not say that most professional organizations recommend the claimed two-stage sequence or subsequent FPG, HbA1c, or OGTT testing. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Reduction in the incidence of type 2 diabetes with lifestyle intervention or metformin
(2002) · New England Journal of Medicine · doi:10.1056/nejmoa012512 · Cited by 19,011 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gotzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · BMJ · doi:10.1136/bmj.b2700 · Cited by 17,648 · see citing works
Liberati, A., Altman, D. G., Tetzlaff, J., Mulrow, C., Gøtzsche, P. C., Ioannidis, J. P. A., Clarke, M., Devereaux, P. J., Kleijnen, J., & Moher, D. (2009) · Annals of Internal Medicine · doi:10.7326/0003-4819-151-4-200908180-00136 · Cited by 10,349 · see citing works
Sterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299 · see citing works
Diabetes Prevention Program Research Group (2009) · The Lancet · doi:10.1016/s0140-6736(09)61457-4 · Cited by 3,020 · see citing works
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · Bulletin of the World Health Organization · doi:10.2471/blt.07.045120 · Cited by 1,632 · see citing works
Survey non-response in the Netherlands: Effects on prevalence estimates and associations
Van Loon, A. (2003) · Annals of Epidemiology · doi:10.1016/s1047-2797(02)00257-0 · Cited by 423 · see citing works
Risk assessment tools for identifying individuals at risk of developing type 2 diabetes
Buijsse, B., Simmons, R. K., Griffin, S. J., & Schulze, M. B. (2011) · Epidemiologic Reviews · doi:10.1093/epirev/mxq019 · Cited by 297 · see citing works
The InterAct Consortium (2011) · Diabetologia · doi:10.1007/s00125-011-2182-9 · Cited by 208 · see citing works
Lin, J.-W., Chang, Y.-C., Li, H.-Y., Chien, Y.-F., Wu, M.-Y., Tsai, R.-Y., Hsieh, Y.-C., Chen, Y.-J., Hwang, J.-J., & Chuang, L.-M. (2009) · Diabetes Care · doi:10.2337/dc09-0694 · Cited by 60 · see citing works
Flagged first, then by citations
Research works printed but not linked: 47
- Wild S, Roglic G, Green A, Sicree R, King H. Global prevalence of diabetes: estimates for the year 2000 and projections for 2030. Diabetes Care 2004;27:1047-53.
- Alberti KG, Zimmet P, Shaw J. International Diabetes Federation: a consensus on type 2 diabetes prevention. Diabet Med 2007;24:451-63.
- Collins GS, Mallett S, Omar O, Yu LM. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting. BMC Med 2011;9:103.
- Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. BMJ 2011;343:d7163.
- Mann DM, Bertoni AG, Shimbo D, Carnethon MR, Chen H, Jenny NS, et al. Comparative validity of 3 diabetes mellitus risk prediction scoring models in a multiethnic US cohort: the Multi-Ethnic Study of Atherosclerosis. Am J Epidemiol 2010;171:980-8.
- Moons KG, Royston P, Vergouwe Y, Grobbee DE, Altman DG. Prognosis and prognostic research: what, why, and how? BMJ 2009;338:b375.
- Altman DG, Vergouwe Y, Royston P, Moons KG. Prognosis and prognostic research: validating a prognostic model. BMJ 2009;338:b605.
- Collins GS, Moons KG. Comparing risk prediction models. BMJ 2012;344:e3186.
- Siontis GC, Tzoulaki I, Siontis KC, Ioannidis JP. Comparisons of established risk prediction models for cardiovascular disease: systematic review. BMJ 2012;344:e3318.
- Reilly BM, Evans AT. Translating clinical research into clinical practice: impact of using prediction rules to make decisions. Ann Intern Med 2006;144:201-9.
- Schmid R, Vollenweider P, Bastardot F, Waeber G, Marques-Vidal P. Validation of 7 type 2 diabetes mellitus risk scores in a population-based cohort: CoLaus study. Arch Intern Med 2012;172:188-9.
- Steyerberg EW, Vickers AJ, Cook NR, Gerds T, Gonen M, Obuchowski N, et al. Assessing the performance of prediction models: a framework for traditional and novel measures. Epidemiology 2010;21:128-38.
- Beulens JW, Monninkhof EM, Verschuren WM, van der Schouw YT, Smit J, Ocke MC, et al. Cohort profile: the EPIC-NL study. Int J Epidemiol 2010;39:1170-8.
- Volovics A, van den Brandt P. Methods for the analysis of case-cohort studies. Biometric J 1997;39:195-214.
- Sluijs I, van der A D, Beulens JW, Spijkerman AM, Ros MM, Grobbee DE, et al. Ascertainment and verification of diabetes in the EPIC-NL study. Neth J Med 2010;68:333-9.
- Alssema M, Vistisen D, Heymans MW, Nijpels G, Glumer C, Zimmet PZ, et al. The Evaluation of Screening and Early Detection Strategies for Type 2 Diabetes and Impaired Glucose Tolerance (DETECT-2) update of the Finnish diabetes risk score for prediction of incident type 2 diabetes. Diabetologia 2011;54:1004-12.
- Kahn HS, Cheng YJ, Thompson TJ, Imperatore G, Gregg EW. Two risk-scoring systems for predicting incident diabetes mellitus in US adults age 45 to 64 years. Ann Intern Med 2009;150:741-51.
- Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361-87.
- Van Houwelingen HC. Validation, calibration, revision and combination of prognostic survival models. Stat Med 2000;19:3401-15.
- Liddell FD. Simple exact analysis of the standardised mortality ratio. J Epidemiol Community Health 1984;38:85-8.
- Vergouwe Y, Moons KG, Steyerberg EW. External validity of risk models: use of benchmark values to disentangle a case-mix effect from incorrect coefficients. Am J Epidemiol 2010;172:971-80.
- Janssen KJ, Moons KG, Kalkman CJ, Grobbee DE, Vergouwe Y. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol 2008;61:76-86.
- Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 2006;59:1087-91.
- Marshall A, Altman DG, Royston P, Holder RL. Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study. BMC Med Res Methodol 2010;10:7.
- Rathmann W, Kowall B, Heier M, Herder C, Holle R, Thorand B, et al. Prediction models for incident type 2 diabetes mellitus in the older population: KORA S4/F4 cohort study. Diabet Med 2010;27:1116-23.
- Chen L, Magliano DJ, Balkau B, Colagiuri S, Zimmet PZ, Tonkin AM, et al. AUSDRISK: an Australian Type 2 Diabetes Risk Assessment Tool based on demographic, lifestyle and simple anthropometric measures. Med J Aust 2010;192:197-202.
- Rosella LC, Manuel DG, Burchill C, Stukel TA. A population-based risk algorithm for the development of diabetes: development and validation of the Diabetes Population Risk Tool (DPoRT). J Epidemiol Community Health 2011;65:613-20.
- Hippisley-Cox J, Coupland C, Robson J, Sheikh A, Brindle P. Predicting risk of type 2 diabetes in England and Wales: prospective derivation and validation of QDScore. BMJ 2009;338:b880.
- Balkau B, Lange C, Fezeu L, Tichet J, de Lauzon-Guillain B, Czernichow S, et al. Predicting diabetes: clinical, biological, and genetic approaches: data from the Epidemiological Study on the Insulin Resistance Syndrome (DESIR). Diabetes Care 2008;31:2056-61.
- Schulze MB, Hoffmann K, Boeing H, Linseisen J, Rohrmann S, Mohlig M, et al. An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 2007;30:510-5.
- Simmons RK, Harding AH, Wareham NJ, Griffin SJ. Do simple questions about diet and physical activity help to identify those at risk of type 2 diabetes? Diabet Med 2007;24:830-5.
- Wilson PW, Meigs JB, Sullivan L, Fox CS, Nathan DM, D'Agostino RB Sr. Prediction of incident diabetes mellitus in middle-aged adults: the Framingham Offspring Study. Arch Intern Med 2007;167:1068-74.
- Lindstrom J, Tuomilehto J. The diabetes risk score: a practical tool to predict type 2 diabetes risk. Diabetes Care 2003;26:725-31.
- Stern MP, Williams K, Haffner SM. Identification of persons at high risk for type 2 diabetes mellitus: do we need the oral glucose tolerance test? Ann Intern Med 2002;136:575-81.
- Stern MP, Morales PA, Valdez RA, Monterrosa A, Haffner SM, Mitchell BD, et al. Predicting diabetes. Moving beyond impaired glucose tolerance. Diabetes 1993;42:706-14.
- Wannamethee SG, Papacosta O, Whincup PH, Thomas MC, Carson C, Lawlor DA, et al. The potential for a two-stage diabetes risk algorithm combining non-laboratory-based scores with subsequent routine non-fasting blood tests: results from prospective studies in older men and women. Diabet Med 2011;28:23-30.
- Joseph J, Svartberg J, Njolstad I, Schirmer H. Incidence of and risk factors for type-2 diabetes in a general population: the Tromso Study. Scand J Public Health 2010;38:768-75.
- Von Eckardstein A, Schulte H, Assmann G. Risk for diabetes mellitus in middle-aged Caucasian male participants of the PROCAM study: implications for the definition of impaired fasting glucose by the American Diabetes Association. Prospective Cardiovascular Munster. J Clin Endocrinol Metab 2000;85:3101-8.
- D'Agostino RB Sr, Grundy S, Sullivan LM, Wilson P. Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 2001;286:180-7.
- Herder C, Baumert J, Zierer A, Roden M, Meisinger C, Karakas M, et al. Immunological and cardiometabolic risk factors in the prediction of type 2 diabetes and coronary events: MONICA/KORA Augsburg case-cohort study. PLoS One 2011;6:e19852.
- Van Dieren S, Nothlings U, van der Schouw YT, Spijkerman AM, Rutten GE, van der AD, et al. Non-fasting lipids and risk of cardiovascular disease in patients with diabetes mellitus. Diabetologia 2011;54:73-7.
- Harrell FE Jr, Lee KL, Califf RM, Pryor DB, Rosati RA. Regression modelling strategies for improved prognostic prediction. Stat Med 1984;3:143-52.
- McGeechan K, Macaskill P, Irwig L, Liew G, Wong TY. Assessing new biomarkers and predictive models for use in clinical practice: a clinician's guide. Arch Intern Med 2008;168:2304-10.
- Rathmann W, Martin S, Haastert B, Icks A, Holle R, Lowel H, et al. Performance of screening questionnaires and risk scores for undiagnosed diabetes: the KORA Survey 2000. Arch Intern Med 2005;165:436-41.
- Schmidt MI, Duncan BB, Bang H, Pankow JS, Ballantyne CM, Golden SH, et al. Identifying individuals at high risk for diabetes: the Atherosclerosis Risk in Communities study. Diabetes Care 2005;28:2013-8.
- Griffin SJ, Little PS, Hales CN, Kinmonth AL, Wareham NJ. Diabetes risk score: towards earlier detection of type 2 diabetes in general practice. Diabetes Metab Res Rev 2000;16:164-71.
- Rahman M, Simmons RK, Harding AH, Wareham NJ, Griffin SJ. A simple risk score identifies individuals at high risk of developing type 2 diabetes: a prospective cohort study. Fam Pract 2008;25:191-6. RESEARCH on 26 June 2023 by guest. Protected by copyright. http://www.bmj.com/ BMJ: first published as 10.1136/bmj.e5900 on 18 September 2012. Downloaded from
Cited by
Sorted by citations
Moons, K. G. M., Altman, D. G., Reitsma, J. B., Ioannidis, J. P. A., Macaskill, P., Steyerberg, E. W., Vickers, A. J., Ransohoff, D. F., & Collins, G. S. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0698 · Cited by 5,147 · see citing works
Self-citationPrediction models for cardiovascular disease risk in the general population: systematic review
Damen, J. A. A. G., Hooft, L., Schuit, E., Debray, T. P. A., Collins, G. S., Tzoulaki, I., Lassale, C. M., Siontis, G. C. M., Chiocchia, V., Roberts, C., et al. (2016) · BMJ · doi:10.1136/bmj.i2416 · Cited by 935 · see citing works
Self-citationSafaei, M., Sundararajan, E. A., Driss, M., Boulila, W., & Shapi'i, A. (2021) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2021.104754 · Cited by 720 · see citing works
Artificial intelligence in diabetes management: Advancements, opportunities, and challenges
Guan, Z., Li, H., Liu, R., Cai, C., Liu, Y., Li, J., Wang, X., Huang, S., Wu, L., Liu, D., et al. (2023) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101213 · Cited by 287 · see citing works
“35 However, ML is a promising tool that can maximize predictive performance compared with conventional statistical models.”
MethodStefan, N., & Schulze, M. B. (2023) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(23)00086-4 · Cited by 215 · see citing works
Rubin, K. H., Glintborg, D., Nybo, M., Abrahamsen, B., & Andersen, M. (2017) · The Journal of Clinical Endocrinology & Metabolism · doi:10.1210/jc.2017-01354 · Cited by 195 · see citing works
Lynam, A. L., Dennis, J. M., Owen, K. R., Oram, R. A., Jones, A. G., Shields, B. M., & Ferrat, L. A. (2020) · Diagnostic and Prognostic Research · doi:10.1186/s41512-020-00075-2 · Cited by 142 · see citing works
Meta‐analysis and aggregation of multiple published prediction models
Debray, T. P. A., Koffijberg, H., Nieboer, D., Vergouwe, Y., Steyerberg, E. W., & Moons, K. G. M. (2014) · Statistics in Medicine · doi:10.1002/sim.6080 · Cited by 73 · see citing works
Self-citationWilkinson, L., Yi, N., Mehta, T., Judd, S., & Garvey, W. T. (2020) · PLOS Medicine · doi:10.1371/journal.pmed.1003232 · Cited by 47 · see citing works
Lucaroni, F., Cicciarella Modica, D., Macino, M., Palombi, L., Abbondanzieri, A., Agosti, G., Biondi, G., Morciano, L., & Vinci, A. (2019) · BMJ Open · doi:10.1136/bmjopen-2019-030234 · Cited by 42 · see citing works
A Novel Testing Model for Opportunistic Screening of Pre-Diabetes and Diabetes among U.S. Adults
Zhang, Y., Hu, G., Zhang, L., Mayo, R., & Chen, L. (2015) · PLOS ONE · doi:10.1371/journal.pone.0120382 · Cited by 41 · see citing works
“We did not include data before NHANES 2005 because the OGTT tests (with a 75-gram glucose drink) were only added to the NHANES since the 2005 – 2006 survey.”
BackgroundInfluentialAbbasi, A., Corpeleijn, E., Gansevoort, R. T., Gans, R. O. B., Struck, J., Schulte, J., Hillege, H. L., van der Harst, P., Stolk, R. P., Navis, G., et al. (2014) · Diabetologia · doi:10.1007/s00125-014-3278-9 · Cited by 27 · see citing works
Self-citationFico, G., Hernanzez, L., Cancela, J., Dagliati, A., Sacchi, L., Martinez-Millana, A., Posada, J., Manero, L., Verdú, J., Facchinetti, A., et al. (2019) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-019-0887-8 · Cited by 22 · see citing works
- Authors
- Abbasi, A., Peelen, L. M., Corpeleijn, E., van der Schouw, Y. T., Stolk, R. P., Spijkerman, A. M. W., van der A, D. L., Moons, K. G. M., Navis, G., Bakker, S. J. L., & Beulens, J. W. J.
- Affiliations
- University Medical Center Groningen (NL)
- University Medical Center Utrecht (NL)
- National Institute for Public Health and the Environment (NL)
- Corresponding author
- Abbasi, A.
- ORCID
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2012-09-18
- Volume
- 345
- Issue
- sep18 2
- Pages
- e5900-e5900
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC3445426
- OpenAlex
- W2133466960
- Cited by
- 321
- References
- 10
- Citations by year
- 2026: 15 · 2025: 11 · 2024: 13 · 2023: 13 · 2022: 19 · 2021: 29 · 2020: 42 · 2019: 26 · 2018: 18 · 2017: 34 · 2016: 28 · 2015: 27 · 2014: 21 · 2013: 24 · 2012: 1
- FWCI
- 18.73
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc
- Locations
- BMJ (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Pure Amsterdam UMC (publishedVersion, other-oa)
- Amsterdam UMC (VU Amsterdam) - Institutional Repository (publishedVersion, other-oa)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Rivm (National Institute for Public Health and the Environment) (submittedVersion)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- Nierstichting
- Diabetes Fonds
- Topics
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes, Cardiovascular Risks, and Lipoproteins
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes Management and Research
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- type 2 diabetes, diabetes risk prediction models, external validation, discrimination (statistics), calibration, biomarkers, glucose concentration, EPIC-NL cohort, new-onset diabetes, risk overestimation
- MeSH terms
- Adult, Aged, Blood Glucose, Data Interpretation, Statistical, Diabetes Mellitus, Type 2, Europe, Female, Humans, Male, Middle Aged, Risk Factors, Reproducibility of Results, Models, Statistical, Cohort Studies, Biomarkers, Incidence, Confidence Intervals, MEDLINE, Risk Assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
