Development and validation of risk prediction equations to estimate future risk of blindness and lower limb amputation in patients with diabetes: cohort study
Open accessAbstract
STUDY QUESTION: Is it possible to develop and externally validate risk prediction equations to estimate the 10 year risk of blindness and lower limb amputation in patients with diabetes aged 25-84 years? METHODS: This was a prospective cohort study using routinely collected data from general practices in England contributing to the QResearch and Clinical Practice Research Datalink (CPRD) databases during the study period 1998-2014. The equations were developed using 763 QResearch practices (n=454,575 patients with diabetes) and validated in 254 different QResearch practices (n=142,419) and 357 CPRD practices (n=206,050). Cox proportional hazards models were used to derive separate risk equations for blindness and amputation in men and women that could be evaluated at 10 years. Measures of calibration and discrimination were calculated in the two validation cohorts. STUDY ANSWER AND LIMITATIONS: Risk prediction equations to quantify absolute risk of blindness and amputation in men and women with diabetes have been developed and externally validated. In the QResearch derivation cohort, 4822 new cases of lower limb amputation and 8063 new cases of blindness occurred during follow-up. The risk equations were well calibrated in both validation cohorts. Discrimination was good in men in the external CPRD cohort for amputation (D statistic 1.69, Harrell's C statistic 0.77) and blindness (D statistic 1.40, Harrell's C statistic 0.73), with similar results in women and in the QResearch validation cohort. The algorithms are based on variables that patients are likely to know or that are routinely recorded in general practice computer systems. They can be used to identify patients at high risk for prevention or further assessment. Limitations include lack of formally adjudicated outcomes, information bias, and missing data. WHAT THIS STUDY ADDS: Patients with type 1 or type 2 diabetes are at increased risk of blindness and amputation but generally do not have accurate assessments of the magnitude of their individual risks. The new algorithms calculate the absolute risk of developing these complications over a 10 year period in patients with diabetes, taking account of their individual risk factors. FUNDING, COMPETING INTERESTS, DATA SHARING: JH-C is co-director of QResearch, a not for profit organisation which is a joint partnership between the University of Nottingham and Egton Medical Information Systems, and is also a paid director of ClinRisk Ltd. CC is a paid consultant statistician for ClinRisk Ltd.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
These calculators estimate 10-year risk of microvascular and macrovascular complications.
“These calculators can be used to predict 10-year risk for microvascular and macrovascular complications.”
From Type 1 diabetes (2018) · cites this paper as [129]
The Discussion says the paper's equations predict 10-year risks of blindness and lower-limb amputation in people with diabetes. It does not say these calculators predict macrovascular complications; the Introduction describes cardiovascular and stroke risk as assessed by other tools. (LLM, conf=0.88)
Experimental — accuracy not yet measuredThe QDiabetes and QRISK3 online tools were built using prospective general practice data from 803,044 people with diabetes, including 44,440 with type 1 diabetes.
“Two examples are the QDiabetes129 and QRISK3130 web calculators that were developed with a prospective general practice dataset of 803 044 people with diabetes (44 440 with type 1 diabetes).”
From Type 1 diabetes (2018) · cites this paper as [129]
The Results (Overall study population) give 454,575 patients in the derivation cohort and 142,419 and 206,050 in two validation cohorts, totaling 803,044. Table 1 (Baseline characteristics) gives 44,430 patients with type 1 diabetes across those cohorts, not 44,440. The web calculator is documented, but the 803,044 figure includes validation patients rather than only patients used to develop its equations. (LLM, conf=0.90)
Experimental — accuracy not yet measuredResearch has found further non-glycaemic factors that raise complication risk.
“Several studies127–130 have identified additional non-glycaemic risk factors for the development of complications.”
From Type 1 diabetes (2018) · cites this paper as [129]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
(1993) · New England Journal of Medicine · doi:10.1056/nejm199309303291401 · Cited by 24,667 · see citing works
Multiple Imputation for Nonresponse in Surveys
Rubin, D. B. (1987) · Wiley Series in Probability and Statistics · doi:10.1002/9780470316696 · Cited by 20,701 · see citing works
Nishimura, R., Osako, T., Nishiyama, Y., Tashima, R., Nakano, M., Fujisue, M., Toyozumi, Y., & Arima, N. (2013) · Oncology · doi:10.1159/000353099 · Cited by 13,239 · see citing works
Missing data: our view of the state of the art
Schafer, J. L., & Graham, J. W. (2002) · Psychological Methods · doi:10.1037/1082-989x.7.2.147 · Cited by 11,638 · see citing works
Multiple imputation using chained equations: Issues and guidance for practice
White, I. R., Royston, P., & Wood, A. M. (2011) · Statistics in Medicine · doi:10.1002/sim.4067 · Cited by 9,792 · see citing works
Stratton, I. M. (2000) · BMJ · doi:10.1136/bmj.321.7258.405 · Cited by 9,059 · see citing works
Turner, R. C., Fox, C., Matthews, McElroy, H. H., Cull, C. A., Holman, R. R., Neil, P., Hadden, D. R., Wright, D., Manley, E., et al. (1998) · The Lancet · Cited by 8,419 · see citing works
2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2013.11.005 · Cited by 4,945 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · BMJ · doi:10.1136/bmj.g7594 · Cited by 4,388 · see citing works
Gupta, A., & Uribarri, J. (2016) · Hormone Research in Paediatrics · doi:10.1159/000444053 · Cited by 4,019 · see citing works
Goff, D. C., Lloyd-Jones, D. M., Bennett, G., Coady, S., D’Agostino, R. B., Gibbons, R., Greenland, P., Lackland, D. T., Levy, D., O’Donnell, C. J., et al. (2014) · Circulation · doi:10.1161/01.cir.0000437741.48606.98 · Cited by 3,697 · see citing works
Evaluating the Yield of Medical Tests
Harrell, F. E. (1982) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.1982.03320430047030 · Cited by 3,490 · see citing works
Collins, P. G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · British Journal of Cancer · doi:10.1038/bjc.2014.639 · Cited by 3,312 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. G. M. (2015) · Annals of Internal Medicine · doi:10.7326/m14-0697 · Cited by 3,267 · see citing works
Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2
Hippisley-Cox, J., Coupland, C., Vinogradova, Y., Robson, J., Minhas, R., Sheikh, A., & Brindle, P. (2008) · BMJ · doi:10.1136/bmj.39609.449676.25 · Cited by 1,459 · see citing works
Self-citationThe use of fractional polynomials to model continuous risk variables in epidemiology
Royston, P., Ambler, G., & Sauerbrei, W. (1999) · International Journal of Epidemiology · doi:10.1093/ije/28.5.964 · Cited by 1,066 · see citing works
Stratton, I. M., Kohner, E. M., Aldington, S. J., Turner, R. C., Holman, R. R., Manley, S. E., & Matthews, D. R. (2001) · Diabetologia · doi:10.1007/s001250051594 · Cited by 1,019 · see citing works
Using the outcome for imputation of missing predictor values was preferred
Moons, K. G. M., Donders, R. A. R. T., Stijnen, T., & Harrell, F. E. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.01.009 · Cited by 994 · see citing works
Management of Diabetic Retinopathy
Mohamed, Q. A., Gillies, M. C., & Wong, T. Y. (2007) · JAMA · doi:10.1001/jama.298.8.902 · Cited by 905 · see citing works
Hyperglycemie and Microvascular and Macrovascular Disease in Diabetes
Klein, R. (1995) · Diabetes Care · doi:10.2337/diacare.18.2.258 · Cited by 849 · see citing works
Prevalence of Depression in Adults With Diabetes: An epidemiological evaluation
Gavard, J. A., Lustman, P. J., & Clouse, R. E. (1993) · Diabetes Care · doi:10.2337/diacare.16.8.1167 · Cited by 624 · see citing works
Palmer, A. J., Roze, S., Valentine, W. J., Minshall, M. E., FOOS, V., Lurati, F. M., Lammert, M., & Spinas, G. A. (2004) · Current Medical Research and Opinion · doi:10.1185/030079904x1980 · Cited by 523 · see citing works
Estimating Utility Values for Health States of Type 2 Diabetic Patients Using the EQ-5D (UKPDS 62)
Clarke, P., Gray, A. M., & Holman, R. R. (2002) · Medical Decision Making · doi:10.1177/027298902400448902 · Cited by 512 · see citing works
A new measure of prognostic separation in survival data
Royston, P., & Sauerbrei, W. (2004) · Statistics in Medicine · doi:10.1002/sim.1621 · Cited by 450 · see citing works
Estimating Utility Values for Health States of Type 2 Diabetic Patients Using the EQ-5D (UKPDS 62)
Clarke, P., Gray, A. M., & Holman, R. R. (2002) · Medical Decision Making · doi:10.1177/0272989x0202200412 · Cited by 425 · see citing works
Hippisley‐Cox, J., Coupland, C. A. C., Robson, J. P., Sheikh, A., & Brindle, P. M. (2009) · BMJ · doi:10.1136/bmj.b880 · Cited by 370 · see citing works
Self-citationEvaluating the yield of medical tests
Harrell, F. E. (1982) · JAMA · doi:10.1001/jama.247.18.2543 · Cited by 330 · see citing works
Explained Variation for Survival Models
Royston, P. (2006) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0600600105 · Cited by 319 · see citing works
Screening for type 2 diabetes: literature review and economic modelling
Waugh, N., Scotland, G., McNamee, P., Gillett, M., Brennan, A., Goyder, E., Williams, R., & John, A. (2007) · Health Technology Assessment · doi:10.3310/hta11170 · Cited by 298 · see citing works
Collins, P. G. S., & Altman, D. G. (2012) · BMJ · doi:10.1136/bmj.e4181 · Cited by 245 · see citing works
Validation of the IMS CORE Diabetes Model
McEwan, P. C., FOOS, V., Palmer, J. L., Lamotte, M., Lloyd, A., & Grant, D. J. W. (2014) · Value in Health · doi:10.1016/j.jval.2014.07.007 · Cited by 236 · see citing works
Moss, S. E., Klein, R., & Klein, R. (1999) · Diabetes Care · doi:10.2337/diacare.22.6.951 · Cited by 197 · see citing works
A review and critique of modelling in prioritising and designing screening programmes
Karnon, J. D., Goyder, E., Tappenden, P., McPhie, S., Towers, I., Brazier, J. E., & Madan, J. J. (2007) · Health Technology Assessment · doi:10.3310/hta11520 · Cited by 184 · see citing works
Hippisley‐Cox, J., & Coupland, C. A. C. (2010) · BMC Family Practice · doi:10.1186/1471-2296-11-49 · Cited by 178 · see citing works
Self-citationHippisley-Cox, J., Coupland, C., & Brindle, P. (2013) · British medical journal · doi:10.1136/bmj.f2573 · Cited by 164 · see citing works
Self-citationRith-Najarian, S. J., Branchaud, C., Beaulieu, O., Gohdes, D., Simonson, G., & Mazze, R. (1998) · PubMed · Cited by 148 · see citing works
Diabetes Mellitus and Nontraumatic Lower Extremity Amputation in Black and White Americans
Resnick, H. E., Valsania, P., & Phillips, C. L. (1999) · Archives of Internal Medicine · doi:10.1001/archinte.159.20.2470 · Cited by 138 · see citing works
MANAGEMENT OF DIABETIC RETINOPATHY: A SYSTEMATIC REVIEW
Goodwin, P. (2008) · Evidence-Based Ophthalmology · doi:10.1097/ieb.0b013e31815f0c98 · Cited by 137 · see citing works
Patout, C. A., Birke, J. A., Horswell, R., et al. (2000) · Diabetes Care · doi:10.2337/diacare.23.9.1339 · Cited by 118 · see citing works
Intensive Diabetes Therapy and Ocular Surgery in Type 1 Diabetes
The DCCT/EDIC Research Group (2015) · New England Journal of Medicine · doi:10.1056/nejmoa1409463 · Cited by 110 · see citing works
Imputation is beneficial for handling missing data in predictive models
Steyerberg, E. W., & van Veen, M. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.003 · Cited by 109 · see citing works
Hippisley-Cox, J., & Pringle, M. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16765-2 · Cited by 108 · see citing works
Self-citationSources, uses, strengths and limitations of data collected in primary care in England.
Majeed, A. (2004) · PubMed · Cited by 105 · see citing works
Ethnicity and Risk of Diabetes-Related Lower Extremity Amputation
Leggetter, S., Chaturvedi, N., Fuller, J., & Edmonds, M. E. (2002) · Archives of Internal Medicine · doi:10.1001/archinte.162.1.73 · Cited by 90 · see citing works
The Quality and Outcomes Framework 2009/2010
Champion, G. (2009) · Practice Nursing · doi:10.12968/pnur.2009.20.4.41202 · Cited by 65 · see citing works
Bachmann, M. O., & Nelson, S. J. (1998) · Journal of Epidemiology & Community Health · doi:10.1136/jech.52.1.45 · Cited by 63 · see citing works
External validation of QDSCORE® for predicting the 10‐year risk of developing Type 2 diabetes
Collins, P. G. S., & Altman, D. G. (2011) · Diabetic Medicine · doi:10.1111/j.1464-5491.2011.03237.x · Cited by 61 · see citing works
Mueller, E., Maxion‐Bergemann, S., Gultyaev, D., Walzer, S., Freemantle, N., Mathieu, C., Bolinder, B., Gerber, R. A., Kvasz, M., & Bergemann, R. (2006) · Diabetes Technology & Therapeutics · doi:10.1089/dia.2006.8.219 · Cited by 55 · see citing works
HbA1c and Lower-Extremity Amputation Risk in Low-Income Patients With Diabetes
Zhao, W., Katzmarzyk, P. T., Horswell, R. L., Wang, Y., Johnson, J. L., Heymsfield, S. B., Cefalu, W. T., Ryan, D. H., & Hu, G. (2013) · Diabetes Care · doi:10.2337/dc13-0437 · Cited by 48 · see citing works
Linking environmental effects to health impacts: a computer modelling approach for air pollution
Mindell, J. S., & Barrowcliffe, R. (2005) · Journal of Epidemiology & Community Health · doi:10.1136/jech.2005.036897 · Cited by 37 · see citing works
Barth, J. H., Marshall, S. M., & Watson, I. D. (2008) · Diabetic Medicine · doi:10.1111/j.1464-5491.2008.02436.x · Cited by 30 · see citing works
How patients with diabetes perceive their risk for major complications.
Meltzer, D. O., & Egleston, B. L. (2000) · PubMed · Cited by 27 · see citing works
PRM58 Long-Term Validation of the IMS CORE Diabetes Model in Type 1 and Type 2 Diabetes
FOOS, V., Palmer, J. L., Grant, D., Lloyd, A., Lamotte, M., & McEwan, P. (2012) · Value in Health · doi:10.1016/j.jval.2012.08.1521 · Cited by 10 · see citing works
(2010) · Drug and Therapeutics Bulletin · doi:10.1136/dtb.2009.12.0059 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [33]Hosmer D, Lemeshow S, May S. Applied survival analysis: regression modelling of time to event data. Wiley, 2007.
Cited by
Sorted by citations
DiMeglio, L. A., Evans-Molina, C., & Oram, R. A. (2018) · The Lancet · doi:10.1016/s0140-6736(18)31320-5 · Cited by 1,561 · see citing works
Palmer, S. C., Tendal, B., Mustafa, R. A., Vandvik, P. O., Li, S., Hao, Q., Tunnicliffe, D., Ruospo, M., Natale, P., Saglimbene, V., et al. (2021) · BMJ · doi:10.1136/bmj.m4573 · Cited by 679 · see citing works
- Authors
- Hippisley-Cox, J., & Coupland, C.
- Affiliations
- University of Nottingham (GB)
- ORCID
- Coupland, C.
- Venue
- BMJ
- Publisher
- BMJ
- Published
- 2015-11-11
- Volume
- 351
- Issue
- nov11 1
- Pages
- h5441-h5441
- ISSN
- 1756-1833 (ISSN-L)
- PMC
- PMC4641884
- OpenAlex
- W2136816386
- Cited by
- 63
- References
- 57
- Citations by year
- 2026: 3 · 2025: 3 · 2024: 8 · 2023: 2 · 2022: 9 · 2021: 9 · 2020: 4 · 2019: 6 · 2018: 8 · 2017: 4 · 2016: 6 · 2015: 1
- FWCI
- 5.49
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- BMJ (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Repository@Nottingham (University of Nottingham) (publishedVersion, cc-by)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- Nottingham ePrints (University of Nottingham) (submittedVersion, cc-by)
- Topics
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetic Foot Ulcer Assessment and Management
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes Management and Research
- Health Sciences › Medicine › Endocrinology, Diabetes and Metabolism › Diabetes, Cardiovascular Risks, and Lipoproteins
- Keywords
- diabetes mellitus, blindness, lower limb amputation, risk prediction equations, absolute risk estimation, type 1 diabetes, type 2 diabetes, diabetes complications, risk factors, sex differences, external validation, routine clinical data, Clinical Practice Research Datalink, general practice records, risk stratification, discrimination and calibration, information bias, missing data
- MeSH terms
- Adult, Aged, Aged, 80 and over, Algorithms, Amputation, Surgical, Blindness, Diabetes Mellitus, Type 1, Diabetes Mellitus, Type 2, England, Female, Humans, Male, Middle Aged, Predictive Value of Tests, Prospective Studies, Proportional Hazards Models, Risk Assessment, Lower Extremity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
