CoChat
Sign Up
Cited by 10Retinal Diseases and TreatmentsRetinal Imaging and AnalysisArtificial Intelligence in Healthcare

Identifying the severity of diabetic retinopathy by visual function measures using both traditional statistical methods and interpretable machine learning: a cross-sectional study

Open access
Wright, David M., Chakravarthy, Usha, Das, Radha, Graham, Katie W., Naskas, Timos T., Perais, Jennifer, Kee, Frank, Peto, Tunde +1 more·2023·Diabetologia
DOI 10.1007/s00125-023-06005-3PMC PMC10627908
Open this paper in CoChat — free accountOpen PDF

Abstract

Aims/hypothesis: To determine the extent to which diabetic retinopathy severity stage may be classified using machine learning (ML) and commonly used clinical measures of visual function together with age and sex. Methods: We measured the visual function of 1901 eyes from 1032 participants in the Northern Ireland Sensory Ageing Study, deriving 12 variables from nine visual function tests. Missing values were imputed using chained equations. Participants were divided into four groups using clinical measures and grading of ophthalmic images: no diabetes mellitus (no DM), diabetes but no diabetic retinopathy (DM no DR), diabetic retinopathy without diabetic macular oedema (DR no DMO) and diabetic retinopathy with DMO (DR with DMO). Ensemble ML models were fitted to classify group membership for three tasks, distinguishing (A) the DM no DR group from the no DM group; (B) the DR no DMO group from the DM no DR group; and (C) the DR with DMO group from the DR no DMO group. More conventional multiple logistic regression models were also fitted for comparison. An interpretable ML technique was used to rank the contribution of visual function variables to predictions and to disentangle associations between diabetic eye disease and visual function from artefacts of the data collection process. Results: The performance of the ensemble ML models was good across all three classification tasks, with accuracies of 0.92, 1.00 and 0.84, respectively, for tasks A–C, substantially exceeding the accuracies for logistic regression (0.84, 0.61 and 0.80, respectively). Reading index was highly ranked for tasks A and B, whereas near visual acuity and Moorfields chart acuity were important for task C. Microperimetry variables ranked highly for all three tasks, but this was partly due to a data artefact (a large proportion of missing values). Conclusions/interpretation: Ensemble ML models predicted status of diabetic eye disease with high accuracy using just age, sex and measures of visual function. Interpretable ML methods enabled us to identify profiles of visual function associated with different stages of diabetic eye disease, and to disentangle associations from artefacts of the data collection process. Together, these two techniques have great potential for developing prediction models using untidy real-world clinical data. Graphical Abstract:

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1007/s00125-023-06005-3

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Physiochemical indicator-based DR models were predominantly single-center (75%, 6/8), with only 2 multicenter studies conducted in the United States and Northern Ireland.

    “In contrast, physiochemical indicator-based models were predominantly single-center (75%, 6/8), with only 2 multicenter studies conducted in the United States and Northern Ireland [39, 40], reflecting a limited degree of external validation across populations.”

    From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [40]

    The Methods (Data collection; page unavailable) identify this as a Northern Ireland study that recruited from a population cohort and diabetes clinics. The paper does not establish that it was multicenter or report the claimed tally of six single-center and two multicenter studies. The Discussion (Strengths and limitations; page unavailable) says its models lacked external validation. (LLM, conf=0.82)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 38 · Cited by 10

Research works printed but not linked: 34

  1. [2]Vujosevic S Aldington SJ Silva P Screening for diabetic retinopathy: new perspectives and challenges Lancet Diabetes Endocrinol 2020 8 4 337 347 10.1016/S2213-8587(19)30411-5 32113513 · DOI · Paper page
  2. [3]Simó R Stitt AW Gardner TW Neurodegeneration in diabetic retinopathy: does it really matter? Diabetologia 2018 61 9 1902 1912 10.1007/s00125-018-4692-1 30030554 PMC6096638 · DOI
  3. [4]Zafar S Sachdeva M Frankfort BJ Channa R Retinal neurodegeneration as an early manifestation of diabetic eye disease and potential neuroprotective therapies Curr Diab Rep 2019 19 4 17 10.1007/s11892-019-1134-5 30806815 PMC7192364 · DOI
  4. [5]Montesano G Ometto G Higgins BE Evidence for structural and functional damage of the inner retina in diabetes with no diabetic retinopathy Invest Ophthalmol Vis Sci 2021 62 3 35 35 10.1167/iovs.62.3.35 33760040 PMC7995918 · DOI
  5. [6]Malik R Swanson WH Garway-Heath DF ‘Structure–function relationship’ in glaucoma: past thinking and current concepts Clin Exp Ophthalmol 2012 40 4 369 380 10.1111/j.1442-9071.2012.02770.x 22339936 PMC3693944 · DOI
  6. [7]Broman AT Munoz B Rodriguez J The impact of visual impairment and eye disease on vision-related quality of life in a Mexican-American population: proyecto VER Invest Ophthalmol Vis Sci 2002 43 11 3393 3398 12407148
  7. [8]McAnany JJ Park JC Liu K Contrast sensitivity is associated with outer-retina thickness in early-stage diabetic retinopathy Acta Ophthalmol 2020 98 2 e224 e231 10.1111/aos.14241 31517447 PMC7060819 · DOI
  8. [9]Montesano G Gervasoni A Ferri P Structure–function relationship in early diabetic retinopathy: a spatial correlation analysis with OCT and microperimetry Eye (Lond) 2017 31 6 931 939 10.1038/eye.2017.27 28257130 PMC5518836 · DOI
  9. [10]Boynton GE Stem MS Kwark L Jackson GR Farsiu S Gardner TW Multimodal characterization of proliferative diabetic retinopathy reveals alterations in outer retinal function and structure Ophthalmology 2015 122 5 957 967 10.1016/j.ophtha.2014.12.001 25601533 PMC4414692 · DOI
  10. [11]Joltikov KA de Castro VM Davila JR Multidimensional functional and structural evaluation reveals neuroretinal impairment in early diabetic retinopathy Invest Ophthalmol Vis Sci 2017 58 6 BIO277 BIO290 10.1167/iovs.17-21863 28973314 PMC5624741 · DOI
  11. [14]Neville CE Young IS Kee F Northern Ireland Cohort for the Longitudinal Study of Ageing (NICOLA): health assessment protocol, participant profile and patterns of participation BMC Public Health 2023 23 1 466 10.1186/s12889-023-15355-x 36899371 PMC9999338 · DOI
  12. [15]Ferris FL Wilkinson CP Bird A Clinical classification of age-related macular degeneration Ophthalmology 2013 120 4 844 851 10.1016/j.ophtha.2012.10.036 23332590 PMC11551519 · DOI
  13. [16]Bailey IL Lovie JE The design and use of a new near-vision chart Am J Optom Physiol Opt 1980 57 6 378 387 10.1097/00006324-198006000-00011 7406006 · DOI
  14. [17]McClure ME Hart PM Jackson AJ Stevenson MR Chakravarthy U Macular degeneration: do conventional measurements of impaired visual function equate with visual disability? Br J Ophthalmol 2000 84 3 244 250 10.1136/bjo.84.3.244 10684832 PMC1723404 · DOI
  15. [18]Chakravarthy U Harding SP Rogers CA A randomised controlled trial to assess the clinical effectiveness and cost-effectiveness of alternative treatments to Inhibit VEGF in Age-related choroidal Neovascularisation (IVAN) Health Technol Assess 2015 19 78 1 298 10.3310/hta19780 26445075 PMC4781416 · DOI
  16. [19]Sunness JS Rubin GS Applegate CA Visual function abnormalities and prognosis in eyes with age-related geographic atrophy of the macula and good visual acuity Ophthalmology 1997 104 10 1677 1691 10.1016/S0161-6420(97)30079-7 9331210 PMC2730508 · DOI
  17. [20]Sunness JS Rubin GS Broman A Applegate CA Bressler NM Hawkins BS Low luminance visual dysfunction as a predictor of subsequent visual acuity loss from geographic atrophy in age-related macular degeneration Ophthalmology 2008 115 9 1480 1488.e2 10.1016/j.ophtha.2008.03.009 18486216 PMC2730515 · DOI
  18. [21]Haegerstrom-Portnoy G Brabyn J Schneck ME Jampolsky A The SKILL Card. An acuity test of reduced luminance and contrast. Smith-Kettlewell Institute Low Luminance Invest Ophthalmol Vis Sci 1997 38 1 207 218 9008645
  19. [23]Shah N Dakin SC Redmond T Anderson RS Vanishing optotype acuity: repeatability and effect of the number of alternatives Ophthalmic Physiol Opt 2011 31 1 17 22 10.1111/j.1475-1313.2010.00806.x 21158882 · DOI
  20. [24]Shah N Dakin SC Mulholland PJ Racheva K Matlach J Anderson RS The effect of induced intraocular stray light on recognition thresholds for pseudo-high-pass filtered letters Transl Vis Sci Technol 2022 11 5 4 10.1167/tvst.11.5.4 35511149 PMC9078078 · DOI
  21. [25]Wu Z Ayton LN Guymer RH Luu CD Intrasession test-retest variability of microperimetry in age-related macular degeneration Investig Ophthalmol Vis Sci 2013 54 12 7378 7385 10.1167/iovs.13-12617 24135753 · DOI
  22. [26]Scanlon PH The English National Screening Programme for diabetic retinopathy 2003–2016 Acta Diabetol 2017 54 6 515 525 10.1007/s00592-017-0974-1 28224275 PMC5429356 · DOI
  23. [28]Naimi AI Balzer LB Stacked generalization: an introduction to super learning Eur J Epidemiol 2018 33 5 459 464 10.1007/s10654-018-0390-z 29637384 PMC6089257 · DOI
  24. [29]Phillips RV, van der Laan MJ, Lee H, Gruber S (2023) Practical considerations for specifying a super learner. Int J Epidemiol 52(4):1276–1285. 10.1093/ije/dyad023 36905602 · DOI
  25. [30]Lundberg SM, Lee S-I (2017) A unified approach to interpreting model predictions. In: 31st Conference on Neural Information Processing Systems (NIPS 2017). Long Beach, CA, USA, pp 1–10
  26. [34]Sharanjeet-Kaur Ismail S-A Mutalib HA Ngah NF HbA1c and retinal sensitivity in diabetics using microperimetry J Optom 2019 12 3 174 179 10.1016/j.optom.2018.03.007 29843983 PMC6612021 · DOI
  27. [35]Park JC Chen Y-F Liu M Liu K McAnany JJ Structural and functional abnormalities in early-stage diabetic retinopathy Curr Eye Res 2020 45 8 975 985 10.1080/02713683.2019.1705983 31847599 PMC7347436 · DOI
  28. [36]Boned-Murillo A Diaz-Barreda MD Ferreras A Structural and functional findings in patients with moderate diabetic retinopathy Graefes Arch Clin Exp Ophthalmol 2021 259 12 3625 3635 10.1007/s00417-021-05277-y 34264395 PMC8589761 · DOI
  29. [37]Chai Q Yao Y Guo C Lu H Ma J Structural and functional retinal changes in patients with type 2 diabetes without diabetic retinopathy Ann Med 2002 54 1 1816 1825 10.1080/07853890.2022.2095010 PMC9258434 35786137 · DOI
  30. [38]Nauta M Walsh R Dubowski A Seifert C Uncovering and correcting shortcut learning in machine learning models for skin cancer diagnosis Diagnostics (Basel) 2021 12 1 40 10.3390/diagnostics12010040 35054207 PMC8774502 · DOI
  31. [39]DeGrave AJ Janizek JD Lee S-I AI for radiographic COVID-19 detection selects shortcuts over signal Nat Mach Intell 2021 3 7 610 619 10.1038/s42256-021-00338-7 · DOI · Paper page
  32. [40]Athanasiou M, Sfrintzeri K, Zarkogianni K, Thanopoulou AC, Nikita KS (2020) An explainable XGBoost–based approach towards assessing the risk of cardiovascular disease in patients with type 2 diabetes mellitus. In: 2020 IEEE 20th International Conference on Bioinformatics and Bioengineering (BIBE). IEEE, Cincinnati, OH, USA, pp 859–864
  33. [41]Feng Q, Du M, Zou N, Hu X (2022) Fair machine learning in healthcare: a review. arXiv:2206.14397 (Preprint). 16 Aug 2022. Available from 10.48550/arXiv.2206.14397 · DOI · Paper page
  34. [44]Ehrmann DE Joshi S Goodfellow SD Mazwi ML Eytan D Making machine learning matter to clinicians: model actionability in medical decision-making NPJ Digit Med 2023 6 1 7 10.1038/s41746-023-00753-7 36690689 PMC9871014 · DOI

Authors
Wright, D. M., Chakravarthy, U., Das, R., Graham, K. W., Naskas, T. T., Perais, J., Kee, F., Peto, T., & Hogg, R. E.
Affiliations
Queen's University Belfast (GB)
Corresponding author
Wright, David M.
ORCID
Venue
Diabetologia
Publisher
Springer Science and Business Media LLC
Published
2023-09-19
Volume
66
Issue
12
Pages
2250-2260
ISSN
  • 0012-186X (ISSN-L)
  • 1432-0428
DOI
10.1007/s00125-023-06005-3
PMC
PMC10627908
OpenAlex
W4386878121
Cited by
10
References
38
Citations by year
2026: 6 · 2025: 1 · 2024: 3
FWCI
2.43
Type
journal article
Language
en
Open access
Licence
Locations
  • Diabetologia (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • Research Portal (Queen's University Belfast) (publishedVersion, cc-by)
Funders
  • Economic and Social Research Council
  • UKCRC Centre of Excellence for Public Health Northern Ireland
  • Belfast Association for the Blind
  • The Atlantic Philanthropies
  • Thomas Pocklington Trust
  • Guidedogs for the Blind
  • Diabetes UK
  • Macular Society
  • College of Optometrists
  • Queen's University Belfast
  • Wolfson Foundation
  • Wellcome Trust
  • Health and Social Care Research and Development Divsion of the Public Health Agency
  • Office of the First Minister and Deputy First Minister
  • Centre for Ageing Research and Development in Ireland
Topics
  • Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
diabetic retinopathy severity, diabetic macular edema, visual function assessment, model interpretability, ensemble learning, near visual acuity, microperimetry, missing data imputation, data artefacts
MeSH terms
Machine Learning, Cross-Sectional Studies, Diabetes Mellitus, Diabetic Retinopathy, Humans, Macular Edema, Visual Acuity

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review