CoChat
Sign Up
Cited by 108Retinal Imaging and AnalysisRetinal Diseases and TreatmentsArtificial Intelligence in Healthcare

Performance and Limitation of Machine Learning Algorithms for Diabetic Retinopathy Screening: Meta-analysis

Open access
Wu, Jo-Hsuan, Liu, T Y Alvin, Hsu, Wan-Ting, Ho, Jennifer Hui-Chun, Lee, Chien-Chang·2021·Journal of Medical Internet Research
DOI 10.2196/23863PMC PMC8406115

Published version of a preprint

Open this paper in CoChat — free accountOpen PDF

Abstract

Background: Diabetic retinopathy (DR), whose standard diagnosis is performed by human experts, has high prevalence and requires a more efficient screening method. Although machine learning (ML)–based automated DR diagnosis has gained attention due to recent approval of IDx-DR, performance of this tool has not been examined systematically, and the best ML technique for use in a real-world setting has not been discussed. Objective: The aim of this study was to systematically examine the overall diagnostic accuracy of ML in diagnosing DR of different categories based on color fundus photographs and to determine the state-of-the-art ML approach. Methods: Published studies in PubMed and EMBASE were searched from inception to June 2020. Studies were screened for relevant outcomes, publication types, and data sufficiency, and a total of 60 out of 2128 (2.82%) studies were retrieved after study selection. Extraction of data was performed by 2 authors according to PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and the quality assessment was performed according to the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2). Meta-analysis of diagnostic accuracy was pooled using a bivariate random effects model. The main outcomes included diagnostic accuracy, sensitivity, and specificity of ML in diagnosing DR based on color fundus photographs, as well as the performances of different major types of ML algorithms. Results: The primary meta-analysis included 60 color fundus photograph studies (445,175 interpretations). Overall, ML demonstrated high accuracy in diagnosing DR of various categories, with a pooled area under the receiver operating characteristic (AUROC) ranging from 0.97 (95% CI 0.96-0.99) to 0.99 (95% CI 0.98-1.00). The performance of ML in detecting more-than-mild DR was robust (sensitivity 0.95; AUROC 0.97), and by subgroup analyses, we observed that robust performance of ML was not limited to benchmark data sets (sensitivity 0.92; AUROC 0.96) but could be generalized to images collected in clinical practice (sensitivity 0.97; AUROC 0.97). Neural network was the most widely used method, and the subgroup analysis revealed a pooled AUROC of 0.98 (95% CI 0.96-0.99) for studies that used neural networks to diagnose more-than-mild DR. Conclusions: This meta-analysis demonstrated high diagnostic accuracy of ML algorithms in detecting DR on color fundus photographs, suggesting that state-of-the-art, ML-based DR screening algorithms are likely ready for clinical applications. However, a significant portion of the earlier published studies had methodology flaws, such as the lack of external validation and presence of spectrum bias. The results of these studies should be interpreted with caution.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.2196/23863

Cites

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

The citation lists haven’t been fetched yet

References 80 · Cited by 108

Research works printed but not linked: 73

  1. [1]Fong DS Aiello L Gardner TW King GL Blankenship G Cavallerano JD Ferris FL Klein R American Diabetes Association Diabetic retinopathy Diabetes Care 2003 1 26 1 226 9 10.2337/diacare.26.1.226 12502685 12502685 · DOI
  2. [2]Yau JWY Rogers SL Kawasaki R Lamoureux EL Kowalski JW Bek T Chen S Dekker JM Fletcher A Grauslund J Haffner S Hamman RF Ikram MK Kayama T Klein BEK Klein R Krishnaiah S Mayurasakorn K O'Hare JP Orchard TJ Porta M Rema M Roy MS Sharma T Shaw J Taylor H Tielsch JM Varma R Wang JJ Wang N West S Xu L Yasuda M Zhang X Mitchell P Wong TY Meta-Analysis FED(SG Global prevalence and major risk factors of diabetic retinopathy Diabetes Care 2012 3 35 3 556 64 10.2337/dc11-1909 22301125 22301125 PMC3322721 · DOI · Paper page
  3. [3]Zheng Y He M Congdon N The worldwide epidemic of diabetic retinopathy Indian J Ophthalmol 2012 60 5 428 31 10.4103/0301-4738.100542 22944754 22944754 PMC3491270 · DOI
  4. [4]Vashist P Singh S Gupta N Saxena R Role of early screening for diabetic retinopathy in patients with diabetes mellitus: an overview Indian J Community Med 2011 10 36 4 247 52 10.4103/0970-0218.91324 22279252 22279252 PMC3263142 · DOI
  5. [6]Viswanath K McGavin DDM Diabetic retinopathy: clinical findings and management Community Eye Health 2003 16 46 21 4 17491851 17491851 PMC1705856
  6. [7]Heitmar R Kalitzeos AA Patel SR Prabhu-Das D Cubbidge RP Comparison of subjective and objective methods to determine the retinal arterio-venous ratio using fundus photography J Optom 2015 8 4 252 7 10.1016/j.optom.2014.07.002 26386537 26386537 PMC4591425 · DOI
  7. [8]Hosny A Parmar C Quackenbush J Schwartz LH Aerts HJWL Artificial intelligence in radiology Nat Rev Cancer 2018 12 18 8 500 510 10.1038/s41568-018-0016-5 29777175 29777175 PMC6268174 · DOI · Paper page
  8. [9]Wernick MN Yang Y Brankov JG Yourganov G Strother SC Machine learning in medical imaging IEEE Signal Process Mag 2010 7 27 4 25 38 10.1109/MSP.2010.936730 25382956 25382956 PMC4220564 · DOI
  9. [10]Wong TY Bressler NM Artificial intelligence with deep learning technology looks into diabetic retinopathy screening JAMA 2016 12 13 316 22 2366 2367 10.1001/jama.2016.17563 10.1001/jama.2016.17563 27898977 27898977 · DOI
  10. [11]LeCun Y Bengio Y Hinton G Deep learning Nature 2015 5 28 521 7553 436 44 10.1038/nature14539 26017442 26017442 · DOI · Paper page
  11. [12]Jiang F Jiang Y Zhi H Dong Y Li H Ma S Wang Y Dong Q Shen H Wang Y Artificial intelligence in healthcare: past, present and future Stroke Vasc Neurol 2017 12 2 4 230 243 10.1136/svn-2017-000101 29507784 29507784 PMC5829945 · DOI · Paper page
  12. [13]Gulshan V Peng L Coram M Stumpe MC Wu D Narayanaswamy A Venugopalan S Widner K Madams T Cuadros J Kim R Raman R Nelson PC Mega JL Webster DR Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs JAMA 2016 12 13 316 22 2402 2410 10.1001/jama.2016.17216 27898976 27898976 · DOI · Paper page
  13. [15]Abràmoff MD Lavin PT Birch M Shah N Folk JC Pivotal trial of an autonomous AI-based diagnostic system for detection of diabetic retinopathy in primary care offices NPJ Digit Med 2018 1 39 10.1038/s41746-018-0040-6 31304320 31304320 PMC6550188 · DOI · Paper page
  14. [17]Abbas Q Fondon I Sarmiento A Jiménez S Alemany P Automatic recognition of severity level for diagnosis of diabetic retinopathy using deep visual features Med Biol Eng Comput 2017 11 55 11 1959 1974 10.1007/s11517-017-1638-6 28353133 28353133 · DOI
  15. [19]Adal KM Sidibé D Ali S Chaum E Karnowski TP Mériaudeau F Automated detection of microaneurysms using scale-adapted blob analysis and semi-supervised learning Comput Methods Programs Biomed 2014 4 114 1 1 10 10.1016/j.cmpb.2013.12.009 24529636 24529636 · DOI
  16. [20]Agurto C Barriga ES Murray V Nemeth S Crammer R Bauman W Zamora G Pattichis MS Soliz P Automatic detection of diabetic retinopathy and age-related macular degeneration in digital fundus images Invest Ophthalmol Vis Sci 2011 7 29 52 8 5862 71 10.1167/iovs.10-7075 21666234 21666234 PMC3176039 · DOI
  17. [22]Chaum E Karnowski TP Govindasamy VP Abdelrahman M Tobin KW Automated diagnosis of retinopathy by content-based image retrieval Retina 2008 28 10 1463 77 10.1097/IAE.0b013e31818356dd 18997609 18997609 · DOI
  18. [23]Ganesan K Martis RJ Acharya UR Chua CK Min LC Ng EYK Laude A Computer-aided diabetic retinopathy detection using trace transforms on digital fundus images Med Biol Eng Comput 2014 8 52 8 663 72 10.1007/s11517-014-1167-5 24958614 24958614 · DOI
  19. [24]Gardner GG Keating D Williamson TH Elliott AT Automatic detection of diabetic retinopathy using an artificial neural network: a screening tool Br J Ophthalmol 1996 11 80 11 940 4 10.1136/bjo.80.11.940 8976718 8976718 PMC505667 · DOI
  20. [25]Gargeya R Leng T Automated identification of diabetic retinopathy using deep learning Ophthalmology 2017 12 124 7 962 969 10.1016/j.ophtha.2017.02.008 28359545 28359545 · DOI
  21. [26]Gupta G Kulasekaran S Ram K Joshi N Sivaprakasam M Gandhi R Local characterization of neovascularization and identification of proliferative diabetic retinopathy in retinal fundus images Comput Med Imaging Graph 2017 1 55 124 132 10.1016/j.compmedimag.2016.08.005 27634547 27634547 · DOI
  22. [27]Jelinek HF Cree MJ Leandro JJG Soares JVB Cesar RM Luckie A Automated segmentation of retinal blood vessels and identification of proliferative diabetic retinopathy J Opt Soc Am A Opt Image Sci Vis 2007 5 24 5 1448 56 10.1364/josaa.24.001448 17429492 17429492 · DOI
  23. [28]Malathi K Nedunchelian R A recursive support vector machine (RSVM) algorithm to detect and classify diabetic retinopathy in fundus retina images biomedicalresearch 2018 57 64 10.4066/biomedicalresearch.29-16-2328 · DOI
  24. [29]Fadafen MK Mehrshad N Razavi SM Detection of diabetic retinopathy using computational model of human visual system biomedicalresearch 2018 29 9 1956 1960 10.4066/biomedicalresearch.29-18-551 · DOI
  25. [30]Li Z Keel S Liu C He Y Meng W Scheetz J Lee PY Shaw J Ting D Wong TY Taylor H Chang R He M An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs Diabetes Care 2018 12 41 12 2509 2516 10.2337/dc18-0147 30275284 30275284 · DOI · Paper page
  26. [31]Bala MP Vijayachitra S Extraction of retinal blood vessels and diagnosis of proliferative diabetic retinopathy using extreme learning machine J Med Imaging Hlth Inform 2015 4 01 5 2 248 256 10.1166/jmihi.2015.1380 · DOI
  27. [32]Orlando JI Prokofyeva E Del Fresno M Blaschko MB An ensemble deep learning based approach for red lesion detection in fundus images Comput Methods Programs Biomed 2018 1 153 115 127 10.1016/j.cmpb.2017.10.017 29157445 29157445 · DOI
  28. [33]Orlando JI van Keer K Barbosa Breda J Manterola HL Blaschko MB Clausse A Proliferative diabetic retinopathy characterization based on fractal features: Evaluation on a publicly available dataset Med Phys 2017 12 44 12 6425 6434 10.1002/mp.12627 29044550 29044550 · DOI
  29. [34]Pires R Carvalho T Spurling G Goldenstein S Wainer J Luckie A Jelinek HF Rocha A Automated multi-lesion detection for referable diabetic retinopathy in indigenous health care PLoS One 2015 10 6 e0127664 10.1371/journal.pone.0127664 26035836 26035836 PMC4452786 · DOI
  30. [35]Quellec G Charrière K Boudi Y Cochener B Lamard M Deep image mining for diabetic retinopathy screening Med Image Anal 2017 7 39 178 193 10.1016/j.media.2017.04.012 10.1016/j.media.2017.04.012 28511066 28511066 · DOI
  31. [36]Quellec G Lamard M Abràmoff MD Decencière E Lay B Erginay A Cochener B Cazuguel G A multiple-instance learning framework for diabetic retinopathy screening Med Image Anal 2012 8 16 6 1228 40 10.1016/j.media.2012.06.003 22850462 22850462 · DOI
  32. [37]Rajalakshmi R Subashini R Anjana RM Mohan V Automated diabetic retinopathy detection in smartphone-based fundus photography using artificial intelligence Eye (Lond) 2018 6 32 6 1138 1144 10.1038/s41433-018-0064-9 29520050 29520050 PMC5997766 · DOI · Paper page
  33. [38]Raju M Pagidimarri V Barreto R Kadam A Kasivajjala V Aswath A Development of a deep learning algorithm for automatic diagnosis of diabetic retinopathy Stud Health Technol Inform 2017 245 559 563 29295157 29295157
  34. [39]Ramachandran N Hong SC Sime MJ Wilson GA Diabetic retinopathy screening using deep neural network Clin Exp Ophthalmol 2018 5 46 4 412 416 10.1111/ceo.13056 28881490 28881490 · DOI
  35. [40]Sangeethaa SN Uma Maheswari P An intelligent model for blood vessel segmentation in diagnosing DR using CNN J Med Syst 2018 8 15 42 10 175 10.1007/s10916-018-1030-6 30109508 30109508 · DOI
  36. [41]B. Sumathy S. Poornachandra Automated DR and prediction of various related diseases of retinal fundus images biomedicalresearch 2018 325 332 10.4066/biomedicalresearch.29-17-480 · DOI
  37. [42]Ting DSW Cheung CY Lim G Tan GSW Quang ND Gan A Hamzah H Garcia-Franco R San Yeo IY Lee SY Wong EYM Sabanayagam C Baskaran M Ibrahim F Tan NC Finkelstein EA Lamoureux EL Wong IY Bressler NM Sivaprasad S Varma R Jonas JB He MG Cheng C Cheung GCM Aung T Hsu W Lee ML Wong TY Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes JAMA 2017 12 12 318 22 2211 2223 10.1001/jama.2017.18152 29234807 29234807 PMC5820739 · DOI · Paper page
  38. [43]Usman Akram M Khalid S Tariq A Younus Javed M Detection of neovascularization in retinal images using multivariate m-Mediods based classifier Comput Med Imaging Graph 2013 37 5-6 346 57 10.1016/j.compmedimag.2013.06.008 10.1016/j.compmedimag.2013.06.008 23916066 23916066 · DOI
  39. [44]Welikala RA Fraz MM Dehmeshki J Hoppe A Tah V Mann S Williamson TH Barman SA Genetic algorithm based feature selection combined with dual classification for the automated detection of proliferative diabetic retinopathy Comput Med Imaging Graph 2015 7 43 64 77 10.1016/j.compmedimag.2015.03.003 25841182 25841182 · DOI
  40. [45]Yu S Xiao D Kanagasingam Y Machine learning based automatic neovascularization detection on optic disc region IEEE J. Biomed. Health Inform 2018 5 22 3 886 894 10.1109/jbhi.2017.2710201 29727291 · DOI
  41. [46]Zhang Y An M An active learning classifier for further reducing diabetic retinopathy screening system cost Comput Math Methods Med 2016 2016 4345936 10.1155/2016/4345936 10.1155/2016/4345936 27660645 27660645 PMC5021911 · DOI
  42. [47]Badgujar R Deore P Hybrid nature inspired SMO-GBM classifier for exudate classification on fundus retinal images IRBM 2019 3 40 2 69 77 10.1016/j.irbm.2019.02.003 32595044 · DOI
  43. [48]Bellemo V Lim ZW Lim G Nguyen QD Xie Y Yip MYT Hamzah H Ho J Lee XQ Hsu W Lee ML Musonda L Chandran M Chipalo-Mutati G Muma M Tan GSW Sivaprasad S Menon G Wong TY Ting DSW Artificial intelligence using deep learning to screen for referable and vision-threatening diabetic retinopathy in Africa: a clinical validation study The Lancet Digital Health 2019 5 1 1 e35 e44 10.1016/s2589-7500(19)30004-4 33323239 · DOI · Paper page
  44. [50]Chowdhury AR Chatterjee T Banerjee S A Random Forest classifier-based approach in the detection of abnormalities in the retina Med Biol Eng Comput 2019 1 57 1 193 203 10.1007/s11517-018-1878-0 30076537 30076537 · DOI
  45. [51]Colomer A Igual J Naranjo V Detection of early signs of diabetic retinopathy based on textural and morphological information in fundus images Sensors (Basel) 2020 2 13 20 4 1005 10.3390/s20041005 32069912 PMC7071097 32069912 · DOI
  46. [53]He J Cao T Xu F Wang S Tao H Wu T Sun L Chen J Artificial intelligence-based screening for diabetic retinopathy at community hospital Eye (Lond) 2020 3 34 3 572 576 10.1038/s41433-019-0562-4 31455902 31455902 PMC7042314 · DOI
  47. [54]Hemanth DJ Anitha J Son LH Mittal M Diabetic retinopathy diagnosis from retinal images using modified Hopfield neural network J Med Syst 2018 10 31 42 12 247 10.1007/s10916-018-1111-6 30382410 30382410 · DOI
  48. [55]Kanagasingam Y Xiao D Vignarajan J Preetham A Tay-Kearney M Mehrotra A Evaluation of artificial intelligence-based grading of diabetic retinopathy in primary care JAMA Netw Open 2018 9 07 1 5 e182665 10.1001/jamanetworkopen.2018.2665 30646178 30646178 PMC6324474 · DOI · Paper page
  49. [56]Khojasteh P Passos Júnior LA Carvalho T Rezende E Aliahmad B Papa JP Kumar DK Exudate detection in fundus images using deeply-learnable features Comput Biol Med 2019 1 104 62 69 10.1016/j.compbiomed.2018.10.031 10.1016/j.compbiomed.2018.10.031 30439600 30439600 · DOI
  50. [57]Li F Liu Z Chen H Jiang M Zhang X Wu Z Automatic detection of diabetic retinopathy in retinal fundus photographs based on deep learning algorithm Transl Vis Sci Technol 2019 11 8 6 4 10.1167/tvst.8.6.4 31737428 PMC6855298 31737428 · DOI · Paper page
  51. [58]Long S Huang X Chen Z Pardhan S Zheng D automatic detection of hard exudates in color retinal images using dynamic threshold and SVM classification: algorithm development and evaluation Biomed Res Int 2019 2019 3926930 10.1155/2019/3926930 10.1155/2019/3926930 30809539 30809539 PMC6364257 · DOI
  52. [59]Natarajan S Jain A Krishnan R Rogye A Sivaprasad S Diagnostic accuracy of community-based diabetic retinopathy screening with an offline artificial intelligence system on a smartphone JAMA Ophthalmol 2019 8 08 1182 1188 10.1001/jamaophthalmol.2019.2923 31393538 31393538 PMC6692680 · DOI
  53. [60]Nazir T Irtaza A Shabbir Z Javed A Akram U Mahmood MT Diabetic retinopathy detection through novel tetragonal local octa patterns and extreme learning machines Artif Intell Med 2019 8 99 101695 10.1016/j.artmed.2019.07.003 10.1016/j.artmed.2019.07.003 31606114 31606114 · DOI
  54. [61]Pires R Avila S Wainer J Valle E Abramoff MD Rocha A A data-driven approach to referable diabetic retinopathy detection Artif Intell Med 2019 5 96 93 106 10.1016/j.artmed.2019.03.009 10.1016/j.artmed.2019.03.009 31164214 31164214 · DOI
  55. [62]Raumviboonsuk P Krause J Chotcomwongse P Sayres R Raman R Widner K Campana BJL Phene S Hemarat K Tadarati M Silpa-Archa S Limwattanayingyong J Rao C Kuruvilla O Jung J Tan J Orprayoon S Kangwanwongpaisan C Sukumalpaiboon R Luengchaichawang C Fuangkaew J Kongsap P Chualinpha L Saree S Kawinpanitan S Mitvongsa K Lawanasakol S Thepchatri C Wongpichedchai L Corrado GS Peng L Webster DR Deep learning versus human graders for classifying diabetic retinopathy severity in a nationwide screening program NPJ Digit Med 2019 2 25 10.1038/s41746-019-0099-8 31304372 31304372 PMC6550283 · DOI · Paper page
  56. [63]Riaz H Park J Choi H Kim H Kim J Deep and densely connected networks for classification of diabetic retinopathy Diagnostics 2020 1 02 10 1 24 10.3390/diagnostics10010024 PMC7169456 31906601 · DOI
  57. [64]Shah P Mishra DK Shanmugam MP Doshi B Jayaraj H Ramanjulu R Validation of Deep Convolutional Neural Network-based algorithm for detection of diabetic retinopathy - Artificial intelligence versus clinician for screening Indian J Ophthalmol 2020 2 68 2 398 405 10.4103/ijo.IJO_966_19 31957737 31957737 PMC7003578 · DOI
  58. [65]Son J Shin JY Kim HD Jung K Park KH Park SJ Development and validation of deep learning models for screening multiple abnormal findings in retinal fundus images Ophthalmology 2020 1 127 1 85 94 10.1016/j.ophtha.2019.05.029 10.1016/j.ophtha.2019.05.029 31281057 31281057 · DOI
  59. [66]Sosale Bhavana Aravind Sosale Ramachandra Murthy Hemanth Narayana Srikanth Sharma Usha Gowda Sahana G V Naveenam Muralidhar Simple, Mobile-based Artificial Intelligence Algorithm in the detection of Diabetic Retinopathy (SMART) study BMJ Open Diabetes Res Care 2020 1 8 1 e000892 10.1136/bmjdrc-2019-000892 32049632 PMC7039584 32049632 · DOI
  60. [67]Stevenson CH Hong SC Ogbuehi KC Development of an artificial intelligence system to classify pathology and clinical features on retinal fundus images Clin Exp Ophthalmol 2019 5 47 4 484 489 10.1111/ceo.13433 30370587 30370587 · DOI
  61. [68]Ullah H Saba T Islam N Abbas N Rehman A Mehmood Z Anjum A An ensemble classification of exudates in color fundus images using an evolutionary algorithm based optimal features selection Microsc Res Tech 2019 1 24 82 4 361 372 10.1002/jemt.23178 30677193 · DOI
  62. [70]Voets M Møllersen K Bongo LA Reproduction study using public data of: Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs PLoS One 2019 14 6 e0217541 10.1371/journal.pone.0217541 31170223 31170223 PMC6553744 · DOI
  63. [71]Wang H Yuan G Zhao X Peng L Wang Z He Y Qu C Peng Z Hard exudate detection based on deep model learned information and multi-feature joint representation for diabetic retinopathy screening Comput Methods Programs Biomed 2020 7 191 105398 10.1016/j.cmpb.2020.105398 10.1016/j.cmpb.2020.105398 32092614 32092614 · DOI
  64. [72]Xie L Yang S Squirrell D Vaghefi E Towards implementation of AI in New Zealand national diabetic screening program: Cloud-based, robust, and bespoke PLoS ONE 2020 4 10 15 4 e0225015 10.1371/journal.pone.0225015 32275656 PMC7147747 · DOI
  65. [73]Zago GT Andreão RV Dorizzi B Teatini Salles EO Diabetic retinopathy detection using red lesion localization and convolutional neural networks Comput Biol Med 2020 1 116 103537 10.1016/j.compbiomed.2019.103537 10.1016/j.compbiomed.2019.103537 31747632 31747632 · DOI
  66. [74]Yang W Zheng B Wu M Zhu S Fei F Weng M Zhang X Lu P An evaluation system of fundus photograph-based intelligent diagnostic technology for diabetic retinopathy and applicability for research Diabetes Ther 2019 10 10 5 1811 1822 10.1007/s13300-019-0652-0 31290125 31290125 PMC6778552 · DOI
  67. [75]Lawrence MG The accuracy of digital-video retinal imaging to screen for diabetic retinopathy: an analysis of two digital-video retinal imaging systems using standard stereoscopic seven-field photography and dilated clinical examination as reference standards Trans Am Ophthalmol Soc 2004 102 321 40 15747766 15747766 PMC1280108
  68. [76]Lin DY Blumenkranz MS Brothers RJ Grosvenor DM The sensitivity and specificity of single-field nonmydriatic monochromatic digital fundus photography with remote image interpretation for diabetic retinopathy screening: a comparison with ophthalmoscopy and standardized mydriatic color photography Am J Ophthalmol 2002 8 134 2 204 13 10.1016/s0002-9394(02)01522-2 12140027 12140027 · DOI
  69. [77]Pugh JA Jacobson JM Van Heuven WA Watters JA Tuley MR Lairson DR Lorimor RJ Kapadia AS Velez R Screening for diabetic retinopathy. The wide-angle retinal camera Diabetes Care 1993 6 16 6 889 95 10.2337/diacare.16.6.889 8100761 8100761 · DOI
  70. [78]Lopez-Bastida J Cabrera-Lopez F Serrano-Aguilar P Sensitivity and specificity of digital retinal imaging for screening diabetic retinopathy Diabet Med 2007 4 24 4 403 7 10.1111/j.1464-5491.2007.02074.x 17298591 17298591 · DOI
  71. [79]Saari JM Summanen P Kivelä T Saari KM Sensitivity and specificity of digital retinal images in grading diabetic retinopathy Acta Ophthalmol Scand 2004 4 82 2 126 30 10.1111/j.1600-0420.2004.00240.x 15043527 15043527 · DOI
  72. [80]Nielsen KB Lautrup ML Andersen JKH Savarimuthu TR Grauslund J Deep learning-based algorithms in screening of diabetic retinopathy: a systematic review of diagnostic performance Ophthalmol Retina 2019 4 3 4 294 304 10.1016/j.oret.2018.10.014 31014679 31014679 · DOI
  73. [81]Islam MM Yang H Poly TN Jian W Jack Li Y Deep learning algorithms for detection of diabetic retinopathy in retinal fundus photographs: A systematic review and meta-analysis Comput Methods Programs Biomed 2020 7 191 105320 10.1016/j.cmpb.2020.105320 10.1016/j.cmpb.2020.105320 32088490 32088490 · DOI

Authors
Wu, J.-H., Liu, T. Y. A., Hsu, W.-T., Ho, J. H.-C., & Lee, C.-C.
Affiliations
  • University of California San Diego (US)
  • Johns Hopkins University (US)
  • Harvard University (US)
  • National Yang Ming University (TW)
  • National Taiwan University Hospital (TW)
Corresponding author
Lee, Chien-Chang
ORCID
Venue
Journal of Medical Internet Research
Publisher
JMIR Publications Inc.
Published
2021-07-05
Volume
23
Issue
7
Pages
e23863
ISSN
1438-8871 (ISSN-L)
DOI
10.2196/23863
PMC
PMC8406115
OpenAlex
W3180176682
Cited by
108
References
80
Citations by year
2026: 12 · 2025: 17 · 2024: 34 · 2023: 25 · 2022: 18 · 2021: 2
FWCI
3.23
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • Journal of Medical Internet Research (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, other-oa)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
  • Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
Keywords
diabetic retinopathy, automated diagnosis, color fundus images, machine learning, diagnostic accuracy, artificial neural networks, sensitivity and specificity, retinal image analysis, screening tool, benchmark dataset
MeSH terms
Machine Learning, Algorithms, Diabetes Mellitus, Diabetic Retinopathy, Diagnostic Techniques, Ophthalmological, Humans, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review