Evaluation of ChatGPT-4 in detecting referable diabetic retinopathy using single fundus images
Open accessAbstract
• ChatGPT-4 is unable to adequately read and interpret normal and mild NPDR images limiting utility in low-risk settings. • ChatGPT-4 shows high sensitivity for identifying referable diabetic retinopathy but lacks specificity. • ChatGPT-4 needs improvement in multimodal image fund of knowledge in order to improve real-world applicability. Evaluate ChatGPT-4′s ability to identify referable diabetic retinopathy (DR) from single fundus images. A cross-sectional study comparing ChatGPT-4′s versus retina specialists’ identification of more than mild DR (mtmDR) and vision-threatening DR (VTDR). Images in equal proportions of normal, mild, moderate, and severe nonproliferative DR (NPDR), proliferative DR (PDR), and blurry images with and without suspected PDR were presented to a panel of blinded retina specialists who identified images as readable or unreadable, and potentially as mtmDR or VTDR. These images were also submitted to ChatGPT-4 three times with a standardized prompt regarding mtmDR and VTDR. Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated for ChatGPT-4′s responses regarding mtmDR and VTDR as compared to the retina specialists majority determination. Retina specialists read 158/180 prompts (87.7 %) with excellent interrater reliability while ChatGPT-4 read 132/180 (73.33 %) of the image prompts. For mtmDR, ChatGPT-4 demonstrated a sensitivity of 96.2 %, specificity of 19.1 %, PPV of 69.1 %, and NPV of 72.7 %.…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Aftab et al. reported that ChatGPT-4 exhibited high sensitivity for referable DR, reaching 96.2% in high-risk populations, but its limited specificity and tendency to overcall disease reduce its practicality as a general screening tool, while supporting its use as an auxiliary triage system for high-risk patients.
“Aftab et al. [134] reported that ChatGPT-4 exhibited high sensitivity for referable DR, reaching 96.2% in high-risk populations, but its limited specificity and tendency to overcall disease reduce its practicality as a general screening tool, while supporting its use as an auxiliary triage system for high-risk patients.”
From Artificial intelligence-powered prediction of diabetic complications: from clinical data to molecular omics (2026) · cites this paper as [134]
Cites
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · see citing works
Global Prevalence of Diabetic Retinopathy and Projection of Burden through 2045
Teo, Z. L., Tham, Y.-C., Yu, M., Chee, M. L., Rim, T. H., Cheung, N., Bikbov, M. M., Wang, Y. X., Tang, Y., Lu, Y., et al. (2021) · Ophthalmology · doi:10.1016/j.ophtha.2021.04.027 · Cited by 2,420 · see citing works
Screening for diabetic retinopathy: new perspectives and challenges
Vujosevic, S., Aldington, S. J., Silva, P., Hernández, C., Scanlon, P., Peto, T., & Simó, R. (2020) · The Lancet Diabetes & Endocrinology · doi:10.1016/s2213-8587(19)30411-5 · Cited by 671 · see citing works
Evaluating the Performance of ChatGPT in Ophthalmology
Antaki, F., Touma, S., Milad, D., El-Khoury, J., & Duval, R. (2023) · Ophthalmology Science · doi:10.1016/j.xops.2023.100324 · Cited by 563 · see citing works
EyePACS: An Adaptable Telemedicine System for Diabetic Retinopathy Screening
Cuadros, J., & Bresnick, G. (2009) · Journal of Diabetes Science and Technology · doi:10.1177/193229680900300315 · Cited by 390 · see citing works
Cen, L., Ji, J., Lin, J., Ju, S.-T., Lin, H.-J., Li, T.-P., Wang, Y., Yang, J.-F., Liu, Y.-F., Tan, S., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-25138-w · Cited by 352 · see citing works
Ipp, E., Liljenquist, D., Bode, B., Shah, V. N., Silverstein, S., Regillo, C. D., Lim, J. I., Sadda, S., Domalpally, A., Gray, G., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.34254 · Cited by 256 · see citing works
Self-citationPiyasena, P., Murthy, G. V. S., Yip, J. L. Y., Gilbert, C. E., Zuurmond, M., Pető, T., Gordon, I., Hewage, S. N., & Kamalakannan, S. (2019) · PLoS ONE · doi:10.1371/journal.pone.0198979 · Cited by 180 · see citing works
Addressing Artificial Intelligence Bias in Retinal Diagnostics
Burlina, P., Joshi, N., Paul, W., Pacheco, K. D., & Bressler, N. M. (2021) · Translational Vision Science & Technology · doi:10.1167/tvst.10.2.13 · Cited by 138 · see citing works
Generative Artificial Intelligence Through ChatGPT and Other Large Language Models in Ophthalmology
Tan, T. F., Thirunavukarasu, A. J., Campbell, J. P., Keane, P. A., Pasquale, L. R., Abràmoff, M. D., Kalpathy–Cramer, J., Lum, F., Kim, J. E., Baxter, S. L., et al. (2023) · Ophthalmology Science · doi:10.1016/j.xops.2023.100394 · Cited by 120 · see citing works
Automated Diabetic Retinopathy Screening and Monitoring Using Retinal Fundus Image Analysis
Bhaskaranand, M., Ramachandra, C. A., Bhat, S., Cuadros, J. A., Nittala, M. G., Sadda, S. R., & Solanki, K. V. (2016) · Journal of Diabetes Science and Technology · doi:10.1177/1932296816628546 · Cited by 119 · see citing works
Cost-effectiveness of artificial intelligence screening for diabetic retinopathy in rural China
Huang, X.-M., Yang, B.-F., Zheng, W.-L., Liu, Q., Xiao, F., Ouyang, P.-W., Li, M.-J., Li, X.-Y., Meng, J., Zhang, T.-T., et al. (2022) · BMC Health Services Research · doi:10.1186/s12913-022-07655-6 · Cited by 89 · see citing works
Sosale, B., Sosale, A., Murthy, H., Sengupta, S., & Naveenam, M. (2020) · Indian Journal of Ophthalmology · doi:10.4103/ijo.ijo_1203_19 · Cited by 69 · see citing works
Grzybowski, A., & Brona, P. (2021) · Journal of Clinical Medicine · doi:10.3390/jcm10112352 · Cited by 64 · see citing works
Reimagining Health Data Exchange: An Application Programming Interface–Enabled Roadmap for India
Balsari, S., Fortenko, A., Blaya, J. A., Gropper, A., Jayaram, M., Matthan, R., Sahasranam, R., Shankar, M., Sarbadhikari, S. N., Bierer, B. E., et al. (2018) · Journal of Medical Internet Research · doi:10.2196/10725 · Cited by 46 · see citing works
Grzybowski, A., Rao, D. P., Brona, P., Negiloni, K., Krzywicki, T., & Savoy, F. M. (2023) · Ophthalmic Research · doi:10.1159/000534098 · Cited by 33 · see citing works
Feasibility and acceptance of artificial intelligence-based diabetic retinopathy screening in Rwanda
Whitestone, N., Nkurikiye, J., Patnaik, J. L., Jaccard, N., Lanouette, G., Cherwek, D. H., Congdon, N., & Mathenge, W. (2024) · British Journal of Ophthalmology · doi:10.1136/bjo-2022-322683 · Cited by 31 · see citing works
Appropriateness of ophthalmic symptoms triage by a popular online artificial intelligence chatbot
Tsui, J. C., Wong, M. B., Kim, B. J., Maguire, A. M., Scoles, D., VanderBeek, B. L., & Brucker, A. J. (2023) · Eye · doi:10.1038/s41433-023-02556-2 · Cited by 27 · see citing works
Self-citationRaghu, K., Tamilselvi, S., Devishamani, C. S., Suchetha, M., Rajalakshmi, R., & Raman, R. (2023) · Clinical ophthalmology · doi:10.2147/opth.s435052 · Cited by 26 · see citing works
Mokhashi, N., Grachevskaya, J., Cheng, L., Yu, D., Lu, X., Zhang, Y., & Henderer, J. D. (2022) · Journal of Diabetes Science and Technology · doi:10.1177/1932296821999370 · Cited by 23 · see citing works
Clinician-Driven AI: Code-Free Self-Training on Public Data for Diabetic Retinopathy Referral
Korot, E., Gonçalves, M. B., Huemer, J. C., Beqiri, S., Khalid, H., Kelly, M., Chia, M. A., Mathijs, E., Struyven, R., Moussa, M. S., et al. (2023) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2023.4508 · Cited by 20 · see citing works
Hasan, S. U., & Siddiqui, M. A. R. (2023) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2023.110943 · Cited by 20 · see citing works
Zhu, A., Tailor, P., Verma, R., Zhang, I., Schott, B., Ye, C., Szirth, B., Habiel, M. M., & Khouri, A. S. (2023) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x231158832 · Cited by 14 · see citing works
Artificial Intelligence in Diabetic Retinopathy: Insights from a Meta-Analysis of Deep Learning
Poly, T. N., Mohaimenul, I. M., Chia, Y. H., Nguyen, P., Wu, C., & Li, Y. (2019) · Studies in health technology and informatics · doi:10.3233/shti190532 · Cited by 12 · see citing works
Flagged first, then by citations
1 of the 25 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
Most, J. A., Walker, E. H., Mehta, N. N., Nagel, I. D., Chen, J., Russell, J. F., Scott, N. L., & Borooah, S. (2025) · Ophthalmology Science · doi:10.1016/j.xops.2025.100911 · Cited by 10 · see citing works
Xie, X., Wu, C., Huang, Z., Zhou, Y., Huang, J., Dao, F., Yan, D., Deng, K., Lyu, H., Ma, C., et al. (2026) · Briefings in Bioinformatics · doi:10.1093/bib/bbag083 · Cited by 4 · see citing works
“Moreover, Aftab et al. [134] reported that ChatGPT-4 exhibited high sensitivity for referable DR, reaching 96.2% in high-risk populations, but its limited specificity and tendency to overcall disease reduce its practicality as a general screening tool, while supporting its use as an auxiliary…”
InfluentialEvaluating the clinical utility of multimodal large language models in rare maculopathy
Tran, M. D., Walker, E., Nagel, I. D., Mehta, N. N., Most, J. A., Ferreyra, H. A., Everett, L. A., Yang, P. G., Pennesi, M. E., & Borooah, S. (2025) · Scientific Reports · doi:10.1038/s41598-025-29299-2 · Cited by 2 · see citing works
Mihalache, A., Pandya, B. U., Jhaveri, A., Pereira, A., Tayeb, S. O., Hurley, B. R., Kherani, A., You, Y., & Yan, P. (2026) · Journal of VitreoRetinal Diseases · doi:10.1177/24741264261457974 · Cited by 1 · see citing works
Gastronomik Estetiğin Yeni Dili: Tabak Tasarımında ChatGPT'nin Pratik Bir Uygulaması
Şahin, E. Ö. (2025) · Uluslararası Sosyal Bilimler Akademi Dergisi · doi:10.47994/usbad.1705581
Multimodal large language models for use in diabetic retinopathy screening
Mohammadi, S. S., Aggarwal, S., Aggarwal, K., Wiarda, G., Nguyen, K., Sarmiento, E. A., Nguyen, Q. D., & Gill, M. K. (2026) · Artificial Intelligence in Vision and Ophthalmology · doi:10.35119/aivo.v2i1.157
Tabuchi, H., Tanabe, M., Kihara, K., Moriguchi, T., & Gomi, F. (2026) · SN Comprehensive Clinical Medicine · doi:10.1007/s42399-026-02549-8
Navarro, H. J. (2026) · Journal of Intelligent Decision Making and Information Science · doi:10.59543/jidmis.v3.807
Khan, H. N., Mirza, I., Aftab, O. M., Shah, Y., & Al‐Khazraji, A. (2026) · JGH Open · doi:10.1002/jgh3.70452
Self-citationJun, H. W., Nam, D., Choi, J. Y., & Yoo, T. K. (2026) · Engineering Applications of Artificial Intelligence · doi:10.1016/j.engappai.2026.116609
Mutuma, P. K., Ayonga, W., & Marera, D. (2026) · International Journal of Community Medicine and Public Health · doi:10.18203/2394-6040.ijcmph20264123
- Authors
- Aftab, O., Khan, H., VanderBeek, B. L., Scoles, D., Kim, B. J., & Tsui, J. C.
- Affiliations
- Rutgers New Jersey Medical School (US)
- University of Pennsylvania (US)
- Children's Hospital of Philadelphia (US)
- Rutgers, The State University of New Jersey (US)
- Corresponding author
- Tsui, Jonathan C.
- Venue
- AJO International
- Publisher
- Elsevier BV
- Published
- 2025-07
- Volume
- 2
- Issue
- 2
- Pages
- 100111
- ISSN
- 2950-2535 (ISSN-L)
- OpenAlex
- W4408554056
- Cited by
- 10
- References
- 25
- Citations by year
- 2026: 7 · 2025: 3
- FWCI
- 5.74
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- AJO International (publishedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- ChatGPT-4, referable diabetic retinopathy, diabetic retinopathy screening, fundus images, non-proliferative diabetic retinopathy, proliferative diabetic retinopathy, vision-threatening diabetic retinopathy, large language models, AI diagnostic accuracy, sensitivity and specificity, overdiagnosis, resource-limited settings
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
