Generative Artificial Intelligence Through ChatGPT and Other Large Language Models in Ophthalmology
Open accessThe DOI matches its Crossref record; 44 of the 44 references it cites were found in a registry or an index.
Abstract
The rapid progress of large language models (LLMs) driving generative artificial intelligence applications heralds the potential of opportunities in health care. We conducted a review up to April 2023 on Google Scholar, Embase, MEDLINE, and Scopus using the following terms: "large language models," "generative artificial intelligence," "ophthalmology," "ChatGPT," and "eye," based on relevance to this review. From a clinical viewpoint specific to ophthalmologists, we explore from the different stakeholders' perspectives-including patients, physicians, and policymakers-the potential LLM applications in education, research, and clinical domains specific to ophthalmology. We also highlight the foreseeable challenges of LLM implementation into clinical practice, including the concerns of accuracy, interpretability, perpetuating bias, and data security. As LLMs continue to mature, it is essential for stakeholders to jointly establish standards for best practices to safeguard patient safety. Financial Disclosures: Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Highly accurate protein structure prediction with AlphaFold
Jumper, J., Evans, R., Pritzel, A., Green, T., Figurnov, M., Ronneberger, O., Tunyasuvunakool, K., Bates, R., Žídek, A., Potapenko, A., et al. (2021) · Nature · doi:10.1038/s41586-021-03819-2 · Cited by 43,764
Scaling Laws for Neural Language Models
Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., & Amodei, D. (2020) · arXiv · doi:10.48550/arxiv.2001.08361 · Cited by 9,550 · cited 1×
“22 Improved availability of data from electronic patient records, paper-based documentation, and the scienti fi c literature entails overcoming issues of privacy and copyright which may not be feasible for medicine as a whole, let alone individual specialties such as ophthalmology.”
BackgroundBioBERT: a pre-trained biomedical language representation model for biomedical text mining
Lee, J., Yoon, W., Kim, S., Kim, D., Kim, S., So, C. H., & Kang, J. (2020) · Bioinformatics · doi:10.1093/bioinformatics/btz682 · Cited by 7,218 · cited 2×
“23,24,30 These models may help with the initial writing of perspective pieces and original articles, provided that inputs are carefully curated, outputs are validated to avoid mistakes and plagiarism, and model use is openly disclosed.”
BackgroundSurvey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y. J., Madotto, A., & Fung, P. (2023) · ACM Computing Surveys · doi:10.1145/3571730 · Cited by 5,025 · cited 1×
“72 Therefore, it is important to build LLM applications that acknowledge doubt and uncertainty rather than outputting unmitigated erroneous responses.”
BackgroundSurvey of Hallucination in Natural Language Generation
Ji, Z., Lee, N., Frieske, R., Yu, T., Su, D., Xu, Y., Ishii, E., Bang, Y., Chen, D., Dai, W., et al. (2022) · arXiv · doi:10.48550/arxiv.2202.03629 · Cited by 4,981
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505
Kung, T. H., Cheatham, M., Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., Maningo, J., et al. (2023) · PLOS Digital Health · doi:10.1371/journal.pdig.0000198 · Cited by 4,115
Kung, T. H., Cheatham, M., ChatGPT, Medenilla, A., Sillos, C., De Leon, L., Elepaño, C., Madriaga, M., Aggabao, R., Diaz-Candido, G., et al. (2022) · doi:10.1101/2022.12.19.22283643 · Cited by 4,115
SciBERT: A Pretrained Language Model for Scientific Text
Beltagy, I., Lo, K., & Cohan, A. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1371 · Cited by 3,218
Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., et al. (2023) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2023.1838 · Cited by 2,613
Domain-specific language model pretraining for biomedical natural language processing
Gu, Y., Tinn, R., Cheng, H., Lucas, M., Usuyama, N., Liu, X., Naumann, T., Gao, J., & Poon, H. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3458754 · Cited by 2,170
Large language models are zero-shot reasoners
Kojima, T., Gu, S. (., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1613 · Cited by 2,013
Large Language Models are Zero-Shot Reasoners
Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., & Iwasawa, Y. (2022) · arXiv · doi:10.48550/arxiv.2205.11916 · Cited by 2,013
Artificial intelligence and deep learning in ophthalmology
Ting, D. S. W., Pasquale, L. R., Peng, L., Campbell, J. P., Lee, A. Y., Raman, R., Tan, G. S. W., Schmetterer, L., Keane, P. A., & Wong, T. Y. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313173 · Cited by 1,554
Self-citationAcosta, J. N., Falcone, G. J., Rajpurkar, P., & Topol, E. J. (2022) · Nature Medicine · doi:10.1038/s41591-022-01981-2 · Cited by 1,365 · cited 1×
“53 Large language models have already demonstrated that they may encode suf fi cient information to assist with eye care, and further development and fi ne-tuning will see this potential improve.”
BackgroundDiagnostic accuracy of deep learning in medical imaging: a systematic review and meta-analysis
Aggarwal, R., Sounderajah, V., Martin, G., Ting, D. S. W., Karthikesalingam, A., King, D., Ashrafian, H., & Darzi, A. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00438-z · Cited by 946
Self-citationA large language model for electronic health records
Yang, X., Chen, A., PourNejatian, N., Shin, H. C., Smith, K. E., Parisien, C., Compas, C., Martin, C., Costa, A. B., Flores, M. G., et al. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00742-2 · Cited by 847
ChatGPT: the future of discharge summaries?
Patel, S. B., & Lam, K. (2023) · The Lancet Digital Health · doi:10.1016/s2589-7500(23)00021-3 · Cited by 728 · cited 1×
“57,58 Large language models may contribute to education in the broadest sense.”
BackgroundFriedberg, M., Chen, P., Van Busum, K., Aunon, F., Pham, C., Caloyeras, J., Mattke, S., Pitchforth, E., Quigley, D., Brook, R., et al. (2014) · RAND Health Quarterly · doi:10.7249/rhq-03-04-01 · Cited by 707
The Effectiveness of Artificial Intelligence Conversational Agents in Health Care: Systematic Review
Milne-Ives, M., de Cock, C., Lim, E., Shehadeh, M. H., de Pennington, N., Mole, G., Normando, E., & Meinert, E. (2020) · Journal of Medical Internet Research · doi:10.2196/20346 · Cited by 626
ClinicalBERT: Modeling Clinical Notes and Predicting Hospital Readmission
Huang, K., Altosaar, J., & Ranganath, R. (2019) · arXiv · doi:10.48550/arxiv.1904.05342 · Cited by 618
Galactica: A Large Language Model for Science
Taylor, R., Kardas, M., Cucurull, G., Scialom, T., Hartshorn, A., Saravia, E., Poulton, A., Kerkez, V., & Stojnic, R. (2022) · arXiv · doi:10.48550/arxiv.2211.09085 · Cited by 605
Evaluating the Performance of ChatGPT in Ophthalmology
Antaki, F., Touma, S., Milad, D., El-Khoury, J., & Duval, R. (2023) · Ophthalmology Science · doi:10.1016/j.xops.2023.100324 · Cited by 563 · cited 3×
“14,17,18 Moreover, the success of deep learning models used to analyze ophthalmic investigations d fundus photographs, OCT, visual fi elds, and more d suggests that multimodal LLMs will perform to a high standard in this context too.”
BackgroundAntaki, F., Touma, S., Milad, D., El-Khoury, J., & Duval, R. (2023) · doi:10.1101/2023.01.22.23284882 · Cited by 563
Mitigating Gender Bias in Natural Language Processing: Literature Review
Sun, T., Gaut, A., Tang, S., Huang, Y., ElSherief, M., Zhao, J., Mirza, D., Belding, E., Chang, K.-W., & Wang, W. Y. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1159 · Cited by 492 · cited 1×
“78,79 Other risks to patient safety may arise when LLM applications are misused to spread misinformation or extract con fi dential patient information.”
BackgroundBioWordVec, improving biomedical word embeddings with subword information and MeSH
Zhang, Y., Chen, Q., Yang, Z., Lin, H., & Lu, Z. (2019) · Scientific Data · doi:10.1038/s41597-019-0055-0 · Cited by 455
Patient apprehensions about the use of artificial intelligence in healthcare
Richardson, J. P., Smith, C., Curtis, S., Watson, S., Zhu, X., Barry, B., & Sharp, R. R. (2021) · npj Digital Medicine · doi:10.1038/s41746-021-00509-1 · Cited by 424
Identifying Ethical Considerations for Machine Learning Healthcare Applications
Char, D. S., Abràmoff, M. D., & Feudtner, C. (2020) · The American Journal of Bioethics · doi:10.1080/15265161.2020.1819469 · Cited by 415 · cited 1×
“51,52 The Practitioner Perspective Multimodal LLMs capable of processing images and text are emerging, with important implications for eye care, which relies heavily on large quantities of nontext-based data.”
BackgroundSelf-citationLarge language models are few-shot clinical information extractors
Agrawal, M., Hegselmann, S., Lang, H., Kim, Y., & Sontag, D. (2022) · Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2022.emnlp-main.130 · Cited by 358 · cited 1×
“65 Other models are emerging for protein and genetic analysis, and potential applications in ophthalmology are diverse: drug development, genetic diagnosis, physiological and pathological research, and more.”
BackgroundCan large language models reason about medical questions?
Liévin, V., Hother, C. E., Motzfeldt, A. G., & Winther, O. (2024) · Patterns · doi:10.1016/j.patter.2024.100943 · Cited by 343 · cited 3×
“33 Potential additional features that can be explored include uncertainty-aware LLM applications that provide a probability score of generated responses, along with alternative recommendations when the probability score is low, as well as reporting the differential weights of input tokens that…”
BackgroundCan large language models reason about medical questions?
Liévin, V., Hother, C. E., Motzfeldt, A. G., & Winther, O. (2022) · arXiv · doi:10.48550/arxiv.2207.08143 · Cited by 343
Towards Expert-Level Medical Question Answering with Large Language Models
Singhal, K., Tu, T., Gottweis, J., Sayres, R., Wulczyn, E., Hou, L., Clark, K., Pfohl, S., Cole-Lewis, H., Neal, D., et al. (2023) · arXiv · doi:10.48550/arxiv.2305.09617 · Cited by 338
Predicting conversion to wet age-related macular degeneration using deep learning
Yim, J., Chopra, R., Spitz, T., Winkens, J., Obika, A., Kelly, C., Askham, H., Lukic, M., Huemer, J., Fasler, K., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0867-7 · Cited by 297
Self-citationPutting ChatGPT’s Medical Advice to the (Turing) Test: Survey Study
Nov, O., Singh, N., & Mann, D. (2023) · JMIR Medical Education · doi:10.2196/46939 · Cited by 198
AI-generated research paper fabrication and plagiarism in the scientific community
Elali, F. R., & Rachid, L. N. (2023) · Patterns · doi:10.1016/j.patter.2023.100706 · Cited by 197
Thirunavukarasu, A. J., Hassan, R., Mahmood, S., Sanghera, R., Barzangi, K., El Mukashfi, M., & Shah, S. (2023) · JMIR Medical Education · doi:10.2196/46599 · Cited by 187
Self-citationAI chatbots not yet ready for clinical use.
Au Yeung, J., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023) · Frontiers in Digital Health · doi:10.3389/fdgth.2023.1161098 · Cited by 176
AI chatbots not yet ready for clinical use
Yeung, J. A., Kraljevic, Z., Luintel, A., Balston, A., Idowu, E., Dobson, R. J., & Teo, J. T. (2023) · doi:10.1101/2023.03.02.23286705 · Cited by 176
Kraljevic, Z., Bean, D., Shek, A., Bendayan, R., Hemingway, H., Yeung, J. A., Deng, A., Balston, A., Ross, J., Idowu, E., et al. (2024) · The Lancet Digital Health · doi:10.1016/s2589-7500(24)00025-6 · Cited by 169
Van Riel, N., Auwerx, K., Debbaut, P., Van Hees, S., & Schoenmakers, B. (2017) · BJGP Open · doi:10.3399/bjgpopen17x100833 · Cited by 159
Ignore Previous Prompt: Attack Techniques For Language Models
Perez, F., & Ribeiro, I. (2022) · arXiv · doi:10.48550/arxiv.2211.09527 · Cited by 110 · cited 1×
“81,82 Further, there is growing concern regarding the security of data inputted into LLM applications such as copyrighted material retained as part of training data, as well as the fact that applications like ChatGPT retains users ’ conversation content to improve its model performance.”
BackgroundFoundational Considerations for Artificial Intelligence Using Ophthalmic Images
Abràmoff, M. D., Cunningham, B., Patel, B., Eydelman, M. B., Leng, T., Sakamoto, T., Blodi, B., Grenon, S. M., Wolf, R. M., Manrai, A. K., et al. (2022) · Ophthalmology · doi:10.1016/j.ophtha.2021.08.023 · Cited by 96
Self-citationBlenderBot 3: a deployed conversational agent that continually learns to responsibly engage
Shuster, K., Xu, J., Komeili, M., Ju, D., Smith, E. M., Roller, S., Ung, M., Chen, M., Arora, K., Lane, J., et al. (2022) · arXiv · doi:10.48550/arxiv.2208.03188 · Cited by 95
Large language models will not replace healthcare professionals: curbing popular fears and hype.
Thirunavukarasu, A. J. (2023) · Journal of the Royal Society of Medicine · doi:10.1177/01410768231173123 · Cited by 63 · cited 3×
“44 As with practitioners, LLMs may improve the ef fi ciency and quality of work done by policymakers through implementation of AI-assisted writing, evidence synthesis, and administrative work.”
BackgroundInfluentialSelf-citation
Research works printed but not linked: 17
- Radford, 2018, Improving language understanding by generative pre-training, Comput Sci
- Radford, 2019, Language models are unsupervised multitask learners, Comput Sci, 1, 9
- Raimondi, 2023, Comparative analysis of large language models in the Royal College of Ophthalmologists fellowship exams, Eye, 10.1038/s41433-023-02563-3 · DOI · Paper page
- Lin, 2023, Comparison of GPT-3.5, GPT-4, and human user performance on a practice ophthalmology written examination, Eye, 10.1038/s41433-023-02564-2 · DOI
- Papanikolaou, 2020, DARE: data augmented relation extraction with GPT-2, arXiv preprint arXiv:2004.13845 · arXiv
- Chen, 2022, Applications of natural language processing in ophthalmology: present and future, Front Med (Lausanne), 9
- Thirunavukarasu, 2021, Evaluating the mainstream impact of ophthalmological research with Google Trends, Eye, 35, 3165, 10.1038/s41433-020-01257-4 · DOI
- Pitt, 2022, Providing a second opinion to Dr. Google with the WWW framework, J Gen Intern Med, 37, 222, 10.1007/s11606-021-06895-2 · DOI
- Kuehn, 2013, More than one-third of US individuals use the Internet to self-diagnose, JAMA, 309, 756, 10.1001/jama.2013.629 · DOI
- Korot, 2020, Will AI replace ophthalmologists?, Transl Vis Sci Technol, 9, 2, 10.1167/tvst.9.2.2 · DOI
- Yakar, 2022, Do people favor artificial intelligence over physicians? A survey among the general population and their view on artificial intelligence in medicine, Value Health, 25, 374, 10.1016/j.jval.2021.09.004 · DOI
- Chow, 2022, Patient-centred care in ophthalmology: current practices, effectiveness and challenges, Graefes Arch Clin Exp Ophthalmol, 260, 3149, 10.1007/s00417-022-05666-x · DOI
- Wang, 2022, Leveraging weak supervision to perform named entity recognition in electronic health records progress notes to identify the ophthalmology exam, Int J Med Inform, 167, 10.1016/j.ijmedinf.2022.104864 · DOI
- Mai, 2022, TSSNote-CyaPromBERT: development of an integrated platform for highly accurate promoter prediction and visualization of Synechococcus sp. and Synechocystis sp. through a state-of-the-art natural language processing model BERT, Front Genet, 13, 10.3389/fgene.2022.1067562 · DOI
- Shashikumar, 2021, Artificial intelligence sepsis prediction algorithm learns to say “I don't know”, NPJ Digit Med, 4, 134, 10.1038/s41746-021-00504-6 · DOI · Paper page
- Youssef, 2023, Is the algorithm good in a bad world, or has it learned to be bad? The ethical challenges of “locked” versus “continuously learning” artificial intelligence systems, and "Autonomous" Versus "Assistive" AI Tools in Healthcare, Am J Bioeth, 23, 43, 10.1080/15265161.2023.2191052 · DOI · Paper page
- Korot, 2021, Predicting sex from retinal fundus photographs using automated deep learning, Sci Rep, 11, 10.1038/s41598-021-89743-x · DOI
Cited by
Ong, J. C. L., Seng, B. J. J., Law, J. Z. F., Low, L. L., Kwa, A. L. H., Giacomini, K. M., & Ting, D. S. W. (2024) · Cell Reports Medicine · doi:10.1016/j.xcrm.2023.101356 · Cited by 111
Self-citationLiaw, S. Y., Rusli, K. D. B., Tan, J. Z., Wee, Y. H. C., Neo, N. W. S., & Chua, W. L. (2025) · Nurse Education in Practice · doi:10.1016/j.nepr.2025.104462 · Cited by 17
Evaluation of ChatGPT-4 in detecting referable diabetic retinopathy using single fundus images
Aftab, O., Khan, H., VanderBeek, B. L., Scoles, D., Kim, B. J., & Tsui, J. C. (2025) · AJO International · doi:10.1016/j.ajoint.2025.100111 · Cited by 10
Metadata
- Authors
- Tan, T. F., Thirunavukarasu, A. J., Campbell, J. P., Keane, P. A., Pasquale, L. R., Abramoff, M. D., Kalpathy-Cramer, J., Lum, F., Kim, J. E., Baxter, S. L., & Ting, D. S. W.
- Venue
- Ophthalmology Science
- Publisher
- Elsevier BV
- Published
- 2023-12
- Volume
- 3
- Issue
- 4
- Pages
- 100394
- ISSN
- 2666-9145 (ISSN-L)
- OpenAlex
- W4386572315
- Cited by
- 116
- References
- 44
- FWCI
- 4.01
- Type
- journal article
- Language
- en
- Open access
- gold
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/legal/tdmrep-license (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0/ (vor)
- Funders
- National Medical Research Council (MOH-000655-00, MOH-001014-00, NMRC/HSRG/0087/2018)
- Moorfields Eye Charity (R190028A)
- Research to Prevent Blindness
- Glaucoma Foundation
- Genentech
- National Institutes of Health
- National Eye Institute
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.