Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis
Open accessThe DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Abstract
INTRODUCTION: Accurate diagnosis and timely management are vital for favorable uveitis outcomes. Artificial Intelligence (AI) holds promise in medical decision-making, particularly in ophthalmology. Yet, the diagnostic precision and management advice from AI-based uveitis chatbots lack assessment. METHODS: We appraised diagnostic accuracy and management suggestions of an AI-based chatbot, ChatGPT, versus five uveitis-trained ophthalmologists, using 25 standard cases aligned with new Uveitis Nomenclature guidelines. Participants predicted likely diagnoses, two differentials, and next management steps. Comparative success rates were computed. RESULTS: Ophthalmologists excelled (60-92%) in likely diagnosis, exceeding AI (60%). Considering fully and partially accurate diagnoses, ophthalmologists achieved 76-100% success; AI attained 72%. Despite an 8% AI improvement, its overall performance lagged. Ophthalmologists and AI agreed on diagnosis in 48% cases, with 91.6% exhibiting concurrence in management plans. CONCLUSIONS: The study underscores AI chatbots' potential in uveitis diagnosis and management, indicating their value in reducing diagnostic errors. Further research is essential to enhance AI chatbot precision in diagnosis and recommendations.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Gilson, A., Safranek, C. W., Huang, T., Socrates, V., Chi, L., Taylor, R. A., & Chartash, D. (2023) · JMIR Medical Education · doi:10.2196/45312 · Cited by 2,078
Flanagin, A., Bibbins-Domingo, K., Berkwits, M., & Christiansen, S. L. (2023) · JAMA · doi:10.1001/jama.2023.1344 · Cited by 420 · cited 1×
“11 We hypothesize that the development of an AI model for uveitis diagnosis and initial management could have reasonable accuracy to assist ophthalmologists, especially those without formal uveitis training in remote areas, in providing timely and accurate diagnoses and treatments for their…”
ChatGPT in Clinical Toxicology
Sabry Abdel-Messih, M., & Kamel Boulos, M. N. (2023) · JMIR Medical Education · doi:10.2196/46876 · Cited by 95 · cited 1×
“14 Most of these articles are editorials and comments that discuss its scope, limitations, and ethical considerations; only a few have explored its diagnostic utility in complex diseases, 15,16 and even fewer have examined its usefulness in ophthalmology.”
Assessing the Value of ChatGPT for Clinical Decision Support Optimization
Liu, S., Wright, A. P., Patterson, B. L., Wanderer, J. P., Turer, R. W., Nelson, S. D., McCoy, A. B., Sittig, D. F., & Wright, A. (2023) · doi:10.1101/2023.02.21.23286254 · Cited by 90
Cited by
Large Language Models for Chatbot Health Advice Studies: A Systematic Review
Huo, B., Boyle, A., Marfo, N., Tangamornsuksan, W., Steen, J. P., McKechnie, T., Lee, Y., Mayol, J., Antoniou, S. A., Thirunavukarasu, A. J., et al. (2025) · JAMA Network Open · doi:10.1001/jamanetworkopen.2024.57879 · Cited by 283
Shan, G., Chen, X., Wang, C., Liu, L., Gu, Y., Jiang, H., & Shi, T. (2025) · JMIR Medical Informatics · doi:10.2196/64963 · Cited by 71
Wang, L., Li, J., Zhuang, B., Huang, S., Fang, M., Wang, C., Li, W., Zhang, M., & Gong, S. (2025) · Journal of Medical Internet Research · doi:10.2196/64486 · Cited by 51
Metadata
- Authors
- Rojas-Carabali, W., Cifuentes-González, C., Wei, X., Putera, I., Sen, A., Thng, Z. X., Agrawal, R., Elze, T., Sobrin, L., Kempen, J. H., Lee, B., Biswas, J., Nguyen, Q. D., Gupta, V., de-la-Torre, A., & Agrawal, R.
- Venue
- Ocular Immunology and Inflammation
- Publisher
- Informa UK Limited
- Published
- 2023-09-18
- Volume
- 32
- Issue
- 8
- Pages
- 1526-1531
- ISSN
- 0927-3948 (ISSN-L)
- 1744-5078
- OpenAlex
- W4386820498
- Cited by
- 47
- References
- 4
- FWCI
- 1.63
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis. https://app.cochat.ai/papers/10.1080/09273948.2023.2253471
- The paper
- Rojas-Carabali, W., Cifuentes-González, C., Wei, X., Putera, I., Sen, A., Thng, Z. X., Agrawal, R., Elze, T., Sobrin, L., Kempen, J. H., Lee, B., Biswas, J., Nguyen, Q. D., Gupta, V., de-la-Torre, A., & Agrawal, R. (2024). Evaluating the Diagnostic Accuracy and Management Recommendations of ChatGPT in Uveitis. Ocular Immunology and Inflammation. https://doi.org/10.1080/09273948.2023.2253471
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.