Performance of artificial intelligence in diabetic retinopathy screening: a systematic review and meta-analysis of prospective studies
Open accessThe DOI matches its Crossref record; 57 of the 57 references it cites were found in a registry or an index.
Abstract
Aims: To systematically evaluate the diagnostic value of an artificial intelligence (AI) algorithm model for various types of diabetic retinopathy (DR) in prospective studies over the previous five years, and to explore the factors affecting its diagnostic effectiveness. Materials and methods: A search was conducted in Cochrane Library, Embase, Web of Science, PubMed, and IEEE databases to collect prospective studies on AI models for the diagnosis of DR from January 2017 to December 2022. We used QUADAS-2 to evaluate the risk of bias in the included studies. Meta-analysis was performed using MetaDiSc and STATA 14.0 software to calculate the combined sensitivity, specificity, positive likelihood ratio, and negative likelihood ratio of various types of DR. Diagnostic odds ratios, summary receiver operating characteristic (SROC) plots, coupled forest plots, and subgroup analysis were performed according to the DR categories, patient source, region of study, and quality of literature, image, and algorithm. Results: Finally, 21 studies were included. Meta-analysis showed that the pooled sensitivity, specificity, pooled positive likelihood ratio, pooled negative likelihood ratio, area under the curve, Cochrane Q index, and pooled diagnostic odds ratio of AI model for the diagnosis of DR were 0.880 (0.875-0.884), 0.912 (0.99-0.913), 13.021 (10.738-15.789), 0.083 (0.061-0.112), 0.9798, 0.9388, and 206.80 (124.82-342.63), respectively. The DR categories, patient source, region of study, sample size, quality of literature, image, and algorithm may affect the diagnostic efficiency of AI for DR. Conclusion: AI model has a clear diagnostic value for DR, but it is influenced by many factors that deserve further study. Systematic review registration: https://www.crd.york.ac.uk/prospero/, identifier CRD42023389687.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
1 retracted
RETRACTED ARTICLE: Artificial intelligence and machine learning in precision and genomic medicine
Quazi, S. (2022) · Medical Oncology · doi:10.1007/s12032-022-01711-1 · Cited by 432 · cited 1×
“Meanwhile, the application of AI in the medical fi eld is being continuously developed and subdivided (39).”
RetractedThe PRISMA 2020 statement: An updated guideline for reporting systematic reviews
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., Akl, E. A., Brennan, S. E., et al. (2021) · BMJ · doi:10.1136/bmj.n71 · Cited by 108,283
QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies
Whiting, P. F., Rutjes, A. W. S., Westwood, M. E., Mallett, S., Deeks, J. J., Reitsma, J. B., Leeflang, M. M. G., Sterne, J. A. C., Bossuyt, P. M. M., & the QUADAS-2 Group* (2011) · Annals of Internal Medicine · doi:10.7326/0003-4819-155-8-201110180-00009 · Cited by 14,199
Ting, D. S. W., Cheung, C. Y.-L., Lim, G., Tan, G. S. W., Quang, N. D., Gan, A., Hamzah, H., Garcia-Franco, R., San Yeo, I. Y., Lee, S. Y., et al. (2017) · JAMA · doi:10.1001/jama.2017.18152 · Cited by 2,402 · cited 1×
“(34) 2017 China a retinal specialist(>10 years’ experience) RDR 976 2929 102 31941 35948”
Abràmoff, M. D., Lavin, P. T., Birch, M., Shah, N., & Folk, J. C. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0040-6 · Cited by 1,638 · cited 2×
“Since the researchers did not clarify the pre-training degree and learning amount of each model, we could not include these factors in the analysis; (7) there are problems with direct comparison of diagnostic accuracy.”
Artificial intelligence and deep learning in ophthalmology
Ting, D. S. W., Pasquale, L. R., Peng, L., Campbell, J. P., Lee, A. Y., Raman, R., Tan, G. S. W., Schmetterer, L., Keane, P. A., & Wong, T. Y. (2019) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2018-313173 · Cited by 1,554 · cited 1×
“However, due to the different image quality of fundus color photos, algorithms used by AI models, and performance of cameras used in different studies, the AI diagnostic effect varies signi fi cantly among different studies (40, 41).”
Artificial Intelligence and Deep Learning in Ophthalmology
Wang, Z., Keane, P. A., Chiang, M., Cheung, C. Y., Wong, T. Y., & Ting, D. S. W. (2022) · Artificial Intelligence in Medicine · doi:10.1007/978-3-030-64573-1_200 · Cited by 1,524
A deep learning system for detecting diabetic retinopathy across the disease spectrum
Dai, L., Wu, L., Li, H., Cai, C., Wu, Q., Kong, H., Liu, R., Wang, X., Hou, X., Liu, Y., et al. (2021) · Nature Communications · doi:10.1038/s41467-021-23458-5 · Cited by 620
Diabetic retinopathy: Looking forward to 2030
Tan, T.-E., & Wong, T. Y. (2023) · Frontiers in Endocrinology · doi:10.3389/fendo.2022.1077669 · Cited by 511
Wong, T. Y., & Sabanayagam, C. (2020) · Ophthalmologica · doi:10.1159/000502387 · Cited by 310
Gulshan, V., Rajan, R. P., Widner, K., Wu, D., Wubbels, P., Rhodes, T., Whitehouse, K., Coram, M., Corrado, G., Ramasamy, K., et al. (2019) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2019.2004 · Cited by 303 · cited 1×
“(21) 2019 India retinal specialist mtmDR 615 95 77 1118 1905 ey”
10. Microvascular Complications and Foot Care: Standards of Medical Care in Diabetes—2018
American Diabetes Association (2018) · Diabetes Care · doi:10.2337/dc18-s010 · Cited by 302 · cited 2×
“As a common complication of diabetes, it is estimated that up to 30% of diabetic patients will eventually develop various types of DR (3), and 10% of diabetic patients are at risk of blindness (4).”
Artificial intelligence: Who is responsible for the diagnosis?
Neri, E., Coppola, F., Miele, V., Bibbolino, C., & Grassi, R. (2020) · La radiologia medica · doi:10.1007/s11547-020-01135-9 · Cited by 240 · cited 1×
“Even if AI can diagnose DR independently, the ophthalmologist will eventually have to issue a report and take legal risks (53); therefore, legislation is required to clarify the respective scope of responsibility between doctors and companies providing AI services, which may also promote the popularization of AI diagn…”
Natarajan, S., Jain, A., Krishnan, R., Rogye, A., & Sivaprasad, S. (2019) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2019.2923 · Cited by 228 · cited 1×
“(24) 2019 India a vitreoretinal resident and a vitreoretinal surgeon RDR 23 60 0 311 394 Po”
Prevalence of type 2 diabetes and its complications in India and economic costs to the nation
Pradeepa, R., & Mohan, V (2017) · European Journal of Clinical Nutrition · doi:10.1038/ejcn.2017.40 · Cited by 200
Xie, Y., Zaccagna, F., Rundo, L., Testa, C., Agati, R., Lodi, R., Manners, D. N., & Tonon, C. (2022) · Diagnostics · doi:10.3390/diagnostics12081850 · Cited by 160
Artificial Intelligence for Screening of Multiple Retinal and Optic Nerve Diseases
Dong, L., He, W., Zhang, R., Ge, Z., Wang, Y. X., Zhou, J., Xu, J., Shao, L., Wang, Q., Yan, Y., et al. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.9960 · Cited by 157
Self-citationDiabetic retinopathy screening in the emerging era of artificial intelligence
Grauslund, J. (2022) · Diabetologia · doi:10.1007/s00125-022-05727-0 · Cited by 146 · cited 2×
“However, due to the different image quality of fundus color photos, algorithms used by AI models, and performance of cameras used in different studies, the AI diagnostic effect varies significantly among different studies (40, 41).”
McGrath, T. A., Alabousi, M., Skidmore, B., Korevaar, D. A., Bossuyt, P. M. M., Moher, D., Thombs, B., & McInnes, M. D. F. (2017) · Systematic Reviews · doi:10.1186/s13643-017-0590-8 · Cited by 142
Gomez Rossi, J., Rojas-Perilla, N., Krois, J., & Schwendicke, F. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2022.0269 · Cited by 141 · cited 1×
“Moreover, studies have shown that AI-based DR screening is more cost-effective than manual grading, and may help in providing cost-effective, convenient, and effective medical services (49).”
Nadeem, M. W., Goh, H. G., Hussain, M., Liew, S., Andonović, I., & Khan, M. A. (2022) · Sensors · doi:10.3390/s22186780 · Cited by 140
Artificial intelligence for diabetic retinopathy screening, prediction and management
Gunasekeran, D. V., Ting, D. S. W., Tan, G. S. W., & Wong, T. Y. (2020) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000693 · Cited by 133 · cited 1×
“Frontiers in Endocrinology frontiersin.org pattern of manual identi fi cation, which is easily affected by the experience, skills, and other factors related to the ophthalmologist or relevant technical personnel; therefore, the ef fi ciency of DR screening is relatively low (45).”
Current status and future trends of clinical diagnoses via image-based deep learning
Xu, J., Xue, K., & Zhang, K. (2019) · Theranostics · doi:10.7150/thno.38065 · Cited by 115 · cited 1×
“This model lacks “ explanatory ability ” for the diagnosed diseases, that is, it cannot provide the reasons for the diagnostic results to clinicians (50, 51).”
Zhang, Y., Shi, J., Peng, Y., Zhao, Z., Zheng, Q., Wang, Z., Liu, K., Jiao, S., Qiu, K., Zhou, Z., et al. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2020-001596 · Cited by 109
Self-citationWu, J.-H., Liu, T. Y. A., Hsu, W.-T., Ho, J. H.-C., & Lee, C.-C. (2021) · Journal of Medical Internet Research · doi:10.2196/23863 · Cited by 108
Public Perceptions of Artificial Intelligence and Robotics in Medicine
(2020) · doi:10.1089/end.2020.0137 · Cited by 107 · cited 1×
“What is satisfactory is that currently both doctors and patients have a positive attitude towards the diagnostic ef fi cacy of AI (54, 55), which may lay a foundation for their subsequent cooperation.”
Artificial intelligence: the unstoppable revolution in ophthalmology
Benet, D., & Pellicer-Valero, Ó. J. (2021) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2021.03.003 · Cited by 107
Sandhu, H. S., Elmogy, M., Taher Sharafeldeen, A., Elsharkawy, M., El-Adawy, N., Eltanboly, A., Shalaby, A., Keynton, R., & El-Baz, A. (2020) · American Journal of Ophthalmology · doi:10.1016/j.ajo.2020.01.016 · Cited by 92 · cited 2×
“(29) 2020 US clinical ophthalmologists any DR 75 1 0 35 111 a s m”
Scheetz, J., Koca, D., McGuinness, M., Holloway, E., Tan, Z., Zhu, Z., O’Day, R., Sandhu, S., MacIsaac, R. J., Gilfillan, C., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-94178-5 · Cited by 88 · cited 2×
“(30) 2021 RDR 50 56 years (median) type 1 diabetes (34%) type 2 diabetes (65%) NM NM 13 years (median) NM NM NM”
Sosale, B., Aravind, S. R., Murthy, H., Narayana, S., Sharma, U., Gowda, S. G. V., & Naveenam, M. (2020) · BMJ Open Diabetes Research & Care · doi:10.1136/bmjdrc-2019-000892 · Cited by 85 · cited 2×
“(32) 2019 India retina specialist RDR 231 49 17 603 900”
Tang, F., Luenam, P., Ran, A. R., Quadeer, A. A., Raman, R., Sen, P., Khan, R., Anantharaman, G., Haridas, S., Iglicki, M., et al. (2021) · Ophthalmology Retina · doi:10.1016/j.oret.2021.01.013 · Cited by 77
London, A. J. (2022) · Cell Reports Medicine · doi:10.1016/j.xcrm.2022.100622 · Cited by 76 · cited 3×
“It involves the analysis and evaluation of image-related data through the establishment of relevant databases and application models, and information processing (5).”
Sosale, B., Sosale, A., Murthy, H., Sengupta, S., & Naveenam, M. (2020) · Indian Journal of Ophthalmology · doi:10.4103/ijo.ijo_1203_19 · Cited by 69 · cited 2×
“(31) 2020 India vitreoretinal specialist RDR 121 23 1 152 297”
Teo, Z. L., Tham, Y.-C., Yu, M., Cheng, C.-Y., Wong, T. Y., & Sabanayagam, C. (2020) · Eye · doi:10.1038/s41433-020-0776-5 · Cited by 62 · cited 1×
“Secondly, DR Patients in remote areas can miss the opportunity to undergo timely treatment due to the lack of skilled ophthalmologists (46).”
Nebbioso, M., Lambìase, A., Armentano, M., Tucciarone, G., Sacchetti, M., Greco, A., & Alisi, L. (2021) · Survey of Ophthalmology · doi:10.1016/j.survophthal.2021.04.003 · Cited by 57 · cited 5×
“Criteria for inclusion in the meta-analysis were: (1) the study was a diagnostic study; (2) the subjects were type 2 or type 1 diabetic patients with DR; (3) the diagnostic measure was AI technology, and DR was diagnosed from fundus color images; (4) the study was complete with available data on sensitivity (SE), spec…”
BackgroundMethodResultInfluentialRogers, T. W., Gonzalez-Bueno, J., Garcia Franco, R., Lopez Star, E., Méndez Marín, D., Vassallo, J., Lansingh, V. C., Trikha, S., & Jaccard, N. (2021) · Eye · doi:10.1038/s41433-020-0927-8 · Cited by 56
Yip, M. Y. T., Lim, G., Lim, Z. W., Nguyen, Q. D., Chong, C. C. Y., Yu, M., Bellemo, V., Xie, Y., Lee, X. Q., Hamzah, H., et al. (2020) · npj Digital Medicine · doi:10.1038/s41746-020-0247-1 · Cited by 55 · cited 1×
“Further exploration of imaging features, increasing the sample size of the training set and test set, or further improving the performance of the algorithm are all feasible methods to solve these problems (43).”
Wewetzer, L., Held, L. A., & Steinhäuser, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0255034 · Cited by 52 · cited 1×
“At present, diagnostic meta-analyses on the accuracy of DR detection by AI have mainly focused on a specific algorithm (14, 15).”
BackgroundMing, S., Xie, K., Lei, X., Yang, Y., Zhao, Z., Li, S., Jin, X., & Lei, B. (2021) · International Ophthalmology · doi:10.1007/s10792-020-01685-x · Cited by 45
Baget‐Bernaldiz, M., Romero‐Aroca, P., Santos-Blanco, E., Navarro-Gil, R., Valls, A., Moreno, A., Rashwan, H. A., & Puig, D. (2021) · Diagnostics · doi:10.3390/diagnostics11081385 · Cited by 43
Rêgo, S., Dutra-Medeiros, M., Soares, F., & Monteiro‐Soares, M. (2020) · Ophthalmologica · doi:10.1159/000512638 · Cited by 42
Ogunyemi, O. I., Gandhi, M., Lee, M., Teklehaimanot, S., Daskivich, L. P., Hindman, D. W., Lopez, K., & Taira, R. K. (2021) · JAMIA Open · doi:10.1093/jamiaopen/ooab066 · Cited by 32
Li, N., Ma, M., Lai, M., Gu, L., Kang, M., Wang, Z., Jiao, S., Dang, K., Deng, J., Ding, X., et al. (2022) · Journal of Diabetes · doi:10.1111/1753-0407.13241 · Cited by 32 · cited 1×
“(22) 2022 China retinal specialist RDR 147 44 9 947 1147 Sh”
Self-citationNovel Treatments for Diabetic Macular Edema and Proliferative Diabetic Retinopathy
Le, N. T., Kroeger, Z., Lin, W. V., Khanani, A. M., & Weng, C. Y. (2021) · Current Diabetes Reports · doi:10.1007/s11892-021-01412-5 · Cited by 31 · cited 1×
“Clinically, proliferative diabetic retinopathy (PDR) patients are usually treated with laser, intravitreous injection of anti-vascular endothelial growth factor (VEGF), or corticosteroid drugs (47).”
Noriega-Campero, A., Meizner, D., Camacho‐García‐Formentí, D., Enciso, J., Quiroz-Mercado, H., Morales-Cantón, V., Almaatouq, A., & Pentland, A. (2021) · JMIR Formative Research · doi:10.2196/25290 · Cited by 30
do Rio, J. M. N., Nderitu, P., Bergeles, C., Sivaprasad, S., Tan, G. S. W., & Raman, R. (2022) · Journal of Clinical Medicine · doi:10.3390/jcm11030614 · Cited by 26
Wang, Y., Shi, D., Tan, Z., Niu, Y. Y., Jiang, Y., Xiong, R., Peng, G., & He, M. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.740987 · Cited by 23 · cited 1×
“(35) 2021 China three trained graders RDR 192 291 6 2115 2604”
Yang, C. G., Liu, Q., Guo, H., Zhang, M., Zhang, L., Zhang, G., Zeng, J., Huang, Z., Meng, Q., & Cui, Y. (2021) · Frontiers in Medicine · doi:10.3389/fmed.2021.773881 · Cited by 22
Wongchaisuwat, N., Trinavarat, A., Rodanant, N., Thoongsuwan, S., Phasukkijwatana, N., Prakhunhungsit, S., Preechasuk, L., & Wongchaisuwat, P. (2021) · Translational Vision Science & Technology · doi:10.1167/tvst.10.13.17 · Cited by 19 · cited 2×
“(36) 2021 RDR NM NM NM NM NM NM NM NM NM”
Pawar, B., Lobo, S. N., Joseph, M. A., Jegannathan, S., & Jayraj, H. (2021) · Middle East African Journal of Ophthalmology · doi:10.4103/meajo.meajo_406_20 · Cited by 15 · cited 1×
“(26) 2021 India four ophthalmologists RDR 47 14 0 150 211 rura”
BackgroundCao, B., Zhang, N., Zhang, Y., Fu, Y., & Zhao, D. (2020) · Aging · doi:10.18632/aging.202168 · Cited by 13
Interpretation of artificial intelligence studies for the ophthalmologist
Tan, T., Xu, X., Wang, Z., Liu, Y., & Ting, D. S. W. (2020) · Current Opinion in Ophthalmology · doi:10.1097/icu.0000000000000695 · Cited by 9 · cited 1×
“If the above treatment is given at an early stage to patients with non-proliferative diabetic retinopathy (NPDR) due to an incorrect diagnosis, it will not only waste medical resources, but may also cause serious complications (48).”
Self-citationYao, H.-Y., Wu, S., Zhan, Z., & Li, Z. (2022) · Journal of Ophthalmology · doi:10.1155/2022/9681034 · Cited by 9
Self-citationArtificial Intelligence and the Risk for Intuition Decline in Clinical Medicine
Duarte‐Rojo, A., & Sejdić, E. (2022) · The American Journal of Gastroenterology · doi:10.14309/ajg.0000000000001618 · Cited by 8 · cited 1×
“This model lacks “ explanatory ability ” for the diagnosed diseases, that is, it cannot provide the reasons for the diagnostic results to clinicians (50, 51).”
Jeong, S. H., Han, J. I., Cho, S. W., Lee, D. W., Kim, C. G., Lee, T. G., & Kim, J. W. (2016) · International Journal of Ophthalmology · doi:10.18240/ijo.2016.10.12 · Cited by 5 · cited 1×
“If the above treatment is given at an early stage to patients with non-proliferative diabetic retinopathy (NPDR) due to an incorrect diagnosis, it will not only waste medical resources, but may also cause serious complications (48).”
Bode, B. W., Ipp, E., Shah, V. N., & Sadda, S. R. (2019) · Diabetes · doi:10.2337/db19-37-lb · Cited by 2
Noriega, A., Meizner, D., Camacho‐García‐Formentí, D., et al. (2020) · doi:10.2196/preprints.25290 · Cited by 1
Cited by
Zhang, Z., Deng, C., & Paulus, Y. M. (2024) · Biomedicines · doi:10.3390/biomedicines12071405 · Cited by 99
“Concomitantly, Gargeya et al. developed a customized CNN for detecting DR and validated it against a larger dataset of color fundus photos from multiple independent sources, emphasizing the generalizability of their model.”
MethodS.Rajarajeshwari, G. K. P., & Selvi, G. C. (2024) · IEEE Access · doi:10.1109/access.2024.3494840 · Cited by 68
“…tools, TL experimentation and have highlighted the recent developments in automated-based DR classification techniques in ML and DL. Reference [18] have shed light on the selection and characteristics of eligible studies, quality assessment, threshold analysis, and heterogeneity test,…”
Kong, M., & Song, S. J. (2024) · Endocrinology and Metabolism · doi:10.3803/enm.2023.1913 · Cited by 52
“The performance of AI in diagnosing DR was found to be promising, with the sensitivity, specificity, and AUC reported as 0.880, 0.912, and 0.9798, respectively [23].”
BackgroundInfluentialChen, K.-S., Chan, H.-C., & Chan, C.-M. (2025) · Diabetes Research and Clinical Practice · doi:10.1016/j.diabres.2025.112379 · Cited by 40
Alqahtani, A. S., Alshareef, W. M., Aljadani, H. T., Hawsawi, W. O., & Shaheen, M. H. (2025) · International Journal of Retina and Vitreous · doi:10.1186/s40942-025-00670-9 · Cited by 47
“These differences in AI training create a varying diagnostic effect leading to high heterogeneity levels [28].”
Metadata
- Authors
- Wang, Z., Li, Z., Li, K., Mu, S., Zhou, X., & Di, Y.
- Affiliations
- Sheng Jing Hospital (CN)
- Corresponding author
- Di, Yu
- Venue
- Frontiers in Endocrinology
- Publisher
- Frontiers Media SA
- Published
- 2023-06-13
- Volume
- 14
- Pages
- 1197783-1197783
- ISSN
- 1664-2392 (ISSN-L)
- PMC
- PMC10296189
- OpenAlex
- W4380626653
- Cited by
- 27
- References
- 55
- Citations by year
- 2026: 6 · 2025: 15 · 2024: 6
- FWCI
- 1.14
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Frontiers in Endocrinology (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China
- Foundation of Liaoning Province Education Administration
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- diabetic retinopathy, artificial intelligence, automated retinal analysis, diagnostic accuracy, sensitivity and specificity, likelihood ratios, diagnostic odds ratio
- MeSH terms
- Algorithms, Artificial Intelligence, Diabetes Mellitus, Diabetic Retinopathy, Humans, Prospective Studies, Software
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.