The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; 10 of the 14 sources its claims cite were checked for support.
Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation
Open accessAbstract
Fundus fluorescein angiography (FFA) is the gold standard for diagnosing chorioretinal diseases, but its interpretation requires significant expertise and time. Despite generative AI's enormous potential in medical report generation, automatic FFA interpretation lacks robust models and sufficient evaluation metrics. This study introduces InterpreFFA, a diagnosis-supervised contrastive learning framework, to emulate ophthalmologists' decision-making process in FFA report generation. Validated on multi-center datasets, InterpreFFA demonstrated superior performance and generalization compared to baseline models. In a simulated clinical setting, two residents used InterpreFFA to diagnose and report FFA cases, with six board-certified ophthalmologists rating the generated reports based on a five-point Likert scale. InterpreFFA significantly improved diagnostic accuracy (85.55 to 90.34%, p < 0.05) and shortened reporting time (153.93 to 108.08 s, p < 0.001). Although AI-generated reports scored slightly lower than manual reports (4.12 vs. 4.38, p < 0.01), InterpreFFA proves to be a promising and cost-effective ancillary tool for enhancing clinical efficiency.
Overview
CoChat Summary
TL;DR
InterpreFFA, a diagnosis-supervised contrastive learning framework for fundus fluorescein angiography (FFA) report generation, was developed and validated on multi-center datasets. In a simulated clinical setting, its AI-generated reports improved residents’ diagnostic accuracy and shortened reporting time, though AI-only reports scored lower than manual reports.
Key findings
- InterpreFFA outperformed three baseline models and previous studies on natural language generation (NLG) and clinical efficacy (CE) metrics; it achieved AUCs of 0.959 for “Normal,” 0.890 for “Aneurysm,” and 0.886 for “Occlusion.”
- In the clinical effectiveness test with two residents, AI-aided mode improved diagnostic accuracy from 85.55% to 90.34% (p < 0.05) and reduced average reporting time from 153.93 ± 14.67 s to 108.08 ± 19.29 s (p < 0.001).
- Six board-certified ophthalmologists rated AI-only reports lower than normal template-aided reports (4.12 ± 1.29 vs. 4.38 ± 1.09, p < 0.01); AI-aided reports scored highest (4.46 ± 0.99), but the improvement over normal template-aided reports was not statistically significant.
- For VKH, a rare condition, resident sensitivity dropped from 71.43% to 14.29% with AI assistance; overall, 8.3% of AI-generated reports were rated 1 and needed complete discard, 13.1% rated 2 or 3 required major corrections, and 20.9% rated 4 were accepted with minimal edits.
Methods
- Retrospective multi-center dataset: 20,052 FFA images and 1,833 real-world reports from 1,439 patients across ZJU2 (internal), TZ, and XJU2 (external), covering seven eye conditions; internal data were split 3:1:1 into training, validation, and test sets.
- InterpreFFA used an R2Gen memory-driven Transformer backbone with a ResNet101 visual extractor and a novel diagnosis-supervised contrastive loss; it was compared against R2Gen, R2GenRL, and M2KT baselines.
- Evaluation included NLG metrics (BLEU1–4, METEOR, ROUGE-L), CE metrics (sensitivity, specificity, F1-score, AUC), a simulated report-drafting and diagnosis test with two residents, and Likert-scale ratings from six ophthalmologists; statistical tests included McNemar’s test, Bowker’s test, two-sample t-tests, Kendall W, and ANOVA.
Limitations
- The paper states that only limited eye conditions were included and diseases were unbalanced; rare diseases such as VKH may cause underfitting and misleading AI-generated reports.
- The paper states that generated report quality still needs improvement to match ophthalmologists, that NLG and CE metrics are insufficient to reflect true report quality, and that a more standardized rating rubric is needed due to moderate interrater agreement.
- The paper states that all real-world reports were written in Chinese, and that it could not quantitatively compare how residents used or modified AI-generated reports; the text also makes evident that the clinical evaluation was conducted in a simulated setting rather than a real clinical workflow.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
7 Valid · 1 Not valid · 2 Not checked · 66 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Together with the improvements in reporting time and diagnostic accuracy, our proposed InterpreFFA model still offers high cost-effectiveness given the tremendous reporting volume and the shortage of ophthalmologists in real-world clinical settings27,28.”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Discussion
What the sources say
- Partly supportedcites [27] Wang, Y. et al. Economic evaluation for medical artificial intelligence: accuracy vs. cost-effectiveness in a diabetic retinopathy screening case. NPJ Digit. Med. 7, 43 (2024). sourceExperimental — accuracy not yet measured
The Discussion (page unavailable) says AI-based diabetic-retinopathy screening can reduce clinical workload and that cost-effectiveness matters in real-world use. It does not evaluate InterpreFFA’s cost-effectiveness or establish that reporting volume and a shortage of ophthalmologists make that model cost-effective. (LLM, conf=0.84)
- Not checked — no full text of the sourcecites [28] Resnikoff, S., Felch, W., Gauthier, T. M. & Spivey, B. The number of ophthalmologists in practice and training worldwide: a growing gap despite more than 200 000 practitioners. Br. J. Ophthalmol. 96, 783–787 (2012). source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All types of errors were least-occurred in AI-aided reports.”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Results > Evaluation by ophthalmologists
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, only limited types of eye conditions were included in the datasets, and the diseases in the datasets were unbalanced.”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper states — it cites no source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“First, only limited types of eye conditions were included in the datasets, and the diseases in the datasets were unbalanced.”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Discussion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In a simulated clinical setting, these AI-generated reports improved residents’ diagnostic accuracy and quality of report while significantly shortening the average reporting time.”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Discussion
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
33 more claims
Create a free account to see all 76 claims and every source verdict.
- It may state two separate claims as one. 26
- Probably not a substantial claim. 20
- The quote only partly states this claim. 2
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Female | 269 (39.1) | 120 (45.1) | 138 (52.9) | 66 (52.4) | 58 (38.9) | 651 (45.2) |”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Results
Reported 39.1 · recomputed 42.229199
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| Male | 388 (60.9) | 146 (54.9) | 123 (47.1) | 60 (47.6) | 91 (61.1) | 788 (54.8) |”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Results
Reported 788 · recomputed 808
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Abstract
“InterpreFFA significantly improved diagnostic accuracy (85.55 to 90.34%, p < 0.05)”
§ Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation > Results > Evaluation of clinical effectiveness
“| Total | 86.55 (82.14 - 90.00) | 86.55 (82.14 - 90.00) | 100.00 (98.69 - 100.00) | 90.34 (86.40 - 93.24)”
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · cited 1×
“For the FFA images from each case, we employed the ResNet101 pretrainedonImageNet-1katresolution224 × 224asthevisualextractorto extract patch features with a dimension of 2048 30,31 .”
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
BLEU: a method for automatic evaluation of machine translation
Papineni, K., Roukos, S., Ward, T., & Zhu, W.-J. (2001) · Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02 · doi:10.3115/1073083.1073135 · Cited by 21,344
Generating Radiology Reports via Memory-driven Transformer
Chen, Z., Song, Y., Chang, T.-H., & Wan, X. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.112 · Cited by 693 · cited 2×
“Notably,R2Gen,oneofthemostwidelyusedstate-of-the-artmedicalreport generationmodels,demonstratedanexceptionalabilitytogeneratelong reportswithessentialmedicaltermsandmeaningfulimage-textattention mappings 29 .”
A pathology foundation model for cancer diagnosis and prognosis prediction
Wang, X., Zhao, J., Marostica, E., Yuan, W., Jin, J., Zhang, J., Li, R., Tang, H., Wang, K., Li, Y., et al. (2024) · Nature · doi:10.1038/s41586-024-07894-z · Cited by 580
Self-citationA multimodal generative AI copilot for human pathology
Lu, M. Y., Chen, B., Williamson, D. F. K., Chen, R. J., Zhao, M., Chow, A. K., Ikemura, K., Kim, A., Pouli, D., Patel, A., et al. (2024) · Nature · doi:10.1038/s41586-024-07618-3 · Cited by 461 · cited 1×
“…in generative AI have signi fi cantly expanded its application in medicine, yielding promising results in assisting physicians with interpreting different medical images, enhancing human-in-the-loop clinical decision-making and supporting disease diagnosis and prognosis prediction 5 – 7 .”
Resnikoff, S., Felch, W., Gauthier, T.-M., & Spivey, B. (2012) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2011-301378 · Cited by 405
Resnikoff, S., Lansingh, V. C., Washburn, L., Felch, W., Gauthier, T.-M., Taylor, H. R., Eckert, K., Parke, D., & Wiedemann, P. (2020) · British Journal of Ophthalmology · doi:10.1136/bjophthalmol-2019-314336 · Cited by 382 · cited 1×
“Given the increasing orders in FFA and scarcity of experienced ophthal-mologists, particularly in remote or densely populated areas, a reliable and generalizable AI-assisted system for FFA image interpretation is urgently demanded 3 .”
Potential of ChatGPT and GPT-4 for Data Mining of Free-Text CT Reports on Lung Cancer
Fink, M. A., Bischoff, A., Fink, C. A., Moll, M., Kroschke, J., Dulz, L., Heußel, C. P., Kauczor, H.-U., & Weber, T. F. (2023) · Radiology · doi:10.1148/radiol.231362 · Cited by 277
Vision–language foundation model for echocardiogram interpretation
Christensen, M., Vukadinovic, M., Yuan, N., & Ouyang, D. (2024) · Nature Medicine · doi:10.1038/s41591-024-02959-y · Cited by 231 · cited 1×
“…in generative AI have signi fi cantly expanded its application in medicine, yielding promising results in assisting physicians with interpreting different medical images, enhancing human-in-the-loop clinical decision-making and supporting disease diagnosis and prognosis prediction 5 – 7 .”
Research works printed but not linked: 2
- Denkowski, M. & Lavie, A. Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. Proceedings of the Sixth Workshop on Statistical Machine Translation 85–91 (Association for Computational Linguistics, 2011).
- Lin, C.-Y. ROUGE: a package for automatic evaluation of summaries. In Text Summarization Branches Out 74–81 (Association for Computational Linguistics, 2004).
Cited by
Report Generation System for Slit-Lamp Image Interpretation Using Vision-Language Models
Ye, X., Shen, Y., Chen, Q., He, X., Lü, X., Grzybowski, A. E., Jin, K., & Xie, W. (2026) · Ophthalmology and Therapy · doi:10.1007/s40123-026-01352-x · Cited by 2
Self-citationDuan, M.-M., Qi, H., & Tu, X. (2025) · Frontiers in Cell and Developmental Biology · doi:10.3389/fcell.2025.1703606 · Cited by 2
“…our model’s ability to differentiate between multiple pathologies, including CSC, VO, nAMD, and different stages of DR (Long et al., 2025; Gan et al., 2023), more closely mirrors the complex diagnostic challenges encountered in real-world clinical practice (Shao et al., 2025; Ueno et al., 2019).”
Generative AI in healthcare: foundations, applications, challenges and future directions
Zheng, J., Li, B., Li, H., & Lu, Y. (2026) · Journal of Management Analytics · doi:10.1080/23270012.2026.2647940 · Cited by 6
SFGNet: A spectral fusion-gated network for dynamic microvascular segmentation in OCTA
Su, W., Ma, F., Li, S., et al. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.110032 · Cited by 1
Shen, Y., Chen, Q., He, X., Agrawal, R. V., Grzybowski, A. E., Jin, K., & Ye, X. (2026) · Ophthalmology and Therapy · doi:10.1007/s40123-026-01316-1 · Cited by 4
“…approaches, state-of-the-art models such as InterpreFFA leverage memory-driven transformer architectures like R2Gen, which capture dependencies from previous generation steps to produce comprehensive reports containing essential medical terminology and meaningful cross-modal attention maps [22].”
InfluentialSelf-citation
Metadata
- Authors
- Shao, A., Liu, X., Shen, W., Li, Y., Wu, H., Pan, X., Yang, Z., Xu, Y., Zhu, T., Wang, Y., Yang, J., Tham, Y. C., Wu, J., Jin, K., & Ye, J.
- Affiliations
- Second Affiliated Hospital of Zhejiang University (CN)
- Zhejiang University (CN)
- China Jiliang University (CN)
- Brigham and Women's Hospital (US)
- National University of Singapore (SG)
- Corresponding author
- Wu, Jian
- Jin, Kai
- Ye, Juan
- ORCID
- Venue
- npj Digital Medicine
- Published
- 2025-07-02
- Volume
- 8
- Issue
- 1
- Pages
- 396-396
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC12222453
- OpenAlex
- W4411918684
- Cited by
- 16
- References
- 33
- Citations by year
- 2026: 14 · 2025: 2
- FWCI
- 9.22
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc-nd
- creativecommons.org/licenses/by-nc-nd/4.0 (tdm)
- creativecommons.org/licenses/by-nc-nd/4.0 (vor)
- Locations
- npj Digital Medicine (publishedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (82201195)
- National Natural Science Foundation Regional Innovation and Development Joint Fund (U20A20386)
- Key Program of the National Natural Science Foundation of China (82330032)
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Keywords
- fundus fluorescein angiography, generative AI, automated reporting, contrastive learning, chorioretinal diseases, diagnostic accuracy, clinical efficiency
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation. https://app.cochat.ai/papers/10.1038/s41746-025-01759-z
- The paper
- Shao, A., Liu, X., Shen, W., Li, Y., Wu, H., Pan, X., Yang, Z., Xu, Y., Zhu, T., Wang, Y., Yang, J., Tham, Y. C., Wu, J., Jin, K., & Ye, J. (2025). Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation. npj Digital Medicine. https://doi.org/10.1038/s41746-025-01759-z
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.