Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 33 of the 33 references it cites were found in a registry or an index; 10 of the 14 sources its claims cite were checked for support.

Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation

Open access

Abstract

Fundus fluorescein angiography (FFA) is the gold standard for diagnosing chorioretinal diseases, but its interpretation requires significant expertise and time. Despite generative AI's enormous potential in medical report generation, automatic FFA interpretation lacks robust models and sufficient evaluation metrics. This study introduces InterpreFFA, a diagnosis-supervised contrastive learning framework, to emulate ophthalmologists' decision-making process in FFA report generation. Validated on multi-center datasets, InterpreFFA demonstrated superior performance and generalization compared to baseline models. In a simulated clinical setting, two residents used InterpreFFA to diagnose and report FFA cases, with six board-certified ophthalmologists rating the generated reports based on a five-point Likert scale. InterpreFFA significantly improved diagnostic accuracy (85.55 to 90.34%, p < 0.05) and shortened reporting time (153.93 to 108.08 s, p < 0.001). Although AI-generated reports scored slightly lower than manual reports (4.12 vs. 4.38, p < 0.01), InterpreFFA proves to be a promising and cost-effective ancillary tool for enhancing clinical efficiency.

Overview

CoChat Summary

TL;DR

InterpreFFA, a diagnosis-supervised contrastive learning framework for fundus fluorescein angiography (FFA) report generation, was developed and validated on multi-center datasets. In a simulated clinical setting, its AI-generated reports improved residents’ diagnostic accuracy and shortened reporting time, though AI-only reports scored lower than manual reports.

Key findings

  • InterpreFFA outperformed three baseline models and previous studies on natural language generation (NLG) and clinical efficacy (CE) metrics; it achieved AUCs of 0.959 for “Normal,” 0.890 for “Aneurysm,” and 0.886 for “Occlusion.”
  • In the clinical effectiveness test with two residents, AI-aided mode improved diagnostic accuracy from 85.55% to 90.34% (p < 0.05) and reduced average reporting time from 153.93 ± 14.67 s to 108.08 ± 19.29 s (p < 0.001).
  • Six board-certified ophthalmologists rated AI-only reports lower than normal template-aided reports (4.12 ± 1.29 vs. 4.38 ± 1.09, p < 0.01); AI-aided reports scored highest (4.46 ± 0.99), but the improvement over normal template-aided reports was not statistically significant.
  • For VKH, a rare condition, resident sensitivity dropped from 71.43% to 14.29% with AI assistance; overall, 8.3% of AI-generated reports were rated 1 and needed complete discard, 13.1% rated 2 or 3 required major corrections, and 20.9% rated 4 were accepted with minimal edits.

Methods

  • Retrospective multi-center dataset: 20,052 FFA images and 1,833 real-world reports from 1,439 patients across ZJU2 (internal), TZ, and XJU2 (external), covering seven eye conditions; internal data were split 3:1:1 into training, validation, and test sets.
  • InterpreFFA used an R2Gen memory-driven Transformer backbone with a ResNet101 visual extractor and a novel diagnosis-supervised contrastive loss; it was compared against R2Gen, R2GenRL, and M2KT baselines.
  • Evaluation included NLG metrics (BLEU1–4, METEOR, ROUGE-L), CE metrics (sensitivity, specificity, F1-score, AUC), a simulated report-drafting and diagnosis test with two residents, and Likert-scale ratings from six ophthalmologists; statistical tests included McNemar’s test, Bowker’s test, two-sample t-tests, Kendall W, and ANOVA.

Limitations

  • The paper states that only limited eye conditions were included and diseases were unbalanced; rare diseases such as VKH may cause underfitting and misleading AI-generated reports.
  • The paper states that generated report quality still needs improvement to match ophthalmologists, that NLG and CE metrics are insufficient to reflect true report quality, and that a more standardized rating rubric is needed due to moderate interrater agreement.
  • The paper states that all real-world reports were written in Chinese, and that it could not quantitatively compare how residents used or modified AI-generated reports; the text also makes evident that the clinical evaluation was conducted in a simulated setting rather than a real clinical workflow.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41746-025-01759-z

Cites

33 of 33 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 The workflow of this study. This study includes four major parts: I. Dataset establishment; II. Model development; III. Automatic evaluation; IV. Clinical evaluation. NLG natural language generation, CE clinical efficacy.
Fig. 1 The workflow of this study. This study includes four major parts: I. Dataset establishment; II. Model development; III. Automatic evaluation; IV. Clinical evaluation. NLG natural language generation, CE clinical efficacy.CC BY-NC-ND · PubMed Central
Fig. 2 Rating distribution based on the five-point Likert scale. a Rating distribution proportion per score; b Rating distribution per report type.
Fig. 2 Rating distribution based on the five-point Likert scale. a Rating distribution proportion per score; b Rating distribution per report type.CC BY-NC-ND · PubMed Central
Fig. 3 The scores of three types of reports. Data presented as mean ± SD, ns means no significant difference, *p < 0.05, **p < 0.01, and ***p < 0.001.
Fig. 3 The scores of three types of reports. Data presented as mean ± SD, ns means no significant difference, *p < 0.05, **p < 0.01, and ***p < 0.001.CC BY-NC-ND · PubMed Central
Fig. 4 Illustrations of the real-world, only AI, normal template-aided, AI-aided reports from 3 cases. a one BRVO case, b one DR case, and c one AMD case. BRVO branch retinal vein occlusion, DR diabetic retinopathy.
Fig. 4 Illustrations of the real-world, only AI, normal template-aided, AI-aided reports from 3 cases. a one BRVO case, b one DR case, and c one AMD case. BRVO branch retinal vein occlusion, DR diabetic retinopathy.CC BY-NC-ND · PubMed Central

Verification

Claims the paper makes

7 Valid · 1 Not valid · 2 Not checked · 66 Own results and interpretations

33 more claims

Create a free account to see all 76 claims and every source verdict.

See all claims — free account
  • It may state two separate claims as one. 26
  • Probably not a substantial claim. 20
  • The quote only partly states this claim. 2

Create a free account to see these claims.

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

3 of 3

Citations

Cites

Show all 33 references

Research works printed but not linked: 2

  1. Denkowski, M. & Lavie, A. Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. Proceedings of the Sixth Workshop on Statistical Machine Translation 85–91 (Association for Computational Linguistics, 2011).
  2. Lin, C.-Y. ROUGE: a package for automatic evaluation of summaries. In Text Summarization Branches Out 74–81 (Association for Computational Linguistics, 2004).

Cited by

See all 16

Metadata

Authors
Shao, A., Liu, X., Shen, W., Li, Y., Wu, H., Pan, X., Yang, Z., Xu, Y., Zhu, T., Wang, Y., Yang, J., Tham, Y. C., Wu, J., Jin, K., & Ye, J.
Affiliations
  • Second Affiliated Hospital of Zhejiang University (CN)
  • Zhejiang University (CN)
  • China Jiliang University (CN)
  • Brigham and Women's Hospital (US)
  • National University of Singapore (SG)
Corresponding author
  • Wu, Jian
  • Jin, Kai
  • Ye, Juan
Venue
npj Digital Medicine
Published
2025-07-02
Volume
8
Issue
1
Pages
396-396
ISSN
2398-6352 (ISSN-L)
OpenAlex
W4411918684
Cited by
16
References
33
Citations by year
2026: 14 · 2025: 2
FWCI
9.22
Type
journal article
Language
en
Open access
Licence
Locations
  • npj Digital Medicine (publishedVersion, cc-by-nc-nd)
  • PubMed Central (publishedVersion, cc-by-nc-nd)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Funders
  • National Natural Science Foundation of China (82201195)
  • National Natural Science Foundation Regional Innovation and Development Joint Fund (U20A20386)
  • Key Program of the National Natural Science Foundation of China (82330032)
Topics
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
Keywords
fundus fluorescein angiography, generative AI, automated reporting, contrastive learning, chorioretinal diseases, diagnostic accuracy, clinical efficiency

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation. https://app.cochat.ai/papers/10.1038/s41746-025-01759-z
The paper
Shao, A., Liu, X., Shen, W., Li, Y., Wu, H., Pan, X., Yang, Z., Xu, Y., Zhu, T., Wang, Y., Yang, J., Tham, Y. C., Wu, J., Jin, K., & Ye, J. (2025). Generative artificial intelligence for fundus fluorescein angiography interpretation and human expert evaluation. npj Digital Medicine. https://doi.org/10.1038/s41746-025-01759-z

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.