Using artificial intelligence to improve human performance: efficient retinal disease detection training with synthetic images
Open accessAbstract
Background: Artificial intelligence (AI) in medical imaging diagnostics has huge potential, but human judgement is still indispensable. We propose an AI-aided teaching method that leverages generative AI to train students on many images while preserving patient privacy. Methods: A web-based course was designed using 600 synthetic ultra-widefield (UWF) retinal images to teach students to detect disease in these images. The images were generated by stable diffusion, a large generative foundation model, which we fine-tuned with 6285 real UWF images from six categories: five retinal diseases (age-related macular degeneration, glaucoma, diabetic retinopathy, retinal detachment and retinal vein occlusion) and normal. 161 trainee orthoptists took the course. They were evaluated with two tests: one consisting of UWF images and another of standard field (SF) images, which the students had not encountered in the course. Both tests contained 120 real patient images, 20 per category. The students took both tests once before and after training, with a cool-off period in between. Results: On average, students completed the course in 53 min, significantly improving their diagnostic accuracy. For UWF images, student accuracy increased from 43.6% to 74.1% (p<0.0001 by paired t-test), nearly matching the previously published state-of-the-art AI model’s accuracy of 73.3%. For SF images, student accuracy rose from 42.7% to 68.7% (p<0.0001), surpassing the state-of-the-art AI model’s 40%. Conclusion: Synthetic images can be used effectively in medical education. We also found that humans are more robust to novel situations than AI models, thus showcasing human judgement’s essential role in medical diagnosis.
AI snapshot
Written from the paper’s full text.
TL;DR
A web-based course using 600 synthetic ultra-widefield retinal images generated by fine-tuned stable diffusion significantly improved trainee orthoptists’ diagnostic accuracy. After about 53 min of training, students matched or exceeded a state-of-the-art AI model on ultra-widefield images and surpassed it on an unseen standard-field modality, while experienced experts remained more accurate.
Key findings
- For ultra-widefield (UWF) images, 161 trainee orthoptists improved from 43.6% (±18.8%) to 74.1% (±9.3%) accuracy (p<0.0001 by paired t-test), nearly matching the state-of-the-art AI model’s 73.3%; eight experts averaged 91.1% (±4.2%).
- On standard-field (SF) images not encountered in training, student accuracy rose from 42.7% (±18.5%) to 68.7% (±11.5%) (p<0.0001), surpassing the AI model’s 40%; experts averaged 92.8% (±6.8%).
- Students completed the course in 53 min 1 s (±16 min 0 s) on average; fourth-year students studied less than first-year students (32.0 vs 59.4 min, p<0.0001 by Steel-Dwass test) but spent more time per UWF test image (9 s vs 7 s, p=0.0002).
- The synthetic images did not appear to memorise real training images: the most similar real–synthetic pairs were manually inspected and none were visually identical.
Methods
- A web-based training course used 600 synthetic UWF retinal images, 100 per class, generated by stable diffusion V.1.4 fine-tuned with DreamBooth on 6285 real UWF images across six categories: diabetic retinopathy, age-related macular degeneration, glaucoma, retinal vein occlusion, retinal detachment, and normal.
- 161 trainee orthoptists took pre- and post-training tests containing 120 real patient images each (20 per category), once with UWF images and once with SF images; eight experienced experts and a previously published state-of-the-art AI model were also evaluated on the same tests.
- Analyses included paired t-tests for within-subject learning effects, Steel-Dwass tests for group comparisons of study and response times, and self-supervised copy detection to check for memorisation of real images by the generative model.
Limitations
- The paper states there was no direct comparison with current widely used medical education methods, learners were limited to medical personnel, and persistence of diagnostic capability was not examined.
- The paper also notes a low adoption rate of 0.29% for normal synthetic images because many borderline cases had to be excluded, requiring extra ophthalmologist effort.
- Evident from the text: all student participants were trainee orthoptists from a single 4-year university programme in Japan, and the retinal images came from one clinical practice at Tsukazaki Hospital, which may limit generalisability.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
0 of 18 cited sources checked · 0 supported · 0 flagged
- Findingown data
The average similarity scores between original and generated SD images for each of six types were: RD 0.45 (±0.09), RVO 0.45 (±0.07), Gla 0.48 (±0.07), DR 0.45 (±0.09), AMD 0.49 (±0.07), normal 0.47 (±0.08).
“The average similarity scores (ranging from 0 to 1, with 1 being the same image) between the original images and the generated SD images for each of the six types were as follows: RD 0.45 (±0.09), RVO 0.45 (±0.07), Gla 0.48 (±0.07), DR 0.45 (±0.09), AMD 0.49 (±0.07) and normal 0.47 (±0.08).”
Quote in the paperNot a substantial claim - Findingown data
On average, students completed the web-based training course in 53 min 1 s (SD 16 min 0 s).
“The students completed the course in 53 min 1 s (±16 min 0 s)”
- Also stated in: § Results · plain
“On average, students completed the course in 53 min”
Quote in the paperNot a substantial claim - Also stated in: § Results · plain
- Findingown data
Fourth-year students spent the least time studying, almost half of that of first-year students (59.4 min vs 32.0 min, p<0.0001 by Steel-Dwass test).
“fourth-year students spent the least time studying, almost half of that of first-year students (59.4 min vs 32.0 min, p<0.0001 by Steel-Dwass test)”
Quote in the paperNot a substantial claim - Findingown data
There are cases that are hard for humans but correctly scored by AI models and vice versa.
“Preliminary analysis suggests that there are indeed cases that are hard for humans but correctly scored by AI models and vice versa.”
Quote in the paperSubstantial claim - Findingown data
The eight expert clinicians had an average accuracy of 92.8% (±6.8%) on SF images.
“The experts achieved similar performance as before (92.8% (±6.8%))”
Quote in the paperNot a substantial claim- Rests on an assumption: The experts' regular encounters with both UWF and SF imaging explain their stable performance across modalities.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Findingown data
The SD model did not memorise any real images used for fine-tuning.
“None of these pairs were visually identical, suggesting that our SD model did not memorise images.”
- Also stated in: § Discussion · hedged
“our detailed examination using state-of-the-art methods for detecting memorised images could not find such cases”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [16] Pizzi E , Roy SD , Ravindra SN , et al . A self-supervised descriptor for image copy detection. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR); New Orleans, LA, USA. doi:10.1109/CVPR52688.2022.01413 source
Not checked in the free run.
- Also stated in: § Discussion · hedged
- Findingown data
Students' performance improved significantly across all subgroups for SF images.
“with performance also increasing significantly in all subgroups (figure 3)”
- Also stated in: § Results · plain
“Statistically significant learning improvements were observed in all learner groups (**p<0.0001, paired t-test)”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Rests on an assumption: Students' knowledge of retinal diseases deepens with each year of study.
- Also stated in: § Results · plain
- Findingown data
The post-training UWF accuracy of students (74.1%) nearly matched the previously published state-of-the-art AI model's accuracy of 73.3%.
“nearly matching the previously published state-of-the-art AI model’s accuracy of 73.3%”
- Also stated in: § Results · plain
“after learning, they achieved accuracy rates equal to or higher than those of the latest AI model”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] Engelmann, 2022, Detecting multiple retinal diseases in ultra-widefield fundus imaging and data-driven identification of informative regions with deep learning, Nat Mach Intell source
Not checked in the free run.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Also stated in: § Results · plain
- Findingown data
Students' performance improved significantly across all subgroups for UWF images.
“with performance improving significantly across all subgroups (figure 2)”
- Also stated in: § Results · plain
“Statistically significant learning improvements were observed in all learner groups (**p<0.0001, paired t-test)”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Rests on an assumption: Students' knowledge of retinal diseases deepens with each year of study.
- Also stated in: § Results · plain
- Findingown data
The AI model experienced a drop of 33.3 percentage points from UWF to SF images.
“they did not experience the dramatic drop of 33.3 percentage points that the AI model experienced.”
Quote in the paperSubstantial claim- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Findingown data
For UWF images, student diagnostic accuracy increased from 43.6% before training to 74.1% after training (p<0.0001 by paired t-test).
“For UWF images, student accuracy increased from 43.6% to 74.1% (p<0.0001 by paired t-test)”
- Also stated in: § Results · plain
“their average accuracy improved from 43.6% (±18.8%) to 74.1% (±9.3%) (p<0.0001 by paired t-test)”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Also stated in: § Results · plain
- Findingown data
The correlation between the mean accuracy of the students per image and whether the AI model scored the image correctly is relatively weak (Pearson r=0.1537; p=0.0936).
“The correlation between the mean accuracy of the students per image and whether the AI model scored the image correctly is relatively weak (Pearson r=0.1537; p=0.0936)”
Quote in the paperNot a substantial claim - Findingown data
The eight expert clinicians had an average accuracy of 91.1% (±4.2%) on UWF images.
“The eight expert clinicians had an average accuracy of 91.1% (±4.2%) in UWF images.”
Quote in the paperNot a substantial claim- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Findingown data
Students' post-training performance on SF images was lower than on UWF images, except for fourth-year students.
“However, except for fourth-year students, the poststudying performance of the students was lower than for the UWF images.”
Quote in the paperNot a substantial claim- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Findingown data
The AI model's accuracy on SF images dropped to 40%.
“the performance of the AI model, which had only been trained on UWF images, dropped dramatically to just 40%.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] Engelmann, 2022, Detecting multiple retinal diseases in ultra-widefield fundus imaging and data-driven identification of informative regions with deep learning, Nat Mach Intell source
Not checked in the free run.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Findingown data
Experts still performed at least 10% better than students.
“However, experts still performed at least 10% better than students”
Quote in the paperNot a substantial claim- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Interpretationown data
Humans generalise their learning much better than AI models.
“This suggests that humans generalise their learning much better, highlighting the importance of keeping humans in the loop.”
- Also stated in: § Conclusion · plain
“We also found that humans are more robust to novel situations than AI models”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Also stated in: § Conclusion · plain
- Interpretationreasoning
The low adoption rate of 0.29% for standard eye images is due to the need to exclude many borderline cases that are even slightly close to showing signs of disease to classify an image as usual.
“the low adoption rate of 0.29% for standard eye images is due to the need to exclude many borderline cases that are even slightly close to showing signs of disease to classify an image as usual”
Quote in the paperNot a substantial claim - Interpretationown data
Synthetic images can be used effectively in medical education.
“Synthetic images can be used effectively in medical education.”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Rests on an assumption: Performance on the web-based test of real images reflects clinical diagnostic ability in practice.
- Interpretationown data
Our educational effects demonstrated easy acquisition of AI diagnostic capabilities.
“Moreover, our educational effects demonstrated easy acquisition of AI diagnostic capabilities”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Interpretationown data
The time required to improve a non-expert's performance substantially is relatively short.
“The time required to improve a non-expert’s performance substantially is relatively short.”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Interpretationreasoning
Our current approach excels in cost-effectiveness.
“Compared with past teaching methods, our current approach also excels in cost-effectiveness.”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Interpretationreasoning
Our approach of using synthetic images generated by SD avoids ethical and data privacy problems of using actual patient images.
“Our approach of using synthetic images generated by SD avoids these problems and could be used in many future clinical education areas.”
Quote in the paperSubstantial claim - Interpretationreasoning
AI-generated images can be shared more quickly as they do not belong to any particular patient.
“Furthermore, AI-generated images can be shared more quickly as they do not belong to any particular patient”
Quote in the paperSubstantial claim - Interpretationown data
The AI model could complement humans.
“and the AI model could complement humans.”
Quote in the paperSubstantial claim - Premisecitation
The lack of large, curated imaging repositories often hinders the full human potential in medical image diagnosis.
“The lack of large, curated imaging repositories often hinders the full human potential in medical image diagnosis.9”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] Kaundinya, 2021, Diversity of skin images in medical texts: recommendations for student advocacy in medical education, J Med Educ Curric Dev source
Not checked in the free run.
- Interpretationown data
Non-experts could be trained to match or exceed the performance of the AI model in about an hour of training.
“We found that non-experts could be trained to match or exceed the performance of the AI model in about an hour of training.”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- The quote only partly states this claim. 4
- Classified as probably not a substantial claim. 21
- May combine two claims. 11
The quote only partly states this claim.
Findingown dataThe AI model achieved an accuracy of 73.3% on UWF images.
“The AI model achieved an accuracy of 73.3%.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [17] Engelmann, 2022, Detecting multiple retinal diseases in ultra-widefield fundus imaging and data-driven identification of informative regions with deep learning, Nat Mach Intell source
Not checked in the free run.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
The quote only partly states this claim.
Findingown dataFor SF images, the post-training student accuracy (68.7%) surpassed the state-of-the-art AI model's accuracy of 40%.
“surpassing the state-of-the-art AI model’s 40%”
- Also stated in: § Results · plain
“learners achieved better accuracy rates than the AI model after learning”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] Engelmann, 2022, Detecting multiple retinal diseases in ultra-widefield fundus imaging and data-driven identification of informative regions with deep learning, Nat Mach Intell source
Not checked in the free run.
- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
- Also stated in: § Results · plain
The quote only partly states this claim.
Findingown dataStudents' performance was 8.2 percentage points lower on average for SF images than for UWF images.
“Although students’ performance was 8.2 percentage points lower on average for the SF images”
Quote in the paperSubstantial claim- Rests on an assumption: The evaluation test images are representative of real-world retinal disease cases.
Probably not a substantial claim: may not pass S3 (load-bearing).
Findingown dataFourth-year students spent more time per image during the UWF evaluation test than first-year students (9 s vs 7 s, p=0.0002 by Steel-Dwass test).
“yet spent more time per image during the UWF images evaluation test (9 s vs 7 s, p=0.0002 by Steel-Dwass test).”
Quote in the paperNot a substantial claimThe quote only partly states this claim.
Findingown dataFor SF images, student diagnostic accuracy increased from 42.7% before training to 68.7% after training (p<0.0001 by paired t-test).
“For SF images, student accuracy rose from 42.7% to 68.7% (p<0.0001)”
- Also stated in: § Results · plain
“Accuracy improved to 68.7% (±11.5%) (p<0.0001 by paired t-test) after taking the test”
Quote in the paperSubstantial claim- Rests on an assumption: The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
- Also stated in: § Results · plain
Classified as probably not a substantial claim.
Study design and conductnone givenAll images were selected based on diagnoses agreed upon by two or more ophthalmologists.
“All images were selected based on diagnoses agreed upon by two or more ophthalmologists: RS, HT and TY.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenStudents were instructed not to engage in any additional learning activities until the experiment's conclusion.
“Students were instructed by the student leader (FM) not to engage in any additional learning activities until the experiment’s conclusion.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenAt least 1 week passed between taking the test before and after training.
“at least 1 week passed between taking the test before and after training.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe same ophthalmologists assessed the generated images and selected those suitable for teaching based on the clarity of features unique to the target pathology without additional findings suggesting other diseases.
“The same ophthalmologists (RS, HT and TY) assessed the generated images. They selected those suitable for teaching based on the clarity of features unique to the target pathology without additional findings suggesting other diseases.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe study does not examine the persistence of learners' diagnostic capabilities.
“Third, the study does not examine the persistence of learners’ diagnostic capabilities.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim.
Study design and conductnone givenThe images used in the evaluation test are not included in the training data.
“(The images used in the evaluation test are not included.)”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductnone givenThe research was conducted in accordance with the Declaration of Helsinki and approved by the ethics committee of Tsukazaki Hospital.
“Our research was conducted in accordance with the Declaration of Helsinki and approved by the ethics committee of Tsukazaki Hospital.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
Study design and conductown dataWritten consent was obtained from each patient, student, and expert for research use of their data.
“We obtained express written consent from each patient for the research use of their data. Written explanations and consent were received from the students, and prior explanations and consent were obtained from the experts.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe students were not given feedback when taking the test.
“Notably, the students were not given feedback when taking the test”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThere is no direct comparison with the current, widely used medical education methods that rely on textbooks with few images and lengthy explanations.
“First, there is no direct comparison with the current, widely used medical education methods that rely on textbooks with few images and lengthy explanations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
Study design and conductnone givenThe learners in this study are limited to medical personnel.
“Second, the learners in this study are limited to medical personnel.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationThe AI model was trained on a dataset of 5376 patients (8570 eyes, 13 026 images of Optos 200Tx) and achieved an AUC of 0.9848 (±0.0004) for detecting disease in UWF images on the same evaluation tests.
“This model was trained on a dataset of 5376 patients (8570 eyes, 13 026 images of Optos 200Tx) and achieved an area under the curve (AUC) of 0.9848 (±0.0004)17 for detecting disease in UWF images on the same evaluation tests provided to the students.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [17] Engelmann, 2022, Detecting multiple retinal diseases in ultra-widefield fundus imaging and data-driven identification of informative regions with deep learning, Nat Mach Intell source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationThe shortage of medical experts is emerging as a critical challenge that must be addressed to ensure the sustainability of healthcare delivery.
“the shortage of medical experts is emerging as a critical challenge that must be addressed to ensure the sustainability of healthcare delivery.4”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [4] Teo, 2020, Do we have enough ophthalmologists to manage vision-threatening diabetic retinopathy? A global perspective, Eye (Lond) source
Not checked in the free run.
May combine two claims.
Interpretationown dataThe detailed examination using state-of-the-art methods for detecting memorised images could not find cases of memorisation, implying that this approach may preserve privacy.
“However, our detailed examination using state-of-the-art methods for detecting memorised images could not find such cases, implying that this approach may preserve privacy.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S3 (load-bearing).
PremisecitationLarge text-to-image generative foundation models like stable diffusion have been developed to generate many images faster and with higher quality than generative adversarial networks.
“Recently, large text-to-image generative foundation models like stable diffusion (SD)10 have been developed to generate many images faster and with higher quality than generative adversarial networks,11 the previous state-of-the-art.”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [10] Rombach R , Blattmann A , Lorenz D , et al . Stable diffusion. 2022. Available: https://github.com/CompVis/stable-diffusion [Accessed 21 Apr 2023].
Not checked in the free run.
- cites [11] Odena A . Semi-supervised Generative adversarial network. 2016. Available: https://github.com/eriklindernoren/PyTorch-GAN#semi-supervised-gan [Accessed 21 Apr 2023].
Not checked in the free run.
May combine two claims.
Interpretationown dataStudents might sometimes miss cases that the AI model would have identified but also recognise if the model is making a mistake.
“For example, students might sometimes miss cases that the AI model would have identified but also recognise if the model is making a mistake.”
Quote in the paperNot a substantial claimClassified as probably not a substantial claim. May combine two claims.
PremisecitationChallenges such as decreased performance on unseen imaging modalities and the commercially available AI diagnosis, which is difficult for medical insurance systems to cover, need to be addressed.
“However, challenges such as decreased performance on unseen imaging modalities and the commercially available AI diagnosis, which is difficult for medical insurance systems to cover,8 need to be addressed”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [8] Chen, 2021, Economic challenges of artificial intelligence adoption for diabetic retinopathy, Ophthalmology source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationcitationContinuous learning would be necessary to maintain learners' diagnostic capabilities.
“it is easy to imagine from the theory of the forgetting curve26 that continuous learning would be necessary to maintain learners' diagnostic capabilities.”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [26] Murre, 2015, Replication and analysis of ebbinghaus’ forgetting curve, PLOS ONE source
Not checked in the free run.
Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationAI for medical diagnosis holds potential, demonstrating high performance in detecting various diseases.
“Artificial intelligence (AI) for medical diagnosis holds potential, demonstrating high performance in detecting various diseases.57”
Quote in the paperNot a substantial claim0 of 2 sources checkedProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenA web-based, fully automated training course can be easily scaled and made accessible worldwide.
“In addition, a web-based, fully automated training course can be easily scaled and made accessible worldwide.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.
PremisecitationReports indicate enhanced X-ray diagnostic techniques by technicians in medically underserved areas such as South Africa and Australia, as well as specialised diagnostic improvements for CT colonographic data, endometrial tumours and malignant skin tumours.
“Reports indicate enhanced X-ray diagnostic techniques by technicians in medically underserved areas such as South Africa18 and Australia,19 as well as specialised diagnostic improvements for CT colonographic data,20 endometrial tumours21 and malignant skin tumours.22”
Quote in the paperNot a substantial claim0 of 3 sources checked- cites [18] Hazell, 2015, The influence of image interpretation training on the accuracy of abnormality detection and written comments on musculoskeletal radiographs by South African radiographers, J Med Imaging Radiat Sci source
Not checked in the free run.
- cites [19] Smith, 2009, The influence of a continuing education program on the image interpretation accuracy of rural Radiographers, Rural Remote Health
Not checked in the free run.
- cites [20] source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
Study design and conductreasoningThe results of this study do not necessarily negate current education methods.
“In other words, the results of this study do not necessarily negate current education methods.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningThe process of excluding borderline cases necessitated extra effort from ophthalmologists, indicating a point for improvement in this method in the future.
“This process necessitated extra effort from ophthalmologists, indicating a point for improvement in this method in the future.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationMedical students might only see a few examples per disease, making it difficult to recognise all cases in practice where there is a large variety in disease presentation and patient morphology.
“Medical students might only see a few examples per disease, making it difficult to recognise all cases in practice where there is a large variety in disease presentation and patient morphology.”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [9] Kaundinya, 2021, Diversity of skin images in medical texts: recommendations for student advocacy in medical education, J Med Educ Curric Dev source
Not checked in the free run.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
InterpretationreasoningFurther learning could lead to even more significant improvements in diagnostic capabilities.
“On the other hand, further learning could lead to even more significant improvements in diagnostic capabilities”
Quote in the paperNot a substantial claim
The paper relies on this without support.
The observed improvement in students' diagnostic accuracy is due to the training course rather than other factors such as additional learning or test familiarity.
“Students were instructed by the student leader (FM) not to engage in any additional learning activities until the experiment’s conclusion.”
“at least 1 week passed between taking the test before and after training.”
The paper relies on this without support.
Performance on the web-based test of real images reflects clinical diagnostic ability in practice.
“We conduct a test of real images to evaluate the students’ performance.”
“We designed a web-based training course that uses synthetic images to teach students how to recognise disease in UWF retinal images.”
The paper relies on this without support.
Students' knowledge of retinal diseases deepens with each year of study.
“their knowledge in this area is expected to deepen with every year of study.”
The paper relies on this without support.
The experts' regular encounters with both UWF and SF imaging explain their stable performance across modalities.
“which is expected as they encounter both types of imaging regularly in their work.”
The paper relies on this without support.
The evaluation test images are representative of real-world retinal disease cases.
“Both SF and UWF images were selected by the ophthalmologist (HT) based on the clarity of the lesion within their respective imaging ranges, making them suitable for diagnosis.”
The paper relies on this without support.
The six disease categories cover the critical retinal diseases for this training.
“We considered images belonging to one of six classes, namely standard, healthy retinas (Normal) and five critical retinal diseases: diabetic retinopathy (DR), age-related macular degeneration (AMD), glaucoma (Gla), retinal vein occlusion (RVO) and retinal detachment (RD).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
GAI faces technical limitations in generating simulated images for complex diseases, resulting in images that fail to depict atypical manifestations accurately.
“GAI also faces technical limitations in generating simulated images for complex diseases, resulting in images that fail to depict atypical manifestations accurately [57].”
From Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review (2025) · cites this paper as [57]
The Methods (AI-based image generation; page unavailable) report that ophthalmologists selected only generated images with clear, disease-specific features, with low selection rates for some classes. The Discussion (page unavailable) identifies the low selection rate for normal eye images as a limitation. Neither passage says that complex diseases cause generation failures or that atypical manifestations are depicted inaccurately. (LLM, conf=0.76)
Experimental — accuracy not yet measuredIn ophthalmology, AI-generated images of retinal disease improved students' diagnostic accuracy.
“In ophthalmology, AI-generated images of retinal disease improved students’ diagnostic accuracy [85].”
From Artificial intelligence in undergraduate medical education: an updated scoping review (2025) · cites this paper as [85]
Images of retinal disease generated by the stable diffusion model enhance students' learning opportunities in ophthalmic pathology, greatly enhancing the availability of visual teaching resources.
“For instance, images of retinal disease generated by the stable diffusion model enhance students’ learning opportunities in ophthalmic pathology, greatly enhancing the availability of visual teaching resources [57].”
From Applications, Challenges, and Prospects of Generative Artificial Intelligence Empowering Medical Education: Scoping Review (2025) · cites this paper as [57]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Normal-image selection rate is misstated
MinorThe paper says 100 suitable images were selected per class. Selecting 100 of 3500 normal images gives 2.86%, not 0.29%.
Reported 0.29 · recomputed 2.857143
“3500 for normal (0.29%).”
Third-ranked similarity score exceeds the stated highest score
MinorA score of 0.763 cannot rank third if the highest score is 0.747.
“The highest similarity score among all 600 generated images was 0.747 for a normal SD image”
“tied for third place were DR and Gla SD images with a score of 0.763”
Cites
Replication and Analysis of Ebbinghaus’ Forgetting Curve
Murre, J. M. J., & Dros, J. (2015) · PLOS ONE · doi:10.1371/journal.pone.0120644 · Cited by 777 · see citing works
Machine learning: applications of artificial intelligence to imaging and diagnosis
Nichols, J., Chan, H. H., & Baker, M. A. B. (2018) · Biophysical Reviews · doi:10.1007/s12551-018-0449-9 · Cited by 458 · see citing works
Artificial intelligence for diabetic retinopathy screening: a review
Grzybowski, A., Brona, P., Lim, G., Ruamviboonsuk, P., Tan, G. S. W., Abramoff, M., & Ting, D. S. W. (2020) · Eye · doi:10.1038/s41433-019-0566-0 · Cited by 376 · cited 1× · see citing works
“4 7 However, challenges such as decreased performance on unseen imaging modalities and the commercially available AI diagnosis, which is difficult for medical insurance systems to cover, 8 need to be addressed to fully harness AI’s potential and ensure its accessibility and effectiveness for…”
BackgroundArtificial intelligence: opportunities and implications for the health workforce
Hazarika, I. (2020) · International Health · doi:10.1093/inthealth/ihaa007 · Cited by 203 · see citing works
Artificial intelligence in gastrointestinal endoscopy: The future is almost here
Alagappan, M., Brown, J. R. G., Mori, Y., & Berzin, T. M. (2018) · World Journal of Gastrointestinal Endoscopy · doi:10.4253/wjge.v10.i10.239 · Cited by 163 · see citing works
Melarkode, N., Srinivasan, K., Qaisar, S. M., & Plawiak, P. (2023) · Cancers · doi:10.3390/cancers15041183 · Cited by 144 · see citing works
Ultra-widefield retinal imaging: an update on recent advances
PATEL, S. N., Shi, A., Wibbelsman, T. D., & Klufas, M. A. (2020) · Therapeutic Advances in Ophthalmology · doi:10.1177/2515841419899495 · Cited by 111 · see citing works
A Self-Supervised Descriptor for Image Copy Detection
Pizzi, E., Roy, S. D., Ravindra, S. N., Goyal, P., & Douze, M. (2022) · 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr52688.2022.01413 · Cited by 101 · see citing works
Shuaib, A., Arian, H., & Shuaib, A. S. (2020) · International Journal of General Medicine · doi:10.2147/ijgm.s268093 · Cited by 99 · see citing works
Teo, Z. L., Tham, Y.-C., Yu, M., Cheng, C.-Y., Wong, T. Y., & Sabanayagam, C. (2020) · Eye · doi:10.1038/s41433-020-0776-5 · Cited by 62 · see citing works
Ranabhat, C. L., Acharya, S. P., Adhikari, C., & Kim, C.-B. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1041459 · Cited by 57 · see citing works
Sudharsanan, N., & Bloom, D. E. (2018) · Cited by 50 · see citing works
Dachman, A. H., Kelly, K. B., Zintsmaster, M. P., Rana, R., Khankari, S., Novak, J. D., Ali, A. N., Qalbani, A., & Fletcher, J. G. (2008) · Radiology · doi:10.1148/radiol.2491080059 · Cited by 44 · see citing works
Diversity of skin images in medical texts: recommendations for student advocacy in medical education
Kaundinya, T., & Kundu, R. V. (2021) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205211025855 · Cited by 43 · see citing works
Smith, T., Traise, P., & Cook, A. (2009) · Rural and Remote Health · doi:10.22605/rrh1145 · Cited by 39 · see citing works
Hazell, L., Motto, J., & Chipeya, L. (2015) · Journal of medical imaging and radiation sciences · doi:10.1016/j.jmir.2015.03.002 · Cited by 38 · see citing works
Engelmann, J., McTrusty, A. D., MacCormick, I. J. C., Pead, E., Storkey, A., & Bernabeu, M. O. (2022) · Nature Machine Intelligence · doi:10.1038/s42256-022-00566-5 · Cited by 37 · see citing works
Self-citationJamal, M. H., Aziz, A. F. A., Aizuddin, A. N., & Aljunid, S. M. (2022) · Frontiers in Public Health · doi:10.3389/fpubh.2022.918188 · Cited by 26 · see citing works
Economic challenges of artificial intelligence adoption for diabetic retinopathy
Chen, E. M., Chen, D., Chilakamarri, P., Lopez, R., & Parikh, R. (2021) · Ophthalmology · doi:10.1016/j.ophtha.2020.07.043 · Cited by 21 · see citing works
Cornell, E., Robertson, K., McIntosh, R. D., & Rees, J. (2015) · Acta Dermato Venereologica · doi:10.2340/00015555-2058 · Cited by 4 · see citing works
Wang, W., Chen, F., Kong, L., Guo, Y., Cheng, J., & Zhang, Y. (2020) · Gynecologic and Obstetric Investigation · doi:10.1159/000507929 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- Oren, 2018, The role of artificial intelligence in diagnostic radiology: a survey among israeli radiologists, J Digit Imaging, 31, 780
Cited by
Sorted by citations
Artificial intelligence in undergraduate medical education: an updated scoping review
Simoni, J., Urtubia-Fernandez, J., Mengual, E., Simoni, D. A., Royo, M., Egaña-Yin, D., Hertog, O. L. A., López-Ortiz, L., Muñoz-Tomás, A., Santiago-Martínez, P., et al. (2025) · BMC Medical Education · doi:10.1186/s12909-025-08188-2 · Cited by 62 · see citing works
“In ophthalmology, AI-generated images of retinal disease improved students’ diagnostic accuracy [85].”
The AI revolution in glaucoma: Bridging challenges with opportunities
Li, F., Wang, D., Yang, Z., Zhang, Y., Jiang, J., Liu, X., Kong, K., Zhou, F., Tham, C. C. Y., Medeiros, F. A., et al. (2024) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2024.101291 · Cited by 40 · see citing works
Lin, Y., Luo, Z., Ye, Z., Zhong, N., Zhao, L., Zhang, L., Li, X., Chen, Z., & Chen, Y. (2025) · JMIR Medical Education · doi:10.2196/71125 · Cited by 31 · see citing works
“…networks (GANs) can create clinical images displaying various pathological features based on textual descriptions, potentially addressing the shortage of authentic pathological images in traditional medical education due to medical confidentiality and patient privacy restrictions [51,57-59].”
Artificial intelligence in clinical education in ophthalmology: a systematic review
Yuan, L., Kang, D., Dong, X., Liu, L., Grzybowski, A. E., & Jin, K. (2025) · Visual Neuroscience · doi:10.48130/vns-0025-0025 · Cited by 17 · see citing works
AI image generation technology in ophthalmology: Use, misuse and future applications
Phipps, B., Hadoux, X., Sheng, B., Campbell, J. P., Liu, T. Y. A., Keane, P. A., Cheung, C. Y.-L., Chung, T. Y., Wong, T. Y., & van Wijngaarden, P. (2025) · Progress in Retinal and Eye Research · doi:10.1016/j.preteyeres.2025.101353 · Cited by 16 · see citing works
Tabuchi, H., Nakajima, I., Day, M., Yoneda, T., Tanabe, M., Strang, N. C., Engelmann, J., Deguchi, H., Akada, M., Moriguchi, T., et al. (2024) · Scientific Reports · doi:10.1038/s41598-024-80732-4 · Cited by 12 · see citing works
Self-citationGeneration of Fundus Fluorescein Angiography Videos for Health Care Data Sharing
Wu, X., Wang, L., Chen, R., Liu, B., Zhang, W., Yang, X., Feng, Y., He, M., & Shi, D. (2025) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2025.1419 · Cited by 11 · see citing works
Discriminative, generative artificial intelligence, and foundation models in retina imaging
Ruamviboonsuk, P., Arjkongharn, N., Vongsa, N., Pakaymaskul, P., & Kaothanthong, N. (2024) · Taiwan Journal of Ophthalmology · doi:10.4103/tjo.tjo-d-24-00064 · Cited by 9 · see citing works
Enhancing medical students’ diagnostic accuracy of infectious keratitis with AI-generated images
Xie, W., Yuan, Z., Si, Y., Huang, Z., Li, Y.-M., Wu, F. Q., & Yao, Y. (2025) · BMC Medical Education · doi:10.1186/s12909-025-07592-y · Cited by 6 · see citing works
Qureshi, A. R., Micieli, J. A., & Wong, J. C. Y. (2025) · Eye · doi:10.1038/s41433-025-04006-7 · Cited by 6 · see citing works
The application of artificial intelligence-generated content in ophthalmology education
Wang, Y., Zhao, Y., & Li, J. (2025) · Frontiers in Medicine · doi:10.3389/fmed.2025.1617537 · Cited by 4 · see citing works
Gupta, K., Latinovich, M. F., Latinovich, M. F., Singh, K., Patlas, M. N., & Jajodia, A. (2026) · Journal of Medical Systems · doi:10.1007/s10916-026-02350-z · Cited by 4 · see citing works
Embracing generative AI in ophthalmology
Larkin, F., & He, M. (2024) · British Journal of Ophthalmology · doi:10.1136/bjo-2024-326320 · Cited by 3 · see citing works
Impact of generative artificial intelligence on workload, efficiency and labour productivity
Caamaño-Gordillo, D., Mula, J., & de la Torre, R. (2025) · IFAC-PapersOnLine · doi:10.1016/j.ifacol.2025.09.237 · Cited by 3 · see citing works
Liao, M., & Yang, C. (2026) · Organ medicine. · doi:10.1002/orm2.70039 · Cited by 1 · see citing works
Large-scale screening of blinding eye diseases from routine blood tests
Li, S., Ren, J., Wu, J., Li, Y., Wang, X., Zhang, M., Hu, H., Song, Y., Gao, S., Cao, W., et al. (2026) · Nature Health · doi:10.1038/s44360-026-00101-5
Cabitza, F., & Vicente, L. (2026) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2026.103855
Entering the era of photorealistic AI-generated medical images
Xie, W. (2026) · Postgraduate Medical Journal · doi:10.1093/postmj/qgag085
A Review of Artificial Intelligence in Ophthalmology: Key Aspects, Challenges, and Future Directions
Ray, P. (2026) · Eye & ENT Research · doi:10.1002/eer3.70049
Uschmann, S., Spreckelsen, C., & Festag, S. (2026) · PLOS Digital Health · doi:10.1371/journal.pdig.0001487
Benito, M. D., Diana-Albelda, C., García‐Martín, Á., Campos, M. R. P., & Bescós, J. (2026) · Journal of Imaging · doi:10.3390/jimaging12090438
Lauron, L. K. A. Y., Rayco, R. C. M., Laping, I. P., Rejollo, M. R. A., & Tagalog, R. J. (2026) · doi:10.68050/jams.2026.516
- Authors
- Tabuchi, H., Engelmann, J., Maeda, F., Nishikawa, R., Nagasawa, T., Yamauchi, T., Tanabe, M., Akada, M., Kihara, K., Nakae, Y., Kiuchi, Y., & Bernabeu, M. O.
- Affiliations
- Hiroshima University (JP)
- University of Edinburgh (GB)
- Niigata University of Health and Welfare (JP)
- Tsukazaki Hospital (JP)
- Corresponding author
- Tabuchi, Hitoshi
- Venue
- British Journal of Ophthalmology
- Publisher
- BMJ
- Published
- 2024-03-14
- Volume
- 108
- Issue
- 10
- Pages
- 1430-1435
- ISSN
- 0007-1161 (ISSN-L)
- 1468-2079
- PMC
- PMC11503156
- OpenAlex
- W4392797668
- Semantic Scholar
- a1b76a51085a5970a876466744b0f5cadd3b2f48
- Cited by
- 22
- References
- 21
- Citations by year
- 2026: 9 · 2025: 9 · 2024: 4
- FWCI
- 2.06
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- British Journal of Ophthalmology (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Keywords
- synthetic image generation, Stable Diffusion, retinal disease detection, orthoptist training, ultra-widefield retinal imaging, patient privacy, age-related macular degeneration, glaucoma, diabetic retinopathy, retinal detachment, retinal vein occlusion
- MeSH terms
- Artificial Intelligence, Clinical Competence, Diagnostic Techniques, Ophthalmological, Humans, Ophthalmology, Retinal Diseases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review