Rotation-Oriented Collaborative Self-Supervised Learning for Retinal Disease Diagnosis
Abstract
The automatic diagnosis of various conventional ophthalmic diseases from fundus images is important in clinical practice. However, developing such automatic solutions is challenging due to the requirement of a large amount of training data and the expensive annotations for medical images. This paper presents a novel self-supervised learning framework for retinal disease diagnosis to reduce the annotation efforts by learning the visual features from the unlabeled images. To achieve this, we present a rotation-oriented collaborative method that explores rotation-related and rotation-invariant features, which capture discriminative structures from fundus images and also explore the invariant property used for retinal disease classification. We evaluate the proposed method on two public benchmark datasets for retinal disease classification. The experimental results demonstrate that our method outperforms other self-supervised feature learning methods (around 4.2% area under the curve (AUC)). With a large amount of unlabeled data available, our method can surpass the supervised baseline for pathologic myopia (PM) and is very close to the supervised baseline for age-related macular degeneration (AMD), showing the potential benefit of our method in clinical practice.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The innate relationship tasks in combined studies range from augmentation prediction and patch positioning prediction, rotation prediction, and ultrasound video to speech correspondence prediction.
“The innate relationship tasks range from augmentation prediction and patch positioning prediction69, rotation prediction70, and ultrasound video to speech correspondence prediction53.”
From Self-supervised learning for medical image classification: a systematic review and implementation guidelines (2023) · cites this paper as [70]
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
PyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
Xception: Deep Learning with Depthwise Separable Convolutions
Chollet, F. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.195 · Cited by 20,380 · see citing works
Momentum Contrast for Unsupervised Visual Representation Learning
He, K., Fan, H., Wu, Y., Xie, S., & Girshick, R. (2020) · 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr42600.2020.00975 · Cited by 13,347 · see citing works
A simple framework for contrastive learning of visual representations
Chen, T., Kornblith, S., Norouzi, M., & Hinton, G. (2020) · arXiv · doi:10.48550/arxiv.2002.05709 · Cited by 10,857 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
Bootstrap Your Own Latent: A New Approach to Self-Supervised Learning
Grill, J.-B., Strub, F., Altché, F., Tallec, C., Richemond, P. H., Buchatskaya, E., Doersch, C., Pires, B. A., Guo, Z. D., Azar, M. G., et al. (2020) · arXiv · doi:10.48550/arxiv.2006.07733 · Cited by 3,846 · see citing works
Unsupervised Feature Learning via Non-parametric Instance Discrimination
Wu, Z., Xiong, Y., Yu, S. X., & Lin, D. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00393 · Cited by 3,664 · see citing works
Striving for Simplicity: The All Convolutional Net
Springenberg, J. T., Dosovitskiy, A., Brox, T., & Riedmiller, M. (2014) · arXiv · doi:10.48550/arxiv.1412.6806 · Cited by 3,447 · see citing works
Zhang, R., Isola, P., & Efros, A. A. (2016) · Lecture notes in computer science · doi:10.1007/978-3-319-46487-9_40 · Cited by 3,361 · see citing works
The Regression Analysis of Binary Sequences
Cox, D. R. (1958) · Journal of the Royal Statistical Society Series B (Statistical Methodology) · doi:10.1111/j.2517-6161.1958.tb00292.x · Cited by 2,909 · see citing works
Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzles
Noroozi, M., & Favaro, P. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46466-4_5 · Cited by 2,775 · see citing works
Unsupervised Visual Representation Learning by Context Prediction
Doersch, C., Gupta, A., & Efros, A. A. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.167 · Cited by 2,629 · see citing works
Improved Baselines with Momentum Contrastive LearningPreprint
Chen, X., Fan, H., Girshick, R., & He, K. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2003.04297 · Cited by 2,579 · see citing works
Unsupervised Representation Learning by Predicting Image Rotations
Gidaris, S., Singh, P., & Komodakis, N. (2018) · arXiv · doi:10.48550/arxiv.1803.07728 · Cited by 2,477 · see citing works
Morgan, I. G., Ohno‐Matsui, K., & Saw, S. (2012) · The Lancet · doi:10.1016/s0140-6736(12)60272-4 · Cited by 1,879 · see citing works
Cheng, G., Zhou, P., & Han, J. (2016) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2016.2601622 · Cited by 1,771 · see citing works
Klein, R., Klein, R., Sponsel, W. E., Franke, T. M., Cantor, L. B., Martone, J. F., & Ménage, M. J. (1992) · Ophthalmology · doi:10.1016/s0161-6420(92)31774-9 · Cited by 977 · see citing works
Discriminative Unsupervised Feature Learning with Convolutional Neural NetworksPreprint
Dosovitskiy, A., Springenberg, J. T., Riedmiller, M., & Brox, T. (2014) · arXiv (Cornell University) · Cited by 790 · see citing works
Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks
Dosovitskiy, A., Fischer, P., Springenberg, J. T., Riedmiller, M., & Brox, T. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2496141 · Cited by 762 · see citing works
Burlina, P., Joshi, N., Pekala, M. J., Pacheco, K. D., Freund, D., & Bressler, N. M. (2017) · JAMA Ophthalmology · doi:10.1001/jamaophthalmol.2017.3782 · Cited by 685 · see citing works
Multi-task Self-Supervised Visual Learning
Doersch, C., & Zisserman, A. (2017) · IEEE International Conference on Computer Vision · doi:10.1109/iccv.2017.226 · Cited by 682 · see citing works
Risk factors associated with age-related macular degeneration
(2000) · Ophthalmology · doi:10.1016/s0161-6420(00)00409-7 · Cited by 633 · see citing works
Unsupervised Embedding Learning via Invariant and Spreading Instance Feature
Ye, M., Zhang, X., Yuen, P. C., & Chang, S. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00637 · Cited by 629 · see citing works
Superpixel Classification Based Optic Disc and Optic Cup Segmentation for Glaucoma Screening
Cheng, J. T., Liu, J., Xu, Y., Yin, F., Wong, D. W. K., Tan, N.-M., Tao, D., Cheng, C., Aung, T., & Wong, T. Y. (2013) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2013.2247770 · Cited by 594 · see citing works
Self-supervised learning for medical image analysis using image context restoration
Chen, L., Bentley, P., Mori, K., Misawa, K., Fujiwara, M., & Rueckert, D. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.101539 · Cited by 575 · see citing works
Graßmann, F., Mengelkamp, J., Brandl, C., Harsch, S., Zimmermann, M. E., Linkohr, B., Peters, A., Heid, I. M., Palm, C., & Weber, B. H. F. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.02.037 · Cited by 550 · see citing works
Li, X., Hu, X., Yu, L., Zhu, L., Fu, C., & Heng, P. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2019.2951844 · Cited by 451 · see citing works
Self-citationGlaucoma detection based on deep convolutional neural network
Chen, X., Xu, Y., Wong, D. W. K., Wong, T. Y., & Liu, J. (2015) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2015.7318462 · Cited by 429 · see citing works
Disc-Aware Ensemble Network for Glaucoma Screening From Fundus Image
Fu, H., Cheng, J., Xu, Y., Zhang, C., Wong, D. W. K., Liu, J., & Cao, X. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2837012 · Cited by 429 · see citing works
Inception-v3 for flower classification
Xia, X., Cui, X., & Nan, B. (2017) · International Conference on Image, Vision and Computing (ICIVC) · doi:10.1109/icivc.2017.7984661 · Cited by 409 · see citing works
Cheng, G., Han, J., Zhou, P., & Xu, D. (2019) · IEEE Transactions on Image Processing · doi:10.1109/tip.2018.2867198 · Cited by 408 · see citing works
Peng, Y., Dharssi, S., Chen, Q., Keenan, T. D. L., Agrón, E., Wong, W. T., Chew, E. Y., & Lu, Z. (2018) · Ophthalmology · doi:10.1016/j.ophtha.2018.11.015 · Cited by 358 · see citing works
Artificial Intelligence to Detect Papilledema from Ocular Fundus Photographs
Miléa, D., Najjar, R. P., Jiang, Z., Ting, D. S. W., Vasseneix, C., Xu, X., Fard, M. A., Fonseca, P., Vanikieti, K., Lagrèze, W. A., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa1917130 · Cited by 348 · see citing works
Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysis
Zhou, Z., Sodha, V., Siddiquee, M. M. R., Feng, R., Tajbakhsh, N., Gotway, M. B., & Liang, J. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32251-9_42 · Cited by 327 · see citing works
Zoom-in-Net: Deep Mining Lesions for Diabetic Retinopathy Detection
Wang, Z., Yin, Y., Shi, J., Fang, W., Li, H., & Wang, X. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_31 · Cited by 318 · see citing works
Collaborative Learning of Semi-Supervised Segmentation and Classification for Medical Images
Zhou, Y., He, X., Huang, L., Liu, L., Zhu, F., Cui, S., & Shao, L. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00218 · Cited by 308 · see citing works
Self-Supervised Representation Learning by Rotation Feature Decoupling
Feng, Z., Xu, C., & Tao, D. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.01061 · Cited by 225 · see citing works
Application of deep learning for retinal image analysis: A review
Badar, M., Haris, M., & Fatima, A. (2019) · Computer Science Review · doi:10.1016/j.cosrev.2019.100203 · Cited by 207 · see citing works
Ferrara, N. (2010) · Nature Medicine · doi:10.1038/nm1010-1107 · Cited by 204 · see citing works
Self-Supervised Learning for Cardiac MR Image Segmentation by Anatomical Position Prediction
Bai, W., Chen, C., Tarroni, G., Duan, J., Guitton, F., Petersen, S. E., Guo, Y., Matthews, P. M., & Rueckert, D. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32245-8_60 · Cited by 189 · see citing works
Cheng, G., Zhou, P., & Han, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.315 · Cited by 175 · see citing works
(2018) · doi:10.1117/12.2293584 · Cited by 171 · see citing works
Rubik’s Cube+: A self-supervised feature learning framework for 3D medical image analysis
Zhu, J., Li, Y., Hu, Y., Ma, K., Zhou, S. K., & Zheng, Y. (2020) · Medical Image Analysis · doi:10.1016/j.media.2020.101746 · Cited by 162 · see citing works
Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik’s Cube
Zhuang, X., Li, Y., Hu, Y., Ma, K., Yang, Y., & Zheng, Y. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32251-9_46 · Cited by 153 · see citing works
Self-Supervised Feature Learning via Exploiting Multi-Modal Data for Retinal Disease Diagnosis
Li, X., Jia, M. J., Islam, M. T., Yu, L., & Xing, L. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.3008871 · Cited by 150 · see citing works
Self-citation3D Self-Supervised Methods for Medical ImagingPreprint
Taleb, A., Loetzsch, W., Danz, N., Severin, J., Gaertner, T., Bergner, B., & Lippert, C. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2006.03829 · Cited by 139 · see citing works
Detection of age-related macular degeneration via deep learning
Burlina, P., Freund, D., Joshi, N., Wolfson, Y., & Bressler, N. M. (2016) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493240 · Cited by 139 · see citing works
Fundus image classification methods for the detection of glaucoma: A review
Saba, T., Bokhari, S. T. F., Sharif, M. I., Yasmin, M., & Raza, M. (2018) · Microscopy Research and Technique · doi:10.1002/jemt.23094 · Cited by 107 · see citing works
Tajbakhsh, N., Hu, Y., Cao, J., Yan, X., Xiao, Y., Lü, Y., Liang, J., Terzopoulos, D., & Ding, X. (2019) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2019.8759553 · Cited by 80 · see citing works
Improving Cytoarchitectonic Segmentation of Human Brain Areas with Self-supervised Siamese Networks
Spitzer, H., Kiwitz, K., Amunts, K., Stefan Harmeling, B., & Dickscheid, T. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00931-1_76 · Cited by 77 · see citing works
Self-supervised Learning for Spinal MRIs
Jamaludin, A., Kadir, T., & Zisserman, A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67558-9_34 · Cited by 70 · see citing works
PM-Net: Pyramid Multi-label Network for Joint Optic Disc and Cup Segmentation
Yin, P., Wu, Q., Xu, Y., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32239-7_15 · Cited by 36 · see citing works
Age-related macular degeneration
de Jong, E. K., Geerlings, M. J., & den Hollander, A. I. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-816222-4.00010-1 · Cited by 32 · see citing works
How to Learn from Unlabeled Volume Data: Self-supervised 3D Context Feature Learning
Blendowski, M., Nickisch, H., & Heinrich, M. P. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32226-7_72 · Cited by 29 · see citing works
Fundus Image Classification for Diabetic Retinopathy Using Disease Severity Grading
Sakaguchi, A., Wu, R., & Kamata, S. (2019) · International Conference on Biomedical Engineering and Technology (ICBET) · doi:10.1145/3326172.3326198 · Cited by 27 · see citing works
Lesion-Aware Segmentation Network for Atrophy and Detachment of Pathological Myopia on Fundus Images
Guo, Y., Wang, R., Zhou, X., et al. (2020) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi45749.2020.9098669 · Cited by 24 · see citing works
Automatic multiresolution age-related macular degeneration detection from fundus images
Garnier, M., Hurtut, T., Tahar, H. B., et al. (2014) · Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE · doi:10.1117/12.2043099 · Cited by 23 · see citing works
Drusen detection from colored fundus images for diagnosis of age related Macular degeneration
Waseem, S., Akram, M. U., & Ahmed, B. A. (2014) · International Conference on Information and Automation for Sustainability (ICIAfS) · doi:10.1109/iciafs.2014.7069581 · Cited by 11 · see citing works
PNN-based classification of retinal diseases using fundus images
Virmani, J., Singh, G. P., Singh, Y., et al. (2019) · Elsevier eBooks · doi:10.1016/b978-0-12-819361-7.00011-7 · Cited by 11 · see citing works
Automatic lesion segmentation and Pathological Myopia classification in fundus imagesPreprint
Freire, C. R., Moura, J. C. C., Barros, D. M. S., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2002.06382 · Cited by 11 · see citing works
Structure-preserving guided retinal image filtering for optic disc analysis
Cheng, J. T., Li, Z. G., Gu, Z., et al. (2019) · Elsevier eBooks · doi:10.1016/b978-0-08-102816-2.00011-3 · Cited by 6 · see citing works
Pathological Myopic Image Analysis with Transfer LearningPreprint
Xie, R., Liu, L., Liu, J., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.00410 · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479 · see citing works
“Out of 412 studies identi fi ed from PubMed, Scopus, and ArXiv, 79 studies were included in the systematic review. prediction 70 , and ultrasound video to speech correspondence prediction 53 .”
BackgroundYan, R., Qu, L., Wei, Q., Huang, S.-C., Shen, L., Rubin, D. L., Xing, L., & Zhou, Y. (2023) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2022.3233574 · Cited by 183 · see citing works
Self-citationArtificial intelligence promotes the diagnosis and screening of diabetic retinopathy
Huang, X., Wang, H., She, C., Feng, J., Liu, X., Hu, X., Chen, L., & Tao, Y. (2022) · Frontiers in Endocrinology · doi:10.3389/fendo.2022.946915 · Cited by 115 · see citing works
Self-citationLi, Z., Li, H., Ralescu, A. L., Dillman, J. R., Altaye, M., Cecil, K. M., Parikh, N. A., & He, L. (2024) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2024.102993 · Cited by 10 · see citing works
- Authors
- Li, X., Hu, X., Qi, X., Yu, L., Zhao, W., Heng, P.-A., & Xing, L.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- Chinese University of Hong Kong (HK)
- University of Hong Kong (HK)
- Stanford University (US)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2021-09
- Volume
- 40
- Issue
- 9
- Pages
- 2284-2294
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W3153351689
- Cited by
- 105
- References
- 65
- Citations by year
- 2026: 9 · 2025: 23 · 2024: 30 · 2023: 25 · 2022: 17 · 2021: 1
- FWCI
- 9.63
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- The Hong Kong University of Science and Technology
- Topics
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Retinal Imaging and Analysis
- Health Sciences › Medicine › Ophthalmology › Ophthalmology and Visual Health Research
- Health Sciences › Medicine › Ophthalmology › Retinal Diseases and Treatments
- Keywords
- retinal disease diagnosis, fundus images, self-supervised learning, rotation-invariant features, unlabeled data, annotation cost reduction, pathologic myopia, age-related macular degeneration, retinal disease classification
- MeSH terms
- Supervised Machine Learning, Fundus Oculi, Humans, Macular Degeneration, Retina, Retinal Diseases
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
