Optimizing skin disease diagnosis: harnessing online community data with contrastive learning and clustering techniques
Open accessAbstract
Skin diseases pose significant challenges in China. Internet health forums offer a platform for millions of users to discuss skin diseases and share images for early intervention, leaving large amount of valuable dermatology images. However, data quality and annotation challenges limit the potential of these resources for developing diagnostic models. In this study, we proposed a deep-learning model that utilized unannotated dermatology images from diverse online sources. We adopted a contrastive learning approach to learn general representations from unlabeled images and fine-tuned the model on coarsely annotated images from Internet forums. Our model classified 22 common skin diseases. To improve annotation quality, we used a clustering method with a small set of standardized validation images. We tested the model on images collected by 33 experienced dermatologists from 15 tertiary hospitals and achieved a 45.05% top-1 accuracy, outperforming the published baseline model by 3%. Accuracy increased with additional validation images, reaching 49.64% with 50 images per category. Our model also demonstrated transferability to new tasks, such as detecting monkeypox, with a 61.76% top-1 accuracy using only 50 additional images in the training process. We also tested our model on benchmark datasets to show the generalization ability. Our findings highlight the potential of unannotated images from online forums for future dermatology applications and demonstrate the effectiveness of our model for early diagnosis and potential outbreak mitigation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
PanDerm achieved superior performance to SwAVDerm using 33% less training data.
“Notably, it achieved superior performance to SwAVDerm35, a leading dermatology self-supervised learning model, using 33% less training data.”
From A Multimodal Vision Foundation Model for Clinical Dermatology (2025) · cites this paper as [35]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380 · see citing works
CutMix: Regularization Strategy to Train Strong Classifiers With Localizable Features
Yun, S., Han, D., Chun, S., Oh, S. J., Yoo, Y., & Choe, J. (2019) · 2019 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2019.00612 · Cited by 5,400 · see citing works
Training language models to follow instructions with human feedback
Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C. L., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. (2022) · arXiv · doi:10.48550/arxiv.2203.02155 · Cited by 4,208 · see citing works
Deep Convolutional Neural Networks for Image Classification: A Comprehensive Review
Rawat, W., & Wang, Z. (2017) · Neural Computation · doi:10.1162/neco_a_00990 · Cited by 3,503 · see citing works
Unsupervised Learning of Visual Features by Contrasting Cluster Assignments
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., & Joulin, A. (2020) · arXiv · doi:10.48550/arxiv.2006.09882 · Cited by 2,499 · see citing works
A deep learning system for differential diagnosis of skin diseases
Liu, Y., Jain, A., Eng, C., Way, D. H., Lee, K., Bui, P., Kanada, K., de Oliveira Marinho, G., Gallegos, J., Gabriele, S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0842-3 · Cited by 853 · see citing works
Big Self-Supervised Models Advance Medical Image Classification
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., Chen, T., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00346 · Cited by 685 · see citing works
International Classification of Diseases 10th Revision (ICD-10)
DiSantostefano, J. (2009) · The Journal for Nurse Practitioners · doi:10.1016/j.nurpra.2008.09.020 · Cited by 659 · see citing works
AggNet: Deep Learning From Crowds for Mitosis Detection in Breast Cancer Histology Images
Albarqouni, S., Baur, C., Achilles, F., Belagiannis, V., Demirci, S., & Navab, N. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2528120 · Cited by 606 · see citing works
Skin cancer detection: A review using deep learning techniques
Dildar, M., Akram, S., Irfan, M., Khan, H. U., Ramzan, M., Mahmood, A. R., Alsaiari, S. A., Saeed, A. H. M., Alraddadi, M. O., & Mahnashi, M. H. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18105479 · Cited by 597 · see citing works
Disparities in dermatology AI performance on a diverse, curated clinical image set
Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., et al. (2022) · Science Advances · doi:10.1126/sciadv.abq6147 · Cited by 544 · see citing works
Huang, S.-C., Pareek, A., Jensen, M., Lungren, M. P., Yeung, S., & Chaudhari, A. S. (2023) · npj Digital Medicine · doi:10.1038/s41746-023-00811-0 · Cited by 479 · cited 1× · see citing works
“There are prior works exploring contrastive learning in dermatological diagnosis, emphasizing its capability to extract consistent representations and enhance generalizability and diagnostic accuracy 16,17 .”
BackgroundMachine Learning in Dermatology: Current Applications, Opportunities, and Limitations
Chan, S. Y., Reddy, V. D., Myers, B. A., Thibodeaux, Q. G., Brownstone, N. D., & Liao, W. J. (2020) · Dermatology and Therapy · doi:10.1007/s13555-020-00372-0 · Cited by 275 · see citing works
Groh, M., Harris, C., Soenksen, L. R., Lau, F., Han, R., Kim, A., Koochek, A., & Badri, O. (2021) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw53098.2021.00201 · Cited by 247 · see citing works
A Survey on Semi-, Self- and Unsupervised Learning for Image Classification
Schmarje, L., Santarossa, M., Schröder, S., & Koch, R. (2021) · IEEE Access · doi:10.1109/access.2021.3084358 · Cited by 232 · see citing works
A machine learning model for skin disease classification using convolution neural network
Allugunti, V. R. (2022) · International Journal of Computing Programming and Database Management · doi:10.33545/27076636.2022.v3.i1b.53 · Cited by 231 · see citing works
Studies on Different CNN Algorithms for Face Skin Disease Classification Based on Clinical Images
Wu, Z., Zhao, S., Peng, Y., He, X., Zhao, X., Huang, K., Wu, X., Fan, W., Li, F., Chen, M., et al. (2019) · IEEE Access · doi:10.1109/access.2019.2918221 · Cited by 167 · see citing works
Jain, A., Way, D., Gupta, V., Gao, Y., de Oliveira Marinho, G., Hartford, J., Sayres, R., Kanada, K., Eng, C., Nagpal, K., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.7249 · Cited by 164 · see citing works
Checklist for Evaluation of Image-Based Artificial Intelligence Reports in Dermatology
Daneshjou, R., Barata, C., Betz‐Stablein, B., Celebi, M. E., Codella, N., Combalia, M., Guitera, P., Gutman, D. A., Halpern, A. C., Helba, B., et al. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.4915 · Cited by 163 · see citing works
Transparent medical image AI via an image–text foundation model grounded in medical literature
Kim, C., Gadgil, S. U., DeGrave, A. J., Omiye, J. A., Cai, Z. R., Daneshjou, R., & Lee, S.-I. (2024) · Nature Medicine · doi:10.1038/s41591-024-02887-x · Cited by 125 · see citing works
Burden of melanoma in China, 1990–2017: Findings from the 2017 global burden of disease study
Wu, Y., Wang, Y., Wang, L., Yin, P., Lin, Y., & Zhou, M. (2019) · International Journal of Cancer · doi:10.1002/ijc.32764 · Cited by 96 · see citing works
Artificial Intelligence and Mechanistic Modeling for Clinical Decision Making in Oncology
Benzekry, S. (2020) · Clinical Pharmacology & Therapeutics · doi:10.1002/cpt.1951 · Cited by 89 · see citing works
Guha Roy, A., Ren, J., Azizi, S., Loh, A., Natarajan, V., Mustafa, B., Pawlowski, N., Freyberg, J., Liu, Y., Beaver, Z., et al. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102274 · Cited by 77 · see citing works
Artificial intelligence in dermatology
Li, C., Shen, C., Xue, K., Shen, X., Jing, Y., Wang, Z.-Y., Xu, F., Meng, R., Yu, J., & Cui, Y. (2019) · Chinese Medical Journal · doi:10.1097/cm9.0000000000000372 · Cited by 75 · see citing works
Early Prevention and Detection of Skin Cancer Risk using Data Mining
Ahmed, K., Jesmin, T., & Rahman, M. Z. (2013) · International Journal of Computer Applications · doi:10.5120/10065-4662 · Cited by 59 · see citing works
FairDisCo: Fairer AI in Dermatology via Disentanglement Contrastive Learning
Du, S., Hers, B., Bayasi, N., Hamarneh, G., & Garbi, R. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-25069-9_13 · Cited by 43 · see citing works
Hierarchy-aware contrastive learning with late fusion for skin lesion classification
Hsu, B. W., & Tseng, V. S. M. (2022) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2022.106666 · Cited by 40 · see citing works
Resource Allocation Equity in the China’s Rural Three-Tier Healthcare System
Ao, Y., Feng, Q., Zhou, Z., et al. (2022) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph19116589 · Cited by 40 · see citing works
Aractingi, S., & Pellacani, G. (2019) · European Journal of Dermatology · doi:10.1684/ejd.2019.3538 · Cited by 36 · see citing works
Topiwala, A., Al-Zogbi, L., Fleiter, T. R., & Krieger, A. (2019) · IEEE International Conference on Bioinformatics and Bioengineering (BIBE) · doi:10.1109/bibe.2019.00141 · Cited by 24 · see citing works
On a Minimum Distance Procedure for Threshold Selection in Tail Analysis
Drees, H., Janßen, A., Resnick, S. I., et al. (2020) · SIAM Journal on Mathematics of Data Science · doi:10.1137/19m1260463 · Cited by 23 · see citing works
Wu, Y., Zeng, D., Wang, Z., et al. (2021) · doi:10.1109/iccad51958.2021.9643454 · Cited by 21 · see citing works
Diagnosing Common Skin Diseases using Soft Computing Techniques
Parikh, K. S., Shah, T. P., Kota, R., et al. (2015) · International Journal of Bio-Science and Bio-Technology · doi:10.14257/ijbsbt.2015.7.6.28 · Cited by 19 · see citing works
Aggarwal, P. (2021) · JMIR Dermatology · doi:10.2196/31697 · Cited by 14 · see citing works
Consistent representation via contrastive learning for skin lesion diagnosis
Wang, Z., Zhang, L., Shu, X., Wang, Y., & Feng, Y. (2023) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2023.107826 · Cited by 8 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [7]Dildar M Skin cancer detection: a review using deep learning techniques[J] Int. J. Environ. Res. Public Health 2021 18 5479 10.3390/ijerph18105479 34065430 PMC8160886 · DOI · Paper page
- [13]Chen, T., Kornblith, S., Norouzi, M. & Hinton, G. A simple framework for contrastive learning of visual representations. In III, H. D. & Singh, A. (eds.) Proceedings of the 37th International Conference on Machine Learning, 119 of Proceedings of Machine Learning Research, 1597–1607 (2020).
- [14]van den Oord, A., Li, Y. & Vinyals, O. Representation learning with contrastive predictive coding. Preprint at https://arxiv.org/abs/1807.03748 (2019). · arXiv
- [18]Du, S. et al. (eds.) Computer Vision – ECCV 2022 Workshops, 185–202 (2023).
- [31]Razeghi, O., Qiu, G., Williams, H. & Thomas, K. Skin lesion image recognition with computer vision and human in the loop. https://api.semanticscholar.org/CorpusID:15228838 (2012).
Cited by
Sorted by citations
A Multimodal Vision Foundation Model for Clinical Dermatology
Yan, S., Yu, Z., Primiero, C., Vico-Alonso, C., Wang, Z., Yang, L., Tschandl, P., Hu, M., Ju, L., Tan, G., et al. (2025) · Nature Medicine · doi:10.1038/s41591-025-03747-y · Cited by 117 · see citing works
“38) (both widely used foundation models pretrained on natural images with a ViT-Large 42 backbone), and SwAVDerm 35 (a self-supervised model pretrained on a large skin image dataset from search engines).”
InfluentialArtificial intelligence–enabled precision medicine for inflammatory skin diseases
Tang, A. S., Wei, M. L., Haemel, A., La, C., Sirota, M., & Lee, E. Y. (2026) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2025.10.596 · Cited by 6 · see citing works
- Authors
- Shen, Y., Li, H., Sun, C., Ji, H., Zhang, D., Hu, K., Tang, Y., Chen, Y., Wei, Z., & Lv, J.
- Affiliations
- The University of Tokyo (JP)
- Wuhan University of Technology (CN)
- Shanghai Business School (CN)
- Chongqing Medical University (CN)
- Shanghai Artificial Intelligence Laboratory (CN)
- Corresponding author
- Wei, Zikun
- Lv, Junwei
- Venue
- npj Digital Medicine
- Published
- 2024-02-08
- Volume
- 7
- Issue
- 1
- Pages
- 28-28
- ISSN
- 2398-6352 (ISSN-L)
- PMC
- PMC10853166
- OpenAlex
- W4391652257
- Cited by
- 21
- References
- 36
- Citations by year
- 2026: 4 · 2025: 16 · 2024: 1
- FWCI
- 3.58
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- npj Digital Medicine (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Chongqing Medical University
- Topics
- Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Health Sciences › Medicine › Dermatology › Medicine and Dermatology Studies History
- Keywords
- dermatological diagnosis, dermatological image classification, online health communities, contrastive learning, semi-supervised learning, annotation quality, transfer learning, early diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
