Cited by 117Cutaneous Melanoma Detection and ManagementAI in cancer detectionArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 78 of the 85 references it cites were found in a registry or an index; 6 of the 86 sources its claims cite were checked for support.

A multimodal vision foundation model for clinical dermatology

Open access

Abstract

Diagnosing and treating skin diseases require advanced visual skills across domains and the ability to synthesize information from multiple imaging modalities. While current deep learning models excel at specific tasks such as skin cancer diagnosis from dermoscopic images, they struggle to meet the complex, multimodal requirements of clinical practice. Here we introduce PanDerm, a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities. We evaluated PanDerm on 28 diverse benchmarks, including skin cancer screening, risk stratification, differential diagnosis of common and rare skin conditions, lesion segmentation, longitudinal monitoring, and metastasis prediction and prognosis. PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data. We conducted three reader studies to assess PanDerm’s potential clinical utility. PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis, improved clinicians’ skin cancer diagnostic accuracy by 11% on dermoscopy images and enhanced nondermatologist healthcare providers’ differential diagnosis by 16.5% across 128 skin conditions on clinical photographs. These results show PanDerm’s potential to improve patient care across diverse clinical scenarios and serve as a model for developing multimodal foundation models in other medical specialties, potentially accelerating the integration of artificial intelligence support in healthcare.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

PanDerm is a multimodal dermatology foundation model pretrained via self-supervised learning on over 2 million real-world skin images from 11 institutions across 4 imaging modalities. Across 28 benchmarks and 3 reader studies, it achieved state-of-the-art performance and improved clinician and nondermatologist diagnostic accuracy in several tasks.

Key findings

  • Pretraining used 2,149,706 unlabeled multimodal skin images: 757,890 (35.3%) TBP tiles, 537,047 (25.4%) dermatopathology tiles, 460,328 (21.4%) clinical images and 384,441 (17.9%) dermoscopic images. PanDerm achieved state-of-the-art performance on all 28 evaluated tasks, often using only 5–10% of labeled training data; it outperformed SwAVDerm using 33% less training data and reached best performance in 200 epochs versus 500–800 for MILAN, DINOv2 and MAE.
  • Diagnostic gains averaged 5.1%, 8.0%, 4.2% and 0.9% on dermoscopic, clinical, TBP and pathology datasets, respectively. On HAM10000 and PAD-UFES-20, PanDerm surpassed the next-best models by 4.7% and 9.0% (both P < 0.001). For broader skin conditions, weighted F1 improvements were 3.2%, 7.1% and 8.2% on MMT-09, DermNet and MMT-74.
  • In longitudinal tasks, change detection AUROC improved from 0.596 to 0.706 on SDDI1 and from 0.683 to 0.767 on SDDI2; for malignant lesions, PanDerm reached AUROC 0.840, surpassing the next-best model by 15.0%. Metastasis prediction reached AUROC 0.964, and high-risk patients had shorter recurrence-free intervals (HR 5.63; 95% CI 2.87–11.02, P < 0.001), with time-dependent AUCs of 0.950, 0.931 and 0.909 at 3, 5 and 7 years.
  • TBP tasks: photodamage weighted F1 0.896, nevus counting AUROC 0.983, risk stratification AUROC 0.705 and BACC 0.657. Malignant lesion screening sensitivity was 0.893, detecting malignant lesions in 79 of 80 patients while reducing unnecessary examinations by 60.8% versus melanographers (3,498 versus 8,913 lesions). Reader studies showed PanDerm outperformed average human reviewers by 10.2% in early melanoma detection, identified 77.5% (69/89) of melanomas at the first imaging time point versus 32.6% (29) for humans, improved skin cancer diagnostic accuracy from 0.69 to 0.80, and improved generalists’ top-3 accuracy by 16.5% across 128 skin conditions.

Methods

  • Design: PanDerm is a self-supervised multimodal foundation model with a ViT-Large visual encoder, a mask regressor and a CLIP-Large teacher model. Pretraining used masked latent modeling and CLIP feature alignment, with 50% mask ratio, 224 × 224-pixel inputs, 500 epochs, batch size 480 per GPU (effective 1,920), and 4 × 80-GB NVIDIA H100 GPUs for approximately 5 days and 7 hours.
  • Data and evaluation: Pretraining data came from 11 sources across 4 modalities. Evaluation covered 28 benchmarks, including 10 public datasets from 4 imaging modalities and 7 international sites, plus internal datasets such as MMT-09, MMT-74, SD-128, SDDI1, SDDI2, ComBineMel and TBP cohorts. Three reader studies assessed early melanoma detection, human–AI collaboration for skin cancer diagnosis, and human–AI collaboration for 128 skin conditions.
  • Analysis: Main metrics included AUROC, weighted F1, Dice similarity coefficient, Jaccard index, sensitivity, top-1/top-3 accuracy and diagnostic confidence. Statistical approaches included k-fold cross-validation, nonparametric bootstrapping with 1,000 replicates, two-sided t-tests, permutation tests, Kaplan–Meier analysis and Cox proportional hazards regression.

Limitations

  • The paper states that validation covered approximately 200 skin conditions, only a fraction of over 1,000 known dermatological diagnoses and smaller than some prior work (for example, 445 conditions), with limited coverage of rare genetic disorders, complex systemic diseases and clinical variants.
  • The paper states that robustness and fairness evaluations mainly reflect overall accuracy rather than disease-specific analysis, have varying disease coverage across anatomical locations, and focus primarily on single imaging modalities; comprehensive bias assessment requires metrics beyond overall accuracy, and equitable stand-alone performance may not translate to unbiased human–AI collaboration.
  • Evident from the text: the evaluations are retrospective benchmark and reader studies, and the paper does not report prospective clinical deployment or patient-outcome trials.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41591-025-03747-y

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1 Overview of this study. a–c, Pretraining dataset: 2.1 million dermatological images from 11 clinical sources across 4 modalities, shown by modality (a), source (b) and institution (c). d, PanDerm interprets multiple imaging modalities for various dermatology tasks, evaluated in real-world melanoma screening and three reader studies. Image types include dermatopathology (microscopic biopsy specimens), clinical (wide-field lesion and surrounding skin), dermoscopic (close-up dermoscope images) and TBP tiles (lesion crops). e, Architecture: ViT-large encoder, regressor and CLIP-based teacher model, with representation reconstruction and CLIP latent alignment objectives. f, Performance versus pretraining data size and epochs (average AUROC on 8 benchmarks) compared with alternative strategies. g, PanDerm outperforms existing models on 28 evaluation datasets across 4 modalities. All icons in d are from Flaticon.com, except for the risk stratification, lesion change detection and survival analysis icons, which are from Microsoft PowerPoint.
Fig. 1 Overview of this study. a–c, Pretraining dataset: 2.1 million dermatological images from 11 clinical sources across 4 modalities, shown by modality (a), source (b) and institution (c). d, PanDerm interprets multiple imaging modalities for various dermatology tasks, evaluated in real-world melanoma screening and three reader studies. Image types include dermatopathology (microscopic biopsy specimens), clinical (wide-field lesion and surrounding skin), dermoscopic (close-up dermoscope images) and TBP tiles (lesion crops). e, Architecture: ViT-large encoder, regressor and CLIP-based teacher model, with representation reconstruction and CLIP latent alignment objectives. f, Performance versus pretraining data size and epochs (average AUROC on 8 benchmarks) compared with alternative strategies. g, PanDerm outperforms existing models on 28 evaluation datasets across 4 modalities. All icons in d are from Flaticon.com, except for the risk stratification, lesion change detection and survival analysis icons, which are from Microsoft PowerPoint.CC BY · PubMed Central
Fig. 2 PanDerm’s versatile capacity in diverse diagnosis tasks. a, Performance comparison of PanDerm versus other pretrained models on 10 pigmented skin lesion datasets across multiple centers and modalities. n, data size; c, class number. Metrics: AUROC for binary class (c = 2) and W_F1 score for multi-class (c > 2) datasets. The dashed lines indicate the average model performance across datasets. b, Comparison between PanDerm and other pretrained models in label efficiency generalization on four representative datasets, showing performance at various training data percentages. The vertical dashed lines indicate the data quantity needed for PanDerm to match existing model performance. c, External validation for melanoma diagnosis across 7 datasets. d, Performance evaluation of general skin condition classification (up to 74 classes) using clinical images. The error bars in a, c and d show 95% CIs; bar centers in a, c and d represent mean values; dots in b represent mean values. Estimates were computed using nonparametric bootstrapping with 1,000 bootstrap replicates. P values were calculated using a two-sided t-test.
Fig. 2 PanDerm’s versatile capacity in diverse diagnosis tasks. a, Performance comparison of PanDerm versus other pretrained models on 10 pigmented skin lesion datasets across multiple centers and modalities. n, data size; c, class number. Metrics: AUROC for binary class (c = 2) and W_F1 score for multi-class (c > 2) datasets. The dashed lines indicate the average model performance across datasets. b, Comparison between PanDerm and other pretrained models in label efficiency generalization on four representative datasets, showing performance at various training data percentages. The vertical dashed lines indicate the data quantity needed for PanDerm to match existing model performance. c, External validation for melanoma diagnosis across 7 datasets. d, Performance evaluation of general skin condition classification (up to 74 classes) using clinical images. The error bars in a, c and d show 95% CIs; bar centers in a, c and d represent mean values; dots in b represent mean values. Estimates were computed using nonparametric bootstrapping with 1,000 bootstrap replicates. P values were calculated using a two-sided t-test.CC BY · PubMed Central
Fig. 3 Short-term lesion change detection and metastasis prognosis results. a, SDDI1 dataset (n = 585 dermoscopic images) statistics: ratio of changed lesions, ratio of changed malignant lesions during follow-up, and follow-up time distribution. b, Ratio of changed lesions in the SDDI2 dataset (n = 458 dermoscopic images). c, Ablation study on preprocessing methods using SDDI1 and SDDI2 ‘Default’ (direct input), ‘With warp’ (registration only), ‘With mask’ (lesion segmentation) and ‘With whole pipeline’ (complete preprocessing as in Extended Data Fig. 3). For change detection in SDDI1 and SDDI2, all models were evaluated using the whole preprocessing pipeline. d, Performance of binary metastasis prediction (control versus metastasis) in ComBineMel (n = 680 dermoscopic images) by AUROC. e, Scheme of PanDerm for melanoma metastasis and prognosis prediction. MS, metastasis. f, Distribution of metastasis types in the ComBineMel dataset (n = 680 dermoscopic images). g, Kaplan–Meier curves for the RFI in invasive melanoma patients (ComBineMel (n = 305 patients)), stratified by PanDerm prediction scores. h, Forest plot of HRs for PanDerm; stratified groups in invasive melanoma patients. i, Time-dependent AUC of PanDerm versus clinical variable score combinations in ComBineMel. j, Time-dependent AUC comparison of PanDerm and other pretrained models in ComBineMel. The error bars in c, d, i and j and error bands in g show 95% CIs; the bar centers indicate means. All estimates were derived from fivefold cross-validation. P values in d were derived from two-sided t-tests and those in h from Wald tests within Cox proportional hazards models. Icons in e from Flaticon.com.
Fig. 3 Short-term lesion change detection and metastasis prognosis results. a, SDDI1 dataset (n = 585 dermoscopic images) statistics: ratio of changed lesions, ratio of changed malignant lesions during follow-up, and follow-up time distribution. b, Ratio of changed lesions in the SDDI2 dataset (n = 458 dermoscopic images). c, Ablation study on preprocessing methods using SDDI1 and SDDI2 ‘Default’ (direct input), ‘With warp’ (registration only), ‘With mask’ (lesion segmentation) and ‘With whole pipeline’ (complete preprocessing as in Extended Data Fig. 3). For change detection in SDDI1 and SDDI2, all models were evaluated using the whole preprocessing pipeline. d, Performance of binary metastasis prediction (control versus metastasis) in ComBineMel (n = 680 dermoscopic images) by AUROC. e, Scheme of PanDerm for melanoma metastasis and prognosis prediction. MS, metastasis. f, Distribution of metastasis types in the ComBineMel dataset (n = 680 dermoscopic images). g, Kaplan–Meier curves for the RFI in invasive melanoma patients (ComBineMel (n = 305 patients)), stratified by PanDerm prediction scores. h, Forest plot of HRs for PanDerm; stratified groups in invasive melanoma patients. i, Time-dependent AUC of PanDerm versus clinical variable score combinations in ComBineMel. j, Time-dependent AUC comparison of PanDerm and other pretrained models in ComBineMel. The error bars in c, d, i and j and error bands in g show 95% CIs; the bar centers indicate means. All estimates were derived from fivefold cross-validation. P values in d were derived from two-sided t-tests and those in h from Wald tests within Cox proportional hazards models. Icons in e from Flaticon.com.CC BY · PubMed Central
Fig. 4 Skin phenotype assessment and malignant lesion screening using TBP. a, Illustration of PanDerm processing multimodal TBP data for skin phenotype quantification, risk prediction and malignant lesion screening. b,c, Class distribution of skin phenotype quantification for photodamage risk assessment (n = 5,022 TBP tiles) (b) and nevus counting (n = 28,227 TBP tiles) (c) in datasets. d,e, Class distribution of risk prediction (d) and benign and malignant lesions (n = 196,933 TBP tiles) (e). g, Photodamage risk assessment and nevus counting performance by W_F1 and AUROC. h, Risk prediction performance by AUROC and BACC. Error bars in g and h show 95% CIs; bar centers represent mean values. Estimates were computed with nonparametric bootstrapping using 1,000 bootstrap replicates. P values were calculated with a two-sided t-test. j, Malignant lesion screening performance by sensitivity. Left: using only TBP data; right: integrating measurement information. The numbers below the bars indicate the recommended suspicious lesion count. k, Number of malignant lesions detected in the test set. f, UMAP plot of PanDerm screening results for test lesions. i, UMAP plot of human screening results for test lesions. l, UMAP plot of PanDerm risk prediction results for test lesions. All icons in a are from Flaticon.com, except the risk prediction icon, which is from Microsoft PowerPoint.
Fig. 4 Skin phenotype assessment and malignant lesion screening using TBP. a, Illustration of PanDerm processing multimodal TBP data for skin phenotype quantification, risk prediction and malignant lesion screening. b,c, Class distribution of skin phenotype quantification for photodamage risk assessment (n = 5,022 TBP tiles) (b) and nevus counting (n = 28,227 TBP tiles) (c) in datasets. d,e, Class distribution of risk prediction (d) and benign and malignant lesions (n = 196,933 TBP tiles) (e). g, Photodamage risk assessment and nevus counting performance by W_F1 and AUROC. h, Risk prediction performance by AUROC and BACC. Error bars in g and h show 95% CIs; bar centers represent mean values. Estimates were computed with nonparametric bootstrapping using 1,000 bootstrap replicates. P values were calculated with a two-sided t-test. j, Malignant lesion screening performance by sensitivity. Left: using only TBP data; right: integrating measurement information. The numbers below the bars indicate the recommended suspicious lesion count. k, Number of malignant lesions detected in the test set. f, UMAP plot of PanDerm screening results for test lesions. i, UMAP plot of human screening results for test lesions. l, UMAP plot of PanDerm risk prediction results for test lesions. All icons in a are from Flaticon.com, except the risk prediction icon, which is from Microsoft PowerPoint.CC BY · PubMed Central
Fig. 5 Performance of PanDerm in human–AI collaborative skin cancer diagnosis using dermoscopic images. a, Reader study overview: 41 users answered 3,320 questions on the ISIC2018 Task 3 test set (n = 1,511 images, 7 classes). b, Diagnostic accuracy comparison: without versus with PanDerm support (P < 0.001; two-sided paired t-test; n = 41 readers). c, Accuracy comparison without versus with PanDerm by experience level based on experience per experience: low (n = 11), medium (n = 21) and high (n = 9). d, Accuracy comparison without versus with PanDerm by diagnostic class based on readings per class: MEL (n = 332), BCC (n = 166), AKIEC (n = 166), BKL (n = 166), NV (n = 498), DF (n = 166) and VASC (n = 166). The error bars represent 95% CIs; bar centers represent means.
Fig. 5 Performance of PanDerm in human–AI collaborative skin cancer diagnosis using dermoscopic images. a, Reader study overview: 41 users answered 3,320 questions on the ISIC2018 Task 3 test set (n = 1,511 images, 7 classes). b, Diagnostic accuracy comparison: without versus with PanDerm support (P < 0.001; two-sided paired t-test; n = 41 readers). c, Accuracy comparison without versus with PanDerm by experience level based on experience per experience: low (n = 11), medium (n = 21) and high (n = 9). d, Accuracy comparison without versus with PanDerm by diagnostic class based on readings per class: MEL (n = 332), BCC (n = 166), AKIEC (n = 166), BKL (n = 166), NV (n = 498), DF (n = 166) and VASC (n = 166). The error bars represent 95% CIs; bar centers represent means.CC BY · PubMed Central
Fig. 6 Performance of PanDerm in human–AI collaborative assessment of 128 skin conditions using clinical images. a, Reader demographics (n = 37 readers): dermatology group (n = 20 readers) including residents and specialists, and generalist group (n = 17 readers) including pre-vocational trainees, general practitioners, nurses and clinical trial assistants. Each reviewed up to 50 of 200 cases. b, Geographic distribution of readers. c–e, Reader-wise analysis (each data point represents one reader, n = 37 readers): comparisons without versus with PanDerm support for: top 1 diagnostic assessment score (1–4) (c), top 3 diagnostic accuracy (d) and diagnostic confidence score (1–4) (e). f, Diagnosis change ratio after PanDerm support by specialization group. g,h, Class-wise analysis (each data point represents one skin condition class): comparisons without versus with PanDerm support by specialization groups for the top 1 diagnostic assessment score (1–4) (g) and top 3 diagnostic accuracy (h) (n = 128 classes per group). i,j, Comparisons without versus with PanDerm support by disease category for the top 1 diagnostic assessment score (1–4) (i) and the top 3 diagnostic accuracy (j), stratified by inflammatory (n = 78 classes), neoplastic (n = 37 classes) and other (n = 13 classes) conditions. P values in c–e were calculated using two-sided paired t-test across readers, while P values in g–j were calculated using two-sided paired t-test across classes. In all the boxplots, the horizontal lines represent medians and the white dots represent means. The upper and lower box limits indicate the 1st and 3rd quartiles, with whiskers extending to 1.5 times the interquartile range. Error bars represent 95% CIs.
Fig. 6 Performance of PanDerm in human–AI collaborative assessment of 128 skin conditions using clinical images. a, Reader demographics (n = 37 readers): dermatology group (n = 20 readers) including residents and specialists, and generalist group (n = 17 readers) including pre-vocational trainees, general practitioners, nurses and clinical trial assistants. Each reviewed up to 50 of 200 cases. b, Geographic distribution of readers. c–e, Reader-wise analysis (each data point represents one reader, n = 37 readers): comparisons without versus with PanDerm support for: top 1 diagnostic assessment score (1–4) (c), top 3 diagnostic accuracy (d) and diagnostic confidence score (1–4) (e). f, Diagnosis change ratio after PanDerm support by specialization group. g,h, Class-wise analysis (each data point represents one skin condition class): comparisons without versus with PanDerm support by specialization groups for the top 1 diagnostic assessment score (1–4) (g) and top 3 diagnostic accuracy (h) (n = 128 classes per group). i,j, Comparisons without versus with PanDerm support by disease category for the top 1 diagnostic assessment score (1–4) (i) and the top 3 diagnostic accuracy (j), stratified by inflammatory (n = 78 classes), neoplastic (n = 37 classes) and other (n = 13 classes) conditions. P values in c–e were calculated using two-sided paired t-test across readers, while P values in g–j were calculated using two-sided paired t-test across classes. In all the boxplots, the horizontal lines represent medians and the white dots represent means. The upper and lower box limits indicate the 1st and 3rd quartiles, with whiskers extending to 1.5 times the interquartile range. Error bars represent 95% CIs.CC BY · PubMed Central
Extended Data Fig. 1 Performance of PanDerm versus other pretrained models on 10 pigmented skin lesion datasets across multiple centers and modalities. a. Performances are measured by weighted F1 (W F1). b. Performances are measured by AUROC. c. Performances are measured by AUPR. d. Perfor- mances are measured by BACC. n: data size, c: class number. Dashed lines show the average performance of each model across different datasets. Estimates were computed using nonparametric bootstrapping with 1000 bootstrap replicates. P-values calculated using a two-sided t-test. Error bar, 95% CIs; bar centers, means.
Extended Data Fig. 1 Performance of PanDerm versus other pretrained models on 10 pigmented skin lesion datasets across multiple centers and modalities. a. Performances are measured by weighted F1 (W F1). b. Performances are measured by AUROC. c. Performances are measured by AUPR. d. Perfor- mances are measured by BACC. n: data size, c: class number. Dashed lines show the average performance of each model across different datasets. Estimates were computed using nonparametric bootstrapping with 1000 bootstrap replicates. P-values calculated using a two-sided t-test. Error bar, 95% CIs; bar centers, means.CC BY · PubMed Central
Extended Data Fig. 2 Label efficiency generalization results on additional tasks. a. Label efficiency analysis for photodamage risk assessment using Total Body Photography (TBP) images. Results demonstrate model performance with limited labeled data available. PanDerm outperformed the second-best models using only 10% of labeled images. b. Label efficiency analysis for melanoma classification using whole slide dermatopathology images. Results illustrate model performance with limited labeled data. PanDerm surpassed the second-best models using less than 30% of labeled images.
Extended Data Fig. 2 Label efficiency generalization results on additional tasks. a. Label efficiency analysis for photodamage risk assessment using Total Body Photography (TBP) images. Results demonstrate model performance with limited labeled data available. PanDerm outperformed the second-best models using only 10% of labeled images. b. Label efficiency analysis for melanoma classification using whole slide dermatopathology images. Results illustrate model performance with limited labeled data. PanDerm surpassed the second-best models using less than 30% of labeled images.CC BY · PubMed Central
Extended Data Fig. 3 Longitudinal dermoscopic image-based lesion change detection using PanDerm. For comparing subtle changes in paired lesions during short-term follow-up (for example, 3 months), images undergo dark corner detection and removal, skin inpainting, registration, and lesion segmentation. This allows models to focus on subtle differences between lesions at different time points. Panda icon from Flaticon.com.
Extended Data Fig. 3 Longitudinal dermoscopic image-based lesion change detection using PanDerm. For comparing subtle changes in paired lesions during short-term follow-up (for example, 3 months), images undergo dark corner detection and removal, skin inpainting, registration, and lesion segmentation. This allows models to focus on subtle differences between lesions at different time points. Panda icon from Flaticon.com.CC BY · PubMed Central
Extended Data Fig. 4 SHAP (SHapley Additive exPlanations) value plot. It shows the impact of various measurement variables captured by the 3D TBP machine on the model output. The plot displays the relative importance and directional influence of each feature, with colors indicating high (red) to low (blue) feature values, and the x-axis representing the SHAP value or impact on the model’s prediction. Features are ordered by their overall importance, with ‘nevi confidence’ having the highest impact and ‘stdLExt’ the lowest.
Extended Data Fig. 4 SHAP (SHapley Additive exPlanations) value plot. It shows the impact of various measurement variables captured by the 3D TBP machine on the model output. The plot displays the relative importance and directional influence of each feature, with colors indicating high (red) to low (blue) feature values, and the x-axis representing the SHAP value or impact on the model’s prediction. Features are ordered by their overall importance, with ‘nevi confidence’ having the highest impact and ‘stdLExt’ the lowest.CC BY · PubMed Central
Extended Data Fig. 5 Quantitative skin lesion segmentation results. a, b. Segmentation performance measured by dice score (DSC) and Jaccard index (JAC) for PanDerm and baseline models on ISIC2018 (n=2,074 dermoscopic images) and HAM10000 (n=7,011 dermoscopic images) datasets. c, d. Label efficiency generalization performance for PanDerm and baselines, showing mean DSC and JAC on ISIC2018 and HAM10000 datasets. Error bars in a, b indicate 95% confidence intervals; bar centers represent mean values; points in c, d denote mean values. All estimates are derived from five replicas with different seeds. Statistical significance was assessed using two-sided t-tests.
Extended Data Fig. 5 Quantitative skin lesion segmentation results. a, b. Segmentation performance measured by dice score (DSC) and Jaccard index (JAC) for PanDerm and baseline models on ISIC2018 (n=2,074 dermoscopic images) and HAM10000 (n=7,011 dermoscopic images) datasets. c, d. Label efficiency generalization performance for PanDerm and baselines, showing mean DSC and JAC on ISIC2018 and HAM10000 datasets. Error bars in a, b indicate 95% confidence intervals; bar centers represent mean values; points in c, d denote mean values. All estimates are derived from five replicas with different seeds. Statistical significance was assessed using two-sided t-tests.CC BY · PubMed Central
Extended Data Fig. 6 Qualitative skin lesion segmentation results. a. Comparison of PanDerm against baseline models on challenging examples from HAM10000. Red contours indicate ground truth masks, while cyan contours show model predictions. b. PanDerm segmentation results on a random selection of images from HAM10000.
Extended Data Fig. 6 Qualitative skin lesion segmentation results. a. Comparison of PanDerm against baseline models on challenging examples from HAM10000. Red contours indicate ground truth masks, while cyan contours show model predictions. b. PanDerm segmentation results on a random selection of images from HAM10000.CC BY · PubMed Central
Extended Data Fig. 7 Early melanoma detection results (reader study 1). Comparing PanDerm to 12 clinicians (7 experienced dermatologists, 5 dermatology residents). X-axis: 89 melanoma lesion IDs; Y-axis: lesion image sequence length. Points on the histogram represent the initial time points of correct melanoma diagnoses. Points below y=0 correspond to melanoma lesions undetected throughout the sequence.
Extended Data Fig. 7 Early melanoma detection results (reader study 1). Comparing PanDerm to 12 clinicians (7 experienced dermatologists, 5 dermatology residents). X-axis: 89 melanoma lesion IDs; Y-axis: lesion image sequence length. Points on the histogram represent the initial time points of correct melanoma diagnoses. Points below y=0 correspond to melanoma lesions undetected throughout the sequence.CC BY · PubMed Central
Extended Data Fig. 8 Sunburst plot of standard ontology on SD-128 dataset. Four experienced dermatologists collaboratively developed the standard ontology to systematically categorize the 128 skin conditions and facilitate expert evaluation in reader study3.
Extended Data Fig. 8 Sunburst plot of standard ontology on SD-128 dataset. Four experienced dermatologists collaboratively developed the standard ontology to systematically categorize the 128 skin conditions and facilitate expert evaluation in reader study3.CC BY · PubMed Central
Extended Data Fig. 9 Demographic distribution of participants in reader study 3. a. Specialty distribution of participants. b. Career stage distribution of participants. c. Experience levels distribution by years.
Extended Data Fig. 9 Demographic distribution of participants in reader study 3. a. Specialty distribution of participants. b. Career stage distribution of participants. c. Experience levels distribution by years.CC BY · PubMed Central

Verification

Claims the paper makes

66 Own result · 48 Not checked yet · 29 Own method · 14 Own interpretation · 4 No basis given · 2 Supported so far

Verify all 80 references — free account

Show all 83 claims

  • It may state two separate claims as one. 44
  • Probably not a substantial claim. 44
80 of 80
  • The paper relies on this without support.

    Current AI models for dermatology struggle to integrate various data types and imaging modalities, reducing their utility in different real-world clinical settings.

    “These models struggle to integrate various data types and imaging modalities, reducing their utility in different real-world clinical settings.”
  • The paper relies on this without support.

    The absence of integrated AI solutions capable of supporting various clinical workflows currently hampers the practical impact of AI in dermatology.

    “The absence of integrated AI solutions capable of supporting these various workflows currently hampers the practical impact of AI in dermatology.”
  • The paper relies on this without support.

    Through comprehensive pretraining on large and diverse data, foundation models develop versatile representations that can effectively adapt to various clinical scenarios, outperforming previous deep learning models in downstream tasks.

    “Through comprehensive pretraining on large and diverse data, these models develop versatile representations that can effectively adapt to various clinical scenarios, outperforming previous deep learning models in downstream tasks.”
  • The paper relies on this without support.

    Existing dermatology self-supervised approaches are often limited by dataset size, diversity or lack of real patient data.

    “However, these approaches are often limited by dataset size, diversity or the lack of real patient data.”
  • The paper relies on this without support.

    General biomedical foundation models struggle with domain-specific data scarcity and integrating heterogeneous modalities for comprehensive clinical analysis.

    “while general biomedical models, despite their broad scope, struggle with domain-specific data scarcity and integrating heterogeneous modalities for comprehensive clinical analysis.”
  • The paper relies on this without support.

    Current AI systems are often restricted to isolated tasks and unable to address diverse demands of medical decision-making.

    “Current AI systems are often restricted to isolated tasks and are unable to address the diverse demands of medical decision-making.”
  • The paper relies on this without support.

    There are over 1,000 known dermatological conditions.

    “over 1,000 diagnoses”
  • The paper relies on this without support.

    Biopsy serves as the reference standard for melanoma diagnosis.

    “While biopsy serves as our reference standard”
  • The paper relies on this without support.

    The DermaChallenge platform user base mean score is a valid measure of experience.

    “The number of correct answers in the first iteration of these levels, normalized against the mean score of the entire DermaChallenge platform user base, served as a score of experience.”
  • The paper relies on this without support.

    The 128 skin conditions ontology is a valid standard for evaluating diagnostic accuracy.

    “four experienced dermatologists collaboratively developed a standard ontology to systematically categorize the 128 skin conditions”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

2 of 2

Citations

Cites

Research works printed but not linked: 8

  1. Rapini, R. P. Practical Dermatopathology (Elsevier Health Sciences, 2021).
  2. Dosovitskiy, A. et al. An image is worth 16×16 words: transformers for image recognition at scale. In Proc. 9th International Conference on Learning Representations https://openreview.net/forum?id=YicbFdNTTy (OpenReview.net, 2021).
  3. Zhang, X. et al. Cae v2: context autoencoder with clip latent alignment. Trans. Mach. Learning Res. (2023).
  4. Alcantarilla, P. F. & Solutions, T. Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1281–1298 (2011).
  5. Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Proc. International Conference on Machine Learning (eds Dy, J. & Krause, A.) 2127–2136 (PMLR, 2018).
  6. Loshchilov, I. et al. Decoupled weight decay regularization. In Proc. 7th International Conference on Learning Representations https://openreview.net/forum?id=Bkg6RiCqY7 (OpenReview.net, 2019).
  7. Mendonça, T. et al. in Dermoscopy Image Analysis (eds Celebi, M. E. et al.) Ch. 13 420–438 (CRC Press, 2015).
  8. Kahler, S. et al. Automated photodamage assessment from 3D total body photography for an objective assessment of melanoma risk. Australas. J. Dermatol. 64, 8 (2023).

Cited by

Sorted by citations

See all 117

Metadata

Authors
Yan, S., Yu, Z., Primiero, C., Vico-Alonso, C., Wang, Z., Yang, L., Tschandl, P., Hu, M., Ju, L., Tan, G., Tang, V., Ng, A. B., Powell, D., Bonnington, P., See, S., Magnaterra, E., Ferguson, P., Nguyen, J., Guitera, P., Banuls, J., Janda, M., Mar, V., Kittler, H., Soyer, H. P., & Ge, Z.
Affiliations
  • Monash University (AU)
  • The University of Queensland (AU)
  • Medical University of Vienna (AT)
  • Nvidia (United States) (US)
  • University of Florence (IT)
  • Royal Prince Alfred Hospital (AU)
  • The Alfred Hospital (AU)
Corresponding author
Ge, Zongyuan
Venue
Nature Medicine
Published
2025-06-06
Volume
31
Issue
8
Pages
2691-2702
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
OpenAlex
W4411099958
Cited by
117
References
85
Citations by year
2026: 85 · 2025: 32
FWCI
48.05
Type
journal article
Language
en
Open access
Locations
  • Nature Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
multimodal foundation models, skin diseases, skin cancer diagnosis, melanoma detection, dermoscopy, self-supervised pre-training, differential diagnosis, lesion segmentation, long-term monitoring, metastasis prediction, clinical photographs, reader study, clinical implementation of AI
MeSH terms
Deep Learning, Dermatology, Diagnosis, Differential, Humans, Melanoma, Skin Diseases, Skin Neoplasms, Dermoscopy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-07, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-07 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (gpt-6-sol).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review