CoChat
Sign Up
Cited by 117Cutaneous Melanoma Detection and ManagementAI in cancer detectionArtificial Intelligence in Healthcare and Education

A Multimodal Vision Foundation Model for Clinical Dermatology

Open access
Yan, Siyuan, Yu, Zhen, Primiero, Clare, Vico-Alonso, Cristina, Wang, Zhonghua, Yang, Litao, Tschandl, Philipp, Hu, Ming +17 more·2025·Nature Medicine
DOI 10.1038/s41591-025-03747-yPMC PMC12353815
Open this paper in CoChat — free account

Abstract

Diagnosing and treating skin diseases require advanced visual skills across domains and the ability to synthesize information from multiple imaging modalities. While current deep learning models excel at specific tasks such as skin cancer diagnosis from dermoscopic images, they struggle to meet the complex, multimodal requirements of clinical practice. Here we introduce PanDerm, a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities. We evaluated PanDerm on 28 diverse benchmarks, including skin cancer screening, risk stratification, differential diagnosis of common and rare skin conditions, lesion segmentation, longitudinal monitoring, and metastasis prediction and prognosis. PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data. We conducted three reader studies to assess PanDerm’s potential clinical utility. PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis, improved clinicians’ skin cancer diagnostic accuracy by 11% on dermoscopy images and enhanced nondermatologist healthcare providers’ differential diagnosis by 16.5% across 128 skin conditions on clinical photographs. These results show PanDerm’s potential to improve patient care across diverse clinical scenarios and serve as a model for developing multimodal foundation models in other medical specialties, potentially accelerating the integration of artificial intelligence support in healthcare.

AI snapshot

Written from the paper’s full text.

TL;DR

PanDerm is a multimodal dermatology foundation model pretrained via self-supervised learning on over 2 million real-world skin images from 11 institutions across 4 imaging modalities. Across 28 benchmarks and 3 reader studies, it achieved state-of-the-art performance and improved clinician and nondermatologist diagnostic accuracy in several tasks.

Key findings

  • Pretraining used 2,149,706 unlabeled multimodal skin images: 757,890 (35.3%) TBP tiles, 537,047 (25.4%) dermatopathology tiles, 460,328 (21.4%) clinical images and 384,441 (17.9%) dermoscopic images. PanDerm achieved state-of-the-art performance on all 28 evaluated tasks, often using only 5–10% of labeled training data; it outperformed SwAVDerm using 33% less training data and reached best performance in 200 epochs versus 500–800 for MILAN, DINOv2 and MAE.
  • Diagnostic gains averaged 5.1%, 8.0%, 4.2% and 0.9% on dermoscopic, clinical, TBP and pathology datasets, respectively. On HAM10000 and PAD-UFES-20, PanDerm surpassed the next-best models by 4.7% and 9.0% (both P < 0.001). For broader skin conditions, weighted F1 improvements were 3.2%, 7.1% and 8.2% on MMT-09, DermNet and MMT-74.
  • In longitudinal tasks, change detection AUROC improved from 0.596 to 0.706 on SDDI1 and from 0.683 to 0.767 on SDDI2; for malignant lesions, PanDerm reached AUROC 0.840, surpassing the next-best model by 15.0%. Metastasis prediction reached AUROC 0.964, and high-risk patients had shorter recurrence-free intervals (HR 5.63; 95% CI 2.87–11.02, P < 0.001), with time-dependent AUCs of 0.950, 0.931 and 0.909 at 3, 5 and 7 years.
  • TBP tasks: photodamage weighted F1 0.896, nevus counting AUROC 0.983, risk stratification AUROC 0.705 and BACC 0.657. Malignant lesion screening sensitivity was 0.893, detecting malignant lesions in 79 of 80 patients while reducing unnecessary examinations by 60.8% versus melanographers (3,498 versus 8,913 lesions). Reader studies showed PanDerm outperformed average human reviewers by 10.2% in early melanoma detection, identified 77.5% (69/89) of melanomas at the first imaging time point versus 32.6% (29) for humans, improved skin cancer diagnostic accuracy from 0.69 to 0.80, and improved generalists’ top-3 accuracy by 16.5% across 128 skin conditions.

Methods

  • Design: PanDerm is a self-supervised multimodal foundation model with a ViT-Large visual encoder, a mask regressor and a CLIP-Large teacher model. Pretraining used masked latent modeling and CLIP feature alignment, with 50% mask ratio, 224 × 224-pixel inputs, 500 epochs, batch size 480 per GPU (effective 1,920), and 4 × 80-GB NVIDIA H100 GPUs for approximately 5 days and 7 hours.
  • Data and evaluation: Pretraining data came from 11 sources across 4 modalities. Evaluation covered 28 benchmarks, including 10 public datasets from 4 imaging modalities and 7 international sites, plus internal datasets such as MMT-09, MMT-74, SD-128, SDDI1, SDDI2, ComBineMel and TBP cohorts. Three reader studies assessed early melanoma detection, human–AI collaboration for skin cancer diagnosis, and human–AI collaboration for 128 skin conditions.
  • Analysis: Main metrics included AUROC, weighted F1, Dice similarity coefficient, Jaccard index, sensitivity, top-1/top-3 accuracy and diagnostic confidence. Statistical approaches included k-fold cross-validation, nonparametric bootstrapping with 1,000 replicates, two-sided t-tests, permutation tests, Kaplan–Meier analysis and Cox proportional hazards regression.

Limitations

  • The paper states that validation covered approximately 200 skin conditions, only a fraction of over 1,000 known dermatological diagnoses and smaller than some prior work (for example, 445 conditions), with limited coverage of rare genetic disorders, complex systemic diseases and clinical variants.
  • The paper states that robustness and fairness evaluations mainly reflect overall accuracy rather than disease-specific analysis, have varying disease coverage across anatomical locations, and focus primarily on single imaging modalities; comprehensive bias assessment requires metrics beyond overall accuracy, and equitable stand-alone performance may not translate to unbiased human–AI collaboration.
  • Evident from the text: the evaluations are retrospective benchmark and reader studies, and the paper does not report prospective clinical deployment or patient-outcome trials.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1038/s41591-025-03747-y

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Claims the paper makes

0 of 86 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    PanDerm is a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities.

    “Here we introduce PanDerm, a multimodal dermatology foundation model pretrained through self-supervised learning on over 2 million real-world skin disease images from 11 clinical institutions across 4 imaging modalities.”
    • Also stated in: § Main · plain
      “Here we introduce PanDerm, a general-purpose, multimodal dermatology foundation model. Uniquely designed to integrate multiple imaging modalities, PanDerm is pretrained on over 2 million images sourced from 11 institutions across multiple countries, covering 4 imaging modalities spanning diverse dermatological conditions”
    Quote in the paperSubstantial claim
  • Findingown data

    PanDerm often outperformed existing models when using only 10% of labeled data.

    “PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data.”
    • Also stated in: § Main · plain
      “PanDerm achieves state-of-the-art performance on all tasks, often using only 5–10% of the labeled training data typically required.”
    Quote in the paperSubstantial claim
  • Findingown data

    PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis.

    “PanDerm outperformed clinicians by 10.2% in early-stage melanoma detection through longitudinal analysis”
    • Also stated in: § Reader studies · plain
      “In terms of overall accuracy, PanDerm outperformed the average human reviewer by 10.2% and surpassed the best-performing human by 3.6%.”
    Quote in the paperSubstantial claim
  • Findingown data

    PanDerm enhanced nondermatologist healthcare providers' differential diagnosis by 16.5% across 128 skin conditions on clinical photographs.

    “enhanced nondermatologist healthcare providers’ differential diagnosis by 16.5% across 128 skin conditions on clinical photographs”
    • Also stated in: § Reader studies · plain
      “greater improvements in both top 1 diagnostic scores (generalist group, +0.45; dermatology group, +0.25; Fig. 6g) and top 3 accuracy (generalist group, +16.5%; dermatology group, +10.3%; Fig. 6h).”
    Quote in the paperSubstantial claim
  • Findingown data

    PanDerm achieved state-of-the-art performance across all evaluated tasks.

    “PanDerm achieved state-of-the-art performance across all evaluated tasks, often outperforming existing models when using only 10% of labeled data.”
    • Also stated in: § Main · plain
      “PanDerm achieves state-of-the-art performance on all tasks, often using only 5–10% of the labeled training data typically required.”
    Quote in the paperSubstantial claim
  • May combine two claims. 44
  • Classified as probably not a substantial claim. 44

  • May combine two claims.

    Findingown data

    On HAM10000 and PAD-UFES-20, PanDerm surpassed next-best models by 4.7% (P < 0.001) and 9.0% (P < 0.001), respectively.

    “On representative dermoscopy and clinical benchmarks such as HAM10000 (ref. 34) and PAD-UFES-20 (ref. 43), PanDerm surpassed the next-best models by 4.7% (P < 0.001) and 9.0% (P < 0.001), respectively”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [34] Tschandl, P., Rosendahl, C. & Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 (2018). source

      Not checked in the free run.

    • cites [43] Pacheco, A. G. et al. PAD-UFES-20: a skin lesion dataset composed of patient data and clinical images collected from smartphones. Data Brief 32, 106221 (2020). source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    Using TBP images alone, PanDerm achieved sensitivity of 0.893, outperforming next-best model by 4.2%.

    “Using TBP images alone, PanDerm achieved a sensitivity of 0.893, outperforming the next-best model by 4.2% (Fig. 4j, left).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When clinical measurements available, PanDerm maintained advantage with 3.5% higher sensitivity, reaching sensitivity 0.893.

    “When clinical measurements were available for all models, PanDerm maintained its advantage with a 3.5% higher sensitivity (Fig. 4j, right), reaching a sensitivity of 0.893.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm showed exceptional accuracy in distinguishing melanomas likely to metastasize, achieving AUROC of 0.964, surpassing next-best model by 2.0% (P = 0.073).

    “PanDerm showed exceptional accuracy in distinguishing melanomas likely to metastasize, achieving an AUROC of 0.964 (95% CI 0.937–0.991), surpassing the next-best model by 2.0% (P = 0.073) (Fig. 3d).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm shows superior data scalability and training efficiency compared with existing self-supervised algorithms.

    “showing superior data scalability and training efficiency compared with existing self-supervised algorithms (Fig. 1f).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The model achieves unified representation learning across total-body photography and clinical, dermoscopic and dermatopathology images, enabling comprehensive patient analysis throughout diverse clinical workflows.

    “The model achieves unified representation learning across total-body photography (TBP) and clinical, dermoscopic and dermatopathology images, enabling comprehensive patient analysis throughout diverse clinical workflows (Fig. 1d).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Using optimized pipeline for all models, PanDerm achieved AUROC improvements of 4.3% in SDDI1 and 3.7% in SDDI2 over next-best model.

    “Using the optimized pipeline for all models, PanDerm achieved AUROC improvements of 4.3% in SDDI1 (P < 0.001) and 3.7% in SDDI2 over the next-best model (Fig. 3c, middle).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    Even using computationally efficient methods, PanDerm maintained its advantages, outperforming both linear-probe and fine-tune versions of BiomedGPT by 14.3% and 5.1%, respectively, in linear probing.

    “Even using computationally efficient methods, PanDerm maintained its advantages, outperforming both linear-probe and fine-tune versions of BiomedGPT by 14.3% and 5.1%, respectively, in linear probing (Supplementary Table 3).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The processing system combined with PanDerm's lesion change detection capabilities significantly improved change detection accuracy from 0.596 to 0.706 in SDDI1 (P < 0.001) and from 0.683 to 0.767 in SDDI2 (P < 0.001).

    “This processing system, combined with PanDerm’s advanced lesion change detection capabilities45, significantly improved change detection accuracy from 0.596 (95% confidence interval (CI) 0.567–0.624) to 0.706 (95% CI 0.686–0.725) in sequential digital dermoscopic imaging data (SDDI1) (Fig. 3a,c) (P < 0.001) and from 0.683 (95% CI 0.517–0.894) to 0.767 (95% CI 0.649–0.886) in SDDI2 (Fig. 3b,c, left”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [45] Zhang, B. et al. Short-term lesion change detection for melanoma screening with novel Siamese neural network. IEEE Trans. Med. Imaging 40, 840–851 (2020). source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    PanDerm showed strong performance even with limited training data, achieving comparable results to other models while using only 10-30% of labeled images.

    “PanDerm showed strong performance even with limited training data, achieving comparable results to other models while using only 10–30% of the labeled images (Fig. 2b and Supplementary Tables 5–10).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm maintained high predictive accuracy over extended follow-up periods, with time-dependent AUCs of 0.950, 0.931, 0.909 at 3, 5, 7 years, exceeding multi-clinical variables by 6.8%, 2.9%, 5.0%.

    “It maintained high predictive accuracy over extended follow-up periods, with time-dependent areas under the curve (AUCs) of 0.950 (95% CI 0.910–0.991), 0.931 (95% CI 0.887–0.976) and 0.909 (95% CI 0.880–0.937) at 3 years, 5 years and 7 years, exceeding multi-clinical variables by 6.8%, 2.9% and 5.0%, respectively (Fig. 3i).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm showed emergent capabilities in dermatology similar to DINOv2 in natural images, with linear probing performance comparable to full-parameter fine-tuning.

    “while showing emergent capabilities in dermatology similar to those of DINOv2 in natural images, with linear probing performance comparable to full-parameter fine-tuning (Supplementary Table 2).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    For lesions later diagnosed as malignant, PanDerm achieved AUROC of 0.840, surpassing next-best model by 15.0% (P < 0.01).

    “For lesions later diagnosed as malignant, PanDerm achieved an AUROC of 0.840 (95% CI 0.769–0.911), surpassing the next-best model by 15.0% (P < 0.01) (Fig. 3c, right).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm achieved weighted F1 improvements of 3.2%, 7.1% and 8.2% on MMT-09, DermNet and MMT-74, respectively, compared with next-best models (all P < 0.001).

    “PanDerm achieved weighted F1 improvements of 3.2%, 7.1% and 8.2% on MMT-09, DermNet and MMT-74, respectively, compared with the next-best models (all P < 0.001).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm consistently outperformed other AI approaches, showing improvements of 2.3%, 3.0%, 2.5% at 3, 5, 7 years.

    “PanDerm also consistently outperformed other AI approaches (Fig. 3j), showing improvements of 2.3%, 3.0% and 2.5% at 3 years, 5 years and 7 years, respectively.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In lesion-specific risk stratification, PanDerm ranked first with AUROC 0.705 and BACC 0.657, all results statistically significant (P < 0.001).

    “In lesion-specific risk stratification, PanDerm also ranked first with an AUROC of 0.705 (95% CI 0.698–0.712) and balanced accuracy (BACC) of 0.657 (95% CI 0.6513–0.663), with all results statistically significant (P < 0.001; Fig. 4d,h).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm detected malignant lesions in 79 out of 80 patients while reducing unnecessary examinations by 60.8% compared with melanographers (3,498 vs 8,913 lesions recommended).

    “Significantly, it detected malignant lesions in 79 out of 80 patients while reducing unnecessary examinations by 60.8% compared with melanographers (3,498 versus 8,913 lesions recommended for detailed examination) (Fig. 4j,k and Supplementary Table 18).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm maintained high performance even on clinical photographs not used during training, with AUROC gains of 4.0%, 2.6% and 2.1% on three external clinical datasets (all P < 0.001).

    “Notably, it maintained high performance even on clinical photographs that were not used during training, with AUROC gains of 4.0%, 2.6% and 2.1% on the three external clinical datasets (all P < 0.001).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm consistently outperformed all other models on 10 public datasets, achieving significant improvements on 9 of 10 datasets, with average gains of 5.1%, 8.0%, 4.2% and 0.9% on dermoscopic, clinical, TBP and pathology datasets, respectively.

    “PanDerm consistently outperformed all other models, achieving significant improvements on 9 of 10 datasets, with average gains of 5.1%, 8.0%, 4.2% and 0.9% on dermoscopic, clinical, TBP and pathology datasets, respectively (Fig. 2a).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    In cohort of 480 patients with 196,933 lesions from Australia, PanDerm achieved weighted F1 of 0.896 for photodamage assessment and AUROC of 0.983 for nevus counting, surpassing all other models (P < 0.05 and P < 0.001).

    “In a cohort of 480 patients with 196,933 lesions from Australia, PanDerm achieved a weighted F1 score of 0.896 (95% CI 0.879–0.913) for photodamage assessment and an AUROC of 0.983 (95% CI 0.979–0.987) for nevus counting, surpassing all other models (P < 0.05 and P < 0.001, respectively; Fig. 4b,c,g).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm showed significant superiority over all pretrained models on melanoma diagnosis using images from seven external medical centers, achieving higher AUROC scores across all external datasets.

    “PanDerm showed significant superiority over all pretrained models, achieving higher AUROC scores across all external datasets (Fig. 2c).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingreasoning

    PanDerm improved upon both traditional models and recent generalist medical models such as BiomedGPT.

    “These advantages enabled PanDerm to improve upon both traditional models42,60 and recent generalist medical models such as BiomedGPT41.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [42] Dosovitskiy, A. et al. An image is worth 16×16 words: transformers for image recognition at scale. In Proc. 9th International Conference on Learning Representations https://openreview.net/forum?id=YicbFdNTTy (OpenReview.net, 2021).

      Not checked in the free run.

    • cites [60] He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. In Proc. IEEE Conference on Computer Vision and Pattern Recognition 770–778 (IEEE, 2016). source

      Not checked in the free run.

    • cites [41] Zhang, K. et al. A generalist vision–language foundation model for diverse biomedical tasks. Nat. Med. 30, 3129–3141 (2024). source

      Not checked in the free run.

  • May combine two claims.

    Findingown data

    PanDerm processes images about four to five times faster than MedSAM while using less computational resources.

    “In addition, PanDerm offers practical advantages in clinical settings, processing images about four to five times faster than MedSAM while using less computational resources (Supplementary Table 23).”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    PanDerm outperformed average human reviewer by 10.2% and surpassed best-performing human by 3.6% in overall accuracy.

    “In terms of overall accuracy, PanDerm outperformed the average human reviewer by 10.2% and surpassed the best-performing human by 3.6%.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    When used independently, PanDerm achieved higher diagnostic accuracy than both unassisted readers (top 1: 3.6 vs 2.83; P < 0.001) and human-AI collaboration (3.6 vs 3.08; P < 0.001).

    “Furthermore, when used independently, PanDerm achieved higher diagnostic accuracy than both unassisted readers (top 1 scores: 3.6 versus 2.83; P < 0.001) and human–AI collaboration (top 1 scores: 3.6 versus 3.08; P < 0.001).”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    PanDerm with CLIP-large teachers significantly improved performance and outperformed CLIP-large itself.

    “Our model with CLIP-large teachers significantly improved performance and outperformed CLIP-large itself.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Findingown data

    The model showed robust performance, achieving an 80.4% mean (macro-averaged) recall, with notably high recall rates for critical skin lesions: 87.2% for melanoma and 86.0% for BCC.

    “The model showed robust performance, achieving an 80.4% mean (macro-averaged) recall, with notably high recall rates for critical skin lesions: 87.2% for melanoma and 86.0% for BCC.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingnone given

    Notable diagnostic variability between pathologists was observed.

    “Notable diagnostic variability between pathologists was observed.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Premisecitation

    Recent advances in foundation models have emerged as a promising direction to address the challenge of integrating AI solutions in dermatology.

    “Recent advances in foundation models have emerged as a promising direction to address this challenge15,16.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [15] Gui, H., Omiye, J. A., Chang, C. T. & Daneshjou, R. The promises and perils of foundation models in dermatology. J. Invest. Dermatol. 144, 1440–1448 (2024). source

      Not checked in the free run.

    • cites [16] Moor, M. et al. Foundation models for generalist medical artificial intelligence. Nature 616, 259–265 (2023). source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    In clinical practice, diagnosing and treating skin conditions involves a range of tasks, including total-body skin cancer detection and risk assessment, differential diagnosis of hundreds of dermatological conditions, multimodal image analysis, pathology interpretation, monitoring lesion changes and predicting outcomes.

    “In clinical practice, diagnosing and treating skin conditions involves a range of tasks, including total-body skin cancer detection and risk assessment2–5, differential diagnosis of hundreds of dermatological conditions such as inflammatory dermatoses and pigmentary disorders6, multimodal image analysis7,8, pathology interpretation9,10, monitoring lesion changes11,12 and predicting outcomes13,14.”
    Quote in the paperSubstantial claim0 of 7 sources checked
    • cites [2] Primiero, C. A. et al. A narrative review: opportunities and challenges in artificial intelligence skin image analyses using total body photography. J. Invest. Dermatol. 144, 1200–1207 (2024). source

      Not checked in the free run.

    • cites [3] Primiero, C. A. et al. A protocol for annotation of total body photography for machine learning to analyze skin phenotype and lesion classification. Front. Med. 11, 1380984 (2024). source

      Not checked in the free run.

    • cites [4] Olsen, C. M. et al. Risk stratification for melanoma: models derived and validated in a purpose-designed prospective cohort. J. Natl Cancer Inst. 110, 1075–1083 (2018). source

      Not checked in the free run.

    • cites [5] Usher-Smith, J. A., Emery, J., Kassianos, A. P. & Walter, F. M. Risk prediction models for melanoma: a systematic review. Cancer Epidemiol. Biomark. Prev. 23, 1450–1463 (2014). source

      Not checked in the free run.

    • cites [6] Liu, Y. et al. A deep learning system for differential diagnosis of skin diseases. Nat. Med. 26, 900–908 (2020). source

      Not checked in the free run.

    • cites [7] Yap, J., Yolland, W. & Tschandl, P. Multimodal skin lesion classification using deep learning. Exp. Dermatol. 27, 1261–1267 (2018). source

      Not checked in the free run.

    • cites [8] Luo, N. et al. Artificial intelligence-assisted dermatology diagnosis: from unimodal to multimodal. Comput. Biol. Med. https://doi.org/10.1016/j.compbiomed.2023.107413 (2023). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.

    Interpretationreasoning

    PanDerm serves as a model for developing multimodal foundation models in other medical specialties, potentially accelerating integration of AI support in healthcare.

    “These results show PanDerm’s potential to improve patient care across diverse clinical scenarios and serve as a model for developing multimodal foundation models in other medical specialties, potentially accelerating the integration of artificial intelligence support in healthcare.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Some efforts have focused on self-supervised learning specifically for dermatology using public datasets or web-sourced skin images.

    “Some efforts have focused on self-supervised learning specifically for dermatology using public datasets33,34 or web-sourced skin images35.”
    Quote in the paperNot a substantial claim0 of 3 sources checked
    • cites [33] Wang, Z., Lyu, J. & Tang, X. autoSMIM: automatic superpixel-based masked image modeling for skin lesion segmentation. IEEE Trans. Med. Imaging 42, 3501–3511 (2023). source

      Not checked in the free run.

    • cites [34] Tschandl, P., Rosendahl, C. & Kittler, H. The HAM10000 dataset, a large collection of multi-source dermatoscopic images of common pigmented skin lesions. Sci. Data 5, 180161 (2018). source

      Not checked in the free run.

    • cites [35] Shen, Y. et al. Optimizing skin disease diagnosis: harnessing online community data with contrastive learning and clustering techniques. NPJ Digit. Med. 7, 28 (2024). source

      Not checked in the free run.

  • Classified as probably not a substantial claim. May combine two claims.

    Premisecitation

    In general computer vision, foundation models are pretrained on massive datasets such as ImageNet or JFT-300M and most existing dermatology AI models still rely on these models for downstream adaptation.

    “In general computer vision, foundation models are pretrained on massive datasets such as ImageNet31 or JFT-300M (ref. 32) and most existing dermatology AI models still rely on these models for downstream adaptation.”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [31] Russakovsky, O. et al. ImageNet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2015). source

      Not checked in the free run.

    • cites [32] Sun, C., Shrivastava, A., Singh, S. & Gupta, A. Revisiting unreasonable effectiveness of data in deep learning era. In Proc. IEEE International Conference on Computer Vision 843–852 (IEEE, 2017). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    Foundation models have shown impressive performance across medical fields such as ophthalmology, radiology and pathology.

    “Built on rich knowledge representations, these models have shown impressive performance across medical fields such as ophthalmology20, radiology21 and pathology22–25.”
    Quote in the paperNot a substantial claim0 of 6 sources checked
    • cites [20] Zhou, Y. et al. A foundation model for generalizable disease detection from retinal images. Nature 622, 156–163 (2023). source

      Not checked in the free run.

    • cites [21] Pai, S. et al. Foundation model for cancer imaging biomarkers. Nat. Mach. Intell. 6, 354–367 (2024). source

      Not checked in the free run.

    • cites [22] Xu, H. et al. A whole-slide foundation model for digital pathology from real-world data. Nature 630, 181–188 (2024). source

      Not checked in the free run.

    • cites [23] Chen, R. J. et al. Towards a general-purpose foundation model for computational pathology. Nat. Med. 30, 850–862 (2024). source

      Not checked in the free run.

    • cites [24] Vorontsov, E. et al. A foundation model for clinical-grade computational pathology and rare cancers detection. Nat. Med. 30, 2949–2935 (2024). source

      Not checked in the free run.

    • cites [25] Wang, X. et al. A pathology foundation model for cancer diagnosis and prognosis prediction. Nature 634, 970–978 (2024). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    The performance of foundation models is inherently linked to the scale of their parameters and training data.

    “The performance of foundation models is inherently linked to the scale of their parameters and training data28–30.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [28] Achiam, J. et al. Gpt-4 technical report. Preprint at https://arxiv.org/abs/2303.08774 (2023).

      Not checked in the free run.

    • cites [29] Zhai, X., Kolesnikov, A., Houlsby, N. & Beyer, L. Scaling vision transformers. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition 12104–12113 (IEEE, 2022). source

      Not checked in the free run.

    • cites [30] Kaplan, J. et al. Scaling laws for neural language models. Preprint at https://arxiv.org/abs/2001.08361 (2020).

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Monitoring suspicious melanocytic lesions over a 3-month period is a widely accepted procedure for early melanoma detection.

    “Monitoring suspicious melanocytic lesions over a 3-month period is a widely accepted procedure for early melanoma detection, as changes often prompt excision to rule out melanoma, while stability can be reassuring12.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [12] Altamura, D., Avramidis, M. & Menzies, S. W. Assessment of the optimal interval for and sensitivity of short-term sequential digital dermoscopy monitoring for the diagnosis of melanoma. Arch. Dermatol. 144, 502–506 (2008). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The evaluation mainly reflects overall accuracy rather than disease-specific analysis.

    “the evaluation mainly reflects overall accuracy rather than disease-specific analysis”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

    Interpretationreasoning

    These findings suggest promising directions for developing foundation models in other medical specialties in which the integration of diverse imaging modalities and complex clinical workflows is crucial for patient care.

    “These findings suggest promising directions for developing foundation models in other medical specialties in which the integration of diverse imaging modalities and complex clinical workflows is crucial for patient care.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Interpretationreasoning

    The development approach provides a framework for developing medical AI systems that can adapt to varying levels of clinical expertise and healthcare settings.

    “Our development approach, combining systematic data curation, advanced self-supervised learning and rigorous clinical validation, provides a framework for developing medical AI systems that can adapt to varying levels of clinical expertise and healthcare settings.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationreasoning

    These capabilities stem from its rich knowledge representation, developed through pretraining on varied dermatological image modalities and conditions, leading to consistent and significant performance improvements across tasks and modalities.

    “These capabilities stem from its rich knowledge representation, developed through pretraining on varied dermatological image modalities and conditions, leading to consistent and significant performance improvements across tasks and modalities.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Specialty-specific foundation models designed with clinical workflows in mind may offer more practical solutions for specialties in which multiple imaging modalities are crucial.

    “our results suggest that specialty-specific foundation models designed with clinical workflows in mind may offer more practical solutions for specialties in which multiple imaging modalities are crucial.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductreasoning

    Our strategy minimizes the risk of data leakage, ensuring benchmark evaluations accurately reflect real-world model performance.

    “Our strategy minimizes this risk, ensuring that benchmark evaluations accurately reflect real-world model performance.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S2 (checkable).

    Interpretationreasoning

    Comprehensive bias assessment requires metrics beyond overall accuracy.

    “comprehensive bias assessment requires metrics beyond overall accuracy.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Premisecitation

    This approach contrasts with previous efforts, such as SwAVDerm, which relied on web-sourced skin data, inadvertently incorporating images from commonly used benchmarks such as ISIC and DermNet, increasing risk of data leakage and compromising evaluation validity.

    “This approach contrasts with previous efforts, such as SwAVDerm35, which relied on web-sourced skin data, inadvertently incorporating images from commonly used benchmarks such as ISIC55 and DermNet44, increasing the risk of data leakage and compromising evaluation validity.”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [35] Shen, Y. et al. Optimizing skin disease diagnosis: harnessing online community data with contrastive learning and clustering techniques. NPJ Digit. Med. 7, 28 (2024). source

      Not checked in the free run.

    • cites [55] Codella, N. C. et al. Skin lesion analysis toward melanoma detection: a challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC). In Proc. IEEE 15th International Symposium on Biomedical Imaging (eds Meijering, E. & Summers, R.) 168–172 (IEEE, 2018). source

      Not checked in the free run.

    • cites [44] Dermnet. Dermnet https://dermnet.com/ (2023).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The HIBA dataset contains 1,635 dermoscopic images with melanoma and other classes.

    “The HIBA55 dataset is curated from the HIBA data from the ISIC archive55, containing 1,635 dermoscopic images with melanoma and other classes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [55] Codella, N. C. et al. Skin lesion analysis toward melanoma detection: a challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC). In Proc. IEEE 15th International Symposium on Biomedical Imaging (eds Meijering, E. & Summers, R.) 168–172 (IEEE, 2018). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    The HOP study received approval from Metro South Health HREC and University of Queensland HREC.

    “The HOP study has received approval from the Human Research Ethics Committee (HREC) from Metro South Health HREC (HREC/17/QPAH/816) and the University of Queensland HREC (2018000074).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The NSSI dataset study followed the Declaration of Helsinki protocols and was approved by the Princess Alexandra Hospital human research ethics committee.

    “The naevus surveillance study images (NSSI) dataset is part of the Brisbane Naevus Morphology Study, circa 2009–2014. The study followed the Declaration of Helsinki protocols and was approved by the Princess Alexandra Hospital human research ethics committee.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    To compare models, the study implemented pairwise permutation tests, conducting 1,000 permutations per pair and recalculating performance metrics after each permutation.

    “To compare models, we implement pairwise permutation tests, conducting 1,000 permutations per pair and recalculating performance metrics after each permutation.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The study performed t-tests to assess the statistical significance of inter-model performance variations.

    “In addition, we perform t-tests to assess the statistical significance of inter-model performance variations.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The DDI dataset contains 647 clinical images from the United States.

    “The dataset contains 647 clinical images from the United States.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [63] Daneshjou, R. et al. Disparities in dermatology AI performance on a diverse, curated clinical image set. Sci. Adv. 8, eabq6147 (2022). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The Med-Node dataset contains 170 clinical images.

    “The Med-Node82 dataset contains 170 clinical images.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [82] Giotis, I. et al. MED-NODE: a computer-assisted melanoma diagnosis system using non-dermoscopic images. Expert Syst. Appl. 42, 6578–6585 (2015). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    We ensure robust evaluation through fivefold cross-validation, stratifying by both case and label attributes.

    “We ensure robust evaluation through fivefold cross-validation, stratifying by both case and label attributes.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The BCN20000 dataset comprises 12,413 dermoscopic images in 9 categories.

    “The BCN20000 (ref. 79) dataset comprises 12,413 dermoscopic images in 9 categories: nevus, melanoma, basal cell carcinoma, seborrheic keratosis, actinic keratosis, solar lentigo, squamous cell carcinoma, dermatofibroma and vascular lesions, including lesions in hard-to-diagnose locations.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites ref. 79

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Fitzpatrick17K comprises 16,577 clinical images annotated with dermatological diagnoses and Fitzpatrick skin types, encompassing 114 distinct conditions.

    “The Fitzpatrick17K (ref. 62) dataset comprises 16,577 clinical images annotated with both dermatological diagnoses and Fitzpatrick skin types (I–VI). It encompasses 114 distinct conditions”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites ref. 62

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The HAM10000 dataset contains 10,015 dermoscopic images across 7 classes.

    “The HAM10000 (ref. 34) dataset contains 10,015 dermoscopic images across 7 classes: actinic keratoses, basal cell carcinoma, benign keratosis, dermatofibroma, melanocytic nevi, melanoma and vascular lesions.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites ref. 34

      Not checked in the free run.

  • May combine two claims.

    Interpretationreasoning

    CLIP-large outperformed biomedical-specific and dermatology-specific CLIP models probably owing to the limited data scale of skin images in medical-domain CLIP models.

    “probably owing to the limited data scale of skin images in medical-domain CLIP models.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductreasoning

    For skin tumor patch classification, melanoma slide classification, reader studies, metastasis prediction and skin lesion segmentation, the study conducted k-fold cross-validation.

    “For skin tumor patch classification, melanoma slide classification, reader studies, metastasis prediction and skin lesion segmentation, we conduct k-fold cross-validation owing to either a relatively small sample size or following conventional practice.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The study conducted two-sided t-tests comparing PanDerm's performance against the baseline model for each task.

    “To assess statistical significance, we conduct two-sided t-tests comparing PanDerm’s performance against the baseline model for each task.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The SDDI2 dataset has been approved by the Ethics Review Board of the Medical University of Vienna.

    “The SDDI2 dataset has been approved by the Ethics Review Board of the Medical University of Vienna.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductown data

    The ACEMID pathology pilot study received approval from the Alfred Hospital Ethics Committee.

    “The ACEMID pathology (ACEMID_path) pilot study has received approval from the Alfred Hospital Ethics Committee (approval number: 746/23)”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Premisecitation

    PH2 is a clinical image dataset from Portugal with 200 images and 3 classes.

    “PH2 (ref. 81) is a clinical image dataset from Portugal with 200 images and 3 classes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites ref. 81

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductown data

    Four experienced dermatologists collaboratively developed a standard ontology to systematically categorize the 128 skin conditions.

    “Before the evaluation, four experienced dermatologists collaboratively developed a standard ontology to systematically categorize the 128 skin conditions and facilitate expert evaluation (Extended Data Fig. 8).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To ensure a fair comparison, all images are resized to 224 × 224.

    “To ensure a fair comparison, all images are resized to 224 × 224.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    For the remaining datasets, the study used nonparametric bootstrapping with 1,000 replicates to estimate 95% CIs for each model's performance.

    “For the remaining datasets, we use nonparametric bootstrapping with 1,000 replicates to estimate 95% CIs for each model’s performance.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Only de-identified retrospective data were used for research, without the active involvement of patients.

    “Only de-identified retrospective data were used for research, without the active involvement of patients.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    We used identical hyperparameter setups to ensure a fair comparison.

    “Subsequently, we fine-tuned these models using identical hyperparameter setups to ensure a fair comparison.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The ComBineMel dataset study was approved by the Alfred Hospital Ethics Committee on 8 August 2023.

    “The ComBineMel dataset is part of the Computer Biomarkers Evaluation of Invasive Melanoma (ComBine Mel) study. The study was approved by the Alfred Hospital Ethics Committee on 8 August 2023 (approval number: HREC/98200/Alfred-2023).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To mitigate bias, we included ten single time-point melanoma images, preventing reviewers from assuming that the first image in a series was benign.

    “To mitigate bias, we included ten single time-point melanoma images, preventing reviewers from assuming that the first image in a series was benign.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The melanoma slide classification task from CPTAC-CM consists of histopathology WSIs for cancer detection.

    “The melanoma slide classification task83 from the National Cancer Institute’s Clinical Proteomic Tumor Analysis Consortium Cutaneous Melanoma (CPTAC-CM) cohort consists of histopathology WSIs for cancer detection.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [83] Clark, K. et al. The Cancer Imaging Archive (TCIA): maintaining and operating a public information repository. J. Digit. Imaging 26, 1045–1057 (2013). source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    To enhance robustness and mitigate potential data selection bias, we perform five iterations of dataset splitting into training and testing sets, stratified by melanoma stage.

    “To enhance the robustness and mitigate potential data selection bias, we perform five iterations of dataset splitting into training and testing sets, stratified by melanoma stage.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    DermNet contains 19,559 clinical images of 23 types of skin diseases.

    “DermNet44 contains 19,559 clinical images; this dataset consists of images of 23 types of skin diseases and captures common clinical presentations including inflammatory conditions (eczema, psoriasis), infections (bacterial, viral, fungal) and neoplastic diseases.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [44] Dermnet. Dermnet https://dermnet.com/ (2023).

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The ISIC2018 dataset comprises 3,694 dermoscopic images with 2,594 images for training, 100 for validation and 1,000 for testing.

    “The ISIC2018 dataset52 comprises 3,694 dermoscopic images with 2,594 images for training, 100 for validation and 1,000 for testing.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [52] Milton, M. A. A. Automated skin lesion classification using ensemble of deep neural networks in ISIC 2018: skin lesion analysis towards melanoma detection challenge. Preprint at https://arxiv.org/abs/1901.10802 (2019).

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The MSKCC dataset contains 8,984 dermoscopic images with melanoma and other classes.

    “The Memorial Sloan Kettering Cancer Center (MSKCC)55 dataset is curated from the MSKCC data from the ISIC archive55, containing 8,984 dermoscopic images with melanoma and other classes.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [55] Codella, N. C. et al. Skin lesion analysis toward melanoma detection: a challenge at the 2017 International Symposium on Biomedical Imaging (ISBI), hosted by the International Skin Imaging Collaboration (ISIC). In Proc. IEEE 15th International Symposium on Biomedical Imaging (eds Meijering, E. & Summers, R.) 168–172 (IEEE, 2018). source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The MMT data study was approved through the Monash University Human Research Ethics Committee.

    “The MMT data study is part of a research agreement study with Monash eResearch Centre and was approved through the Monash University Human Research Ethics Committee.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Study design and conductcitation

    The study used the clean version of HAM10000, which prevents data leakage by ensuring that lesions from the same patient are not split across sets.

    “All other experiments used the clean version from a previous study78, which prevents data leakage by ensuring that lesions from the same patient are not split across sets.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites previous study78

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Premisecitation

    The PAD-UFES-20 dataset contains 2,298 close-up clinical images with 6 classes.

    “The PAD-UFES-20 (ref. 43) dataset from Brazil contains 2,298 close-up clinical images with 6 classes, including actinic keratosis, basal cell carcinoma of the skin, malignant melanoma, melanocytic nevus of the skin, squamous cell carcinoma and seborrheic keratosis.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites ref. 43

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    The MYM study was approved by the Metro South Health Human Research Ethics Committee on 21 April 2016 (approval number: HREC/16/QPAH/125).

    “The MYM study was approved by the Metro South Health Human Research Ethics Committee on 21 April 2016 (approval number: HREC/16/QPAH/125).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductown data

    The SDDI_Alfred study received approval from the Alfred Hospital Ethics Committee.

    “The SDDI_Alfred study has received approval from the Alfred Hospital Ethics Committee (approval number: 198/19) for the use of sequential dermoscopic imaging data.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim. May combine two claims.

    Study design and conductnone given

    To ensure robust assessment, each case was evaluated by three assessors, with cases showing significant scoring discordance reviewed in consensus meetings to establish final scores.

    “To ensure robust assessment, each case was evaluated by three assessors, with cases showing significant scoring discordance (differences between 3/4 and 1/2) reviewed in consensus meetings to establish final scores.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    ISIC2024 is a multicenter dataset with skin lesion crops from TBP, and the holdout evaluation dataset contains 49,025 crop images from three institutions.

    “ISIC2024 (ref. 47) is a multicenter dataset with skin lesion crops from TBP. We chose holdout data with 49,025 crop images with three institutions (FNQH Cairns, Alfred Hospital, Melanoma Institute Australia) as the evaluation dataset.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites ref. 47

      Not checked in the free run.

80 of 80

  • The paper relies on this without support.

    Current AI models for dermatology struggle to integrate various data types and imaging modalities, reducing their utility in different real-world clinical settings.

    “These models struggle to integrate various data types and imaging modalities, reducing their utility in different real-world clinical settings.”

  • The paper relies on this without support.

    The absence of integrated AI solutions capable of supporting various clinical workflows currently hampers the practical impact of AI in dermatology.

    “The absence of integrated AI solutions capable of supporting these various workflows currently hampers the practical impact of AI in dermatology.”

  • The paper relies on this without support.

    Through comprehensive pretraining on large and diverse data, foundation models develop versatile representations that can effectively adapt to various clinical scenarios, outperforming previous deep learning models in downstream tasks.

    “Through comprehensive pretraining on large and diverse data, these models develop versatile representations that can effectively adapt to various clinical scenarios, outperforming previous deep learning models in downstream tasks.”

  • The paper relies on this without support.

    Existing dermatology self-supervised approaches are often limited by dataset size, diversity or lack of real patient data.

    “However, these approaches are often limited by dataset size, diversity or the lack of real patient data.”

  • The paper relies on this without support.

    General biomedical foundation models struggle with domain-specific data scarcity and integrating heterogeneous modalities for comprehensive clinical analysis.

    “while general biomedical models, despite their broad scope, struggle with domain-specific data scarcity and integrating heterogeneous modalities for comprehensive clinical analysis.”

  • The paper relies on this without support.

    Current AI systems are often restricted to isolated tasks and unable to address diverse demands of medical decision-making.

    “Current AI systems are often restricted to isolated tasks and are unable to address the diverse demands of medical decision-making.”

  • The paper relies on this without support.

    There are over 1,000 known dermatological conditions.

    “over 1,000 diagnoses”

  • The paper relies on this without support.

    Biopsy serves as the reference standard for melanoma diagnosis.

    “While biopsy serves as our reference standard”

  • The paper relies on this without support.

    The DermaChallenge platform user base mean score is a valid measure of experience.

    “The number of correct answers in the first iteration of these levels, normalized against the mean score of the entire DermaChallenge platform user base, served as a score of experience.”

  • The paper relies on this without support.

    The 128 skin conditions ontology is a valid standard for evaluating diagnostic accuracy.

    “four experienced dermatologists collaboratively developed a standard ontology to systematically categorize the 128 skin conditions”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Recent high-impact studies demonstrate the clinical utility of multimodal large language models in dermatology, combining visual and textual reasoning to improve diagnostic accuracy.

    “Recent high-impact studies, such as (5), demonstrate the clinical utility of multimodal large language models in dermatology, combining visual and textual reasoning to improve diagnostic accuracy.”

    From Revolutionizing dermatopathology using AI in skin diagnostics: scoping review (2026) · cites this paper as [5]

    The Abstract reports that PanDerm improved clinicians’ diagnostic accuracy, supporting the clinical-utility part of the claim. But the Main and Methods describe a multimodal vision foundation model trained on several types of skin images, not a multimodal large language model combining visual and textual reasoning. (LLM, conf=0.90)

    Experimental — accuracy not yet measured

  • Most public dermatology datasets are small, unimodal, and paired with heterogeneous labels and metadata, restricting effective model development.

    “Most publicly available dermatology datasets are unimodal, limited in size, and composed mainly of dermoscopic or clinical images 3,4,13,20,21,42-47 , often photographs, paired with heterogeneous metadata or categorical labels that are lacking expressive textual information 48 .”

    From Synthesized clinical notes enable training robust multimodal AI models from unimodal dermatology datasets (2026) · cites this paper as [4]

    The Main section (page not available) says dermatology-specific approaches using public datasets are often limited by dataset size or diversity, and that specialty-specific models typically focus on single imaging modalities. The Methods section describes image datasets with categorical diagnoses, but our copy does not establish that most public datasets have heterogeneous metadata or lack expressive textual information. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • This supports the findings of (5), especially in emphasizing the practical relevance and significance of multimodal inputs in enhancing diagnostic precision and clinical generalizability.

    “This supports the findings of (5), especially in emphasizing the practical relevance and significance of multimodal inputs in enhancing diagnostic precision and clinical generalizability.”

    From Revolutionizing dermatopathology using AI in skin diagnostics: scoping review (2026) · cites this paper as [5]

  • The development of foundation models is reshaping dermatology, with multiple models such as SkinGPT4, PanDerm, and Derm1M.

    “The development of foundation models is also reshaping the dermatology domain, with multiple models such as SkinGPT4 2 , PanDerm 4 , Derm1M 3 , paving the way for an increased use of such models in the domain.”

    From Synthesized clinical notes enable training robust multimodal AI models from unimodal dermatology datasets (2026) · cites this paper as [4]

  • Transformer-based methods support clinical text processing, whole-slide and radiologic image interpretation, and genomic sequence modeling in oncology.

    “In oncology, transformer-based methods support clinical text processing, whole-slide and radiologic image interpretation (vision transformers), and genomic sequence modeling [24,25,26,27].”

    From Artificial Intelligence in Clinical Oncology: From Productivity Enhancement to Creative Discovery (2025) · cites this paper as [26]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Patient-level malignant detection exceeds the number of malignant test lesions

    Major

    The paper says malignant lesions were detected in 79 of 80 patients, but specifies that those 80 test patients had only 28 malignant lesions. At most 28 patients could each have had a malignant lesion detected.

  • Generalist reader-group composition differs between Results and Methods

    Minor

    The Results count eight GPs and two readers in the nurse and assistant roles; the Methods count seven GPs and three other healthcare professionals.

2 of 2

Cites

Flagged first, then by citations

7 of the 85 references couldn’t be matched to a known paper.

Research works printed but not linked: 8

  1. Rapini, R. P. Practical Dermatopathology (Elsevier Health Sciences, 2021).
  2. Dosovitskiy, A. et al. An image is worth 16×16 words: transformers for image recognition at scale. In Proc. 9th International Conference on Learning Representations https://openreview.net/forum?id=YicbFdNTTy (OpenReview.net, 2021).
  3. Zhang, X. et al. Cae v2: context autoencoder with clip latent alignment. Trans. Mach. Learning Res. (2023).
  4. Alcantarilla, P. F. & Solutions, T. Fast explicit diffusion for accelerated features in nonlinear scale spaces. IEEE Trans. Pattern Anal. Mach. Intell. 34, 1281–1298 (2011).
  5. Ilse, M., Tomczak, J. & Welling, M. Attention-based deep multiple instance learning. In Proc. International Conference on Machine Learning (eds Dy, J. & Krause, A.) 2127–2136 (PMLR, 2018).
  6. Loshchilov, I. et al. Decoupled weight decay regularization. In Proc. 7th International Conference on Learning Representations https://openreview.net/forum?id=Bkg6RiCqY7 (OpenReview.net, 2019).
  7. Mendonça, T. et al. in Dermoscopy Image Analysis (eds Celebi, M. E. et al.) Ch. 13 420–438 (CRC Press, 2015).
  8. Kahler, S. et al. Automated photodamage assessment from 3D total body photography for an objective assessment of melanoma risk. Australas. J. Dermatol. 64, 8 (2023).

Cited by

Sorted by citations

Related

Authors
Yan, S., Yu, Z., Primiero, C., Vico-Alonso, C., Wang, Z., Yang, L., Tschandl, P., Hu, M., Ju, L., Tan, G., Tang, V., Ng, A. B., Powell, D., Bonnington, P., See, S., Magnaterra, E., Ferguson, P., Nguyen, J., Guitera, P., Banuls, J., Janda, M., Mar, V., Kittler, H., Soyer, H. P., & Ge, Z.
Affiliations
  • Monash University (AU)
  • The University of Queensland (AU)
  • Medical University of Vienna (AT)
  • Nvidia (United States) (US)
  • University of Florence (IT)
  • Royal Prince Alfred Hospital (AU)
  • The Alfred Hospital (AU)
Corresponding author
Ge, Zongyuan
ORCID
Venue
Nature Medicine
Publisher
Springer Science and Business Media LLC
Published
2025-06-06
Volume
31
Issue
8
Pages
2691-2702
ISSN
  • 1078-8956 (ISSN-L)
  • 1546-170X
DOI
10.1038/s41591-025-03747-y
PMC
PMC12353815
OpenAlex
W4411099958
Semantic Scholar
38c437ccb98cda6fe3b7aa8a6351df8c87ea5472
Cited by
117
References
85
Citations by year
2026: 85 · 2025: 32
FWCI
48.05
Type
journal article
Language
en
Open access
Licence
Locations
  • Nature Medicine (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • Archivio istituzionale della ricerca (Alma Mater Studiorum Università di Bologna) (submittedVersion, other-oa)
  • arXiv (Cornell University)
Topics
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
multimodal foundation models, skin diseases, skin cancer diagnosis, melanoma detection, dermoscopy, self-supervised pre-training, differential diagnosis, lesion segmentation, long-term monitoring, metastasis prediction, clinical photographs, reader study, clinical implementation of AI
MeSH terms
Deep Learning, Dermatology, Diagnosis, Differential, Humans, Melanoma, Skin Diseases, Skin Neoplasms, Dermoscopy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-07.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-07.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review