CoChat
Sign Up
Cited by 96Cutaneous Melanoma Detection and ManagementAI in cancer detectionArtificial Intelligence in Healthcare and Education

Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms

Udrea, A., Mitra, G.D., Costea, D., Noels, E.C., Wakkee, M., Siegel, D.M., de Carvalho, T.M., Nijsten, T.E.C.·2020·Journal of the European Academy of Dermatology and Venereology
DOI 10.1111/jdv.15935
Open this paper in CoChat — free account

Abstract

Background: Machine learning algorithms achieve expert‐level accuracy in skin lesion classification based on clinical images. However, it is not yet shown whether these algorithms could have high accuracy when embedded in a smartphone app, where image quality is lower and there is high variability in image taking scenarios by users. In the past, these applications were criticized due to lack of accuracy. Objective: In this study, we evaluate the accuracy of the newest version of a smartphone application ( SA ) for risk assessment of skin lesions. Methods: This SA uses a machine learning algorithm to compute a risk rating. The algorithm is trained on 131 873 images taken by 31 449 users in multiple countries between January 2016 and August 2018 and rated for risk by dermatologists. To evaluate the sensitivity of the algorithm, we use 285 histopathologically validated skin cancer cases (including 138 malignant melanomas), from two previously published clinical studies (195 cases) and from the SA user database (90 cases). We calculate the specificity on a separate set from the SA user database containing 6000 clinically validated benign cases. Results: The algorithm scored a 95.1% (95% CI , 91.9–97.3%) sensitivity in detecting (pre)malignant conditions (93% for malignant melanoma and 97% for keratinocyte carcinomas and precursors). This level of sensitivity was achieved with a 78.3% (95% CI , 77.2–79.3%) specificity. Conclusions: This SA provides a high sensitivity to detect skin cancer; however, there is still room for improvement in terms of specificity. Future studies are needed to assess the impact of this SA on the health systems and its users.

Integrity signals

Registry

Crossref · 10.1111/jdv.15935

Cites

24 of 24 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • It is possible that the group of 6000 apparently benign lesions might include some missed melanomas.

    “Therefore, it is possible that this group might include some missed melanomas.”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [41]

  • Another 90 melanomas in the SkinVision study were identified from users of the app who had uploaded histology results after a high risk rating by a dermatologist and a high or moderate risk recommendation from the app, which will overestimate accuracy if more easily identifiable melanomas were included.

    “Another 90 melanomas were identified from users of the app who had uploaded histology results after a high risk rating by a dermatologist and a high or moderate risk recommendation from the app (which will overestimate accuracy if more easily identifiable melanomas were included).41”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [41]

  • The SkinVision study used some images and data collected by real users of the app on their own phones, but the selection of malignant and benign lesions from several different sources is likely to have introduced bias.

    “The study has used some images and data collected by real users of the app on their own phones, however the selection of malignant and benign lesions from several different sources is likely to have introduced bias.”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [41]

  • A SkinVision study published after the search was conducted reported improved estimates of sensitivity for the detection of malignant or premalignant lesions for a new version of the app (95.1%, 95% CI 91.9% to 97.3%) with similar specificity (78.3%, 77.3% to 79.3%).

    “A SkinVision study41 published after our search was conducted reported improved estimates of sensitivity for the detection of malignant or premalignant lesions for a new version of the app (95.1%, 95% confidence interval 91.9% to 97.3%) with similar specificity (78.3%, 77.3% to 79.3%).”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [41]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Udrea, A., Mitra, G. D., Costea, D., Noels, E. C., Wakkee, M., Siegel, D. M., de Carvalho, T. M., & Nijsten, T. E. C.
Affiliations
  • Universitatea Națională de Știință și Tehnologie Politehnica București (RO)
  • SkinVision BV Amsterdam The Netherlands
  • Erasmus MC (NL)
  • United States Department of Veterans Affairs (US)
  • Department of Dermatology Erasmus Medical Center Rotterdam The Netherlands
Corresponding author
Udrea, A.
ORCID
Venue
Journal of the European Academy of Dermatology and Venereology
Publisher
Wiley
Published
2019-10-08
Volume
34
Issue
3
Pages
648-655
ISSN
  • 0926-9959 (ISSN-L)
  • 1468-3083
DOI
10.1111/jdv.15935
OpenAlex
W2972754417
Cited by
96
References
24
Citations by year
2026: 11 · 2025: 9 · 2024: 15 · 2023: 15 · 2022: 20 · 2021: 14 · 2020: 12
FWCI
4.79
Type
journal article
Language
en
Open access
closed
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • Journal of the European Academy of Dermatology and Venereology (publishedVersion)
  • PubMed (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (publishedVersion)
  • EUR Research Repository (Erasmus University Rotterdam) (submittedVersion)
Topics
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
mobile application, melanoma, non-melanoma skin cancer, image quality variability, sensitivity and specificity
MeSH terms
Smartphone, Machine Learning, Diagnosis, Differential, Humans, Melanoma, Sensitivity and Specificity, Skin Neoplasms, Reproducibility of Results, Risk Assessment, Mobile Applications

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review