CoChat
Sign Up
Cited by 393Cutaneous Melanoma Detection and ManagementAI in cancer detectionArtificial Intelligence in Healthcare and Education

Diagnostic Inaccuracy of Smartphone Applications for Melanoma Detection

Open access
Wolf, Joel A., Moreau, Jacqueline F., Akilov, Oleg, Patton, Timothy, English, Joseph C., Ho, Jonhan, Ferris, Laura K.·2013·JAMA Dermatology
DOI 10.1001/jamadermatol.2013.2382PMC PMC4019431
Open this paper in CoChat — free account

Abstract

OBJECTIVE: To measure the performance of smartphone applications that evaluate photographs of skin lesions and provide the user with feedback about the likelihood of malignancy. DESIGN: Case-control diagnostic accuracy study. SETTING: Academic dermatology department. PARTICIPANTS AND MATERIALS: Digital clinical images of pigmented cutaneous lesions (60 melanoma and 128 benign control lesions) with a histologic diagnosis rendered by a board-certified dermatopathologist, obtained before biopsy from patients undergoing lesion removal as a part of routine care. MAIN OUTCOME MEASURES: Sensitivity, specificity, and positive and negative predictive values of 4 smartphone applications designed to aid nonclinician users in determining whether their skin lesion is benign or malignant. RESULTS: Sensitivity of the 4 tested applications ranged from 6.8% to 98.1%; specificity, 30.4% to 93.7%; positive predictive value, 33.3% to 42.1%; and negative predictive value, 65.4% to 97.0%. The highest sensitivity for melanoma diagnosis was observed for an application that sends the image directly to a board-certified dermatologist for analysis; the lowest, for applications that use automated algorithms to analyze images. CONCLUSIONS: The performance of smartphone applications in assessing melanoma risk is highly variable, and 3 of 4 smartphone applications incorrectly classified 30% or more of melanomas as unconcerning. Reliance on these applications, which are not subject to regulatory oversight, in lieu of medical consultation can delay the diagnosis of melanoma and harm users.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamadermatol.2013.2382

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Automated smartphone apps have low sensitivity, ranging from 7% to 87% with a median of 69%.

    “Regarding automated smartphone apps, the major concerns are the lack of assessment in clinical practice conditions, the lack of assessment in primary care populations, and their low sensitivity, ranging from 7% to 87% (median 69%).”

    From Use of Smartphones for Early Detection of Melanoma: Systematic Review (2018) · cites this paper as [49]

    The Results section reports a low sensitivity of 6.8% and says the highest sensitivity, 98.1%, belonged to the dermatologist-reviewed app rather than an automated app. It supports the concern about automated apps, but our copy does not state an automated-app range ending at 87% or a median of 69%. No page number is available in the indexed copy. (LLM, conf=0.83)

    Experimental — accuracy not yet measured

  • Across the body of evidence, different apps recommended conflicting management advice for the same lesions.

    “Across the body of evidence presented, different apps recommended conflicting management advice for the same lesions.23 34”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [34]

  • Studies were based on images of suspicious skin lesions that had undergone excision or biopsy, and were further selected to only include conditions identified by the apps; for example, they excluded lesions with clinical and histological features similar to melanoma, or restricted inclusion to melanocytic lesions which are more likely to be recognised by apps.

    “We found studies were based on images of suspicious skin lesions that had undergone excision or biopsy, and were further selected to only include conditions identified by the apps; for example, they excluded lesions with clinical and histological features similar to melanoma,34 or restricted inclusion to melanocytic lesions which are more likely to be recognised by apps.23 28 29 31 32”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [34]

  • The numbers of images excluded for each analysis because of test failure ranged from 3/188 (1.6%) to 90/199 (45.2%).

    “Table 2, table 3, and figure 2 report the numbers excluded for each analysis because of test failure, which ranged from 3/188 (1.6%)34 to 90/199 (45.2%).30”

    From Algorithm based smartphone apps to assess risk of skin cancer in adults: systematic review of diagnostic accuracy studies (2020) · cites this paper as [34]

  • A recent comparison of the accuracy of four smartphone apps in assessing melanoma risk demonstrated wide variation in performance and utility.

    “However, a recent comparison of the accuracy of four smartphone apps in assessing melanoma risk demonstrated wide variation in performance and utility.7”

    From Smartphone applications for melanoma detection by community, patient and generalist clinician users: a review (2015) · cites this paper as [7]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Wolf, J. A., Moreau, J. F., Akilov, O., Patton, T., English, J. C., Ho, J., & Ferris, L. K.
Affiliations
University of Pittsburgh (US)
ORCID
Venue
JAMA Dermatology
Publisher
American Medical Association (AMA)
Published
2013-04-01
Volume
149
Issue
4
Pages
422
ISSN
2168-6068 (ISSN-L)
DOI
10.1001/jamadermatol.2013.2382
PMC
PMC4019431
OpenAlex
W2101764630
Cited by
393
References
9
Citations by year
2026: 6 · 2025: 4 · 2024: 9 · 2023: 12 · 2022: 21 · 2021: 19 · 2020: 38 · 2019: 41 · 2018: 45 · 2017: 32 · 2016: 43 · 2015: 53 · 2014: 43 · 2013: 27
FWCI
29.61
Type
journal article
Language
en
Open access
Locations
  • JAMA Dermatology (publishedVersion)
  • PubMed (publishedVersion)
  • Europe PMC (PubMed Central) (acceptedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Institutes of Health (UL1TR000005)
  • National Center for Research Resources (UL1RR024153)
  • National Center for Advancing Translational Sciences
Topics
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
melanoma, mobile application, melanoma detection, diagnostic accuracy, pigmented skin lesions, automated image analysis, telemedicine
MeSH terms
Algorithms, Diagnosis, Differential, Diagnostic Errors, Humans, Image Interpretation, Computer-Assisted, Melanoma, ROC Curve, Skin Neoplasms, Software, User-Computer Interface, Video Recording, Telemedicine, Computers, Handheld, Electronic Mail, Cell Phone, Dermoscopy

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review