CoChat
Sign Up
Cited by 365Cutaneous Melanoma Detection and ManagementAI in cancer detectionmelanin and skin pigmentation

Association between surgical skin markings in dermoscopic images and diagnostic performance of a deep learning convolutional neural network for melanoma recognition

Open access
Winkler, Julia K., Fink, Christine, Toberer, Ferdinand, Enk, Alexander, Deinlein, Teresa, Hofmann-Wellenhof, Rainer, Thomas, Luc, Lallas, Aimilios +3 more·2019·JAMA Dermatology
DOI 10.1001/jamadermatol.2019.1735PMC PMC6694463
Open this paper in CoChat — free accountOpen PDF

Abstract

IMPORTANCE: Deep learning convolutional neural networks (CNNs) have shown a performance at the level of dermatologists in the diagnosis of melanoma. Accordingly, further exploring the potential limitations of CNN technology before broadly applying it is of special interest. OBJECTIVE: To investigate the association between gentian violet surgical skin markings in dermoscopic images and the diagnostic performance of a CNN approved for use as a medical device in the European market. DESIGN AND SETTING: A cross-sectional analysis was conducted from August 1, 2018, to November 30, 2018, using a CNN architecture trained with more than 120 000 dermoscopic images of skin neoplasms and corresponding diagnoses. The association of gentian violet skin markings in dermoscopic images with the performance of the CNN was investigated in 3 image sets of 130 melanocytic lesions each (107 benign nevi, 23 melanomas). EXPOSURES: The same lesions were sequentially imaged with and without the application of a gentian violet surgical skin marker and then evaluated by the CNN for their probability of being a melanoma. In addition, the markings were removed by manually cropping the dermoscopic images to focus on the melanocytic lesion. MAIN OUTCOMES AND MEASURES: Sensitivity, specificity, and area under the curve (AUC) of the receiver operating characteristic (ROC) curve for the CNN's diagnostic classification in unmarked, marked, and cropped images. RESULTS: In all, 130 melanocytic lesions (107 benign nevi and 23 melanomas) were imaged. In unmarked lesions, the CNN achieved a sensitivity of 95.7% (95% CI, 79%-99.2%) and a specificity of 84.1% (95% CI, 76.0%-89.8%). The ROC AUC was 0.969. In marked lesions, an increase in melanoma probability scores was observed that resulted in a sensitivity of 100% (95% CI, 85.7%-100%) and a significantly reduced specificity of 45.8% (95% CI, 36.7%-55.2%, P < .001). The ROC AUC was 0.922. Cropping images led to the highest sensitivity of 100% (95% CI, 85.7%-100%), specificity of 97.2% (95% CI, 92.1%-99.0%), and ROC AUC of 0.993. Heat maps created by vanilla gradient descent backpropagation indicated that the blue markings were associated with the increased false-positive rate. CONCLUSIONS AND RELEVANCE: This study's findings suggest that skin markings significantly interfered with the CNN's correct diagnosis of nevi by increasing the melanoma probability scores and consequently the false-positive rate. A predominance of skin markings in melanoma training images may have induced the CNN's association of markings with a melanoma diagnosis. Accordingly, these findings suggest that skin markings should be avoided in dermoscopic images intended for analysis by a CNN. TRIAL REGISTRATION: German Clinical Trial Register (DRKS) Identifier: DRKS00013570.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1001/jamadermatol.2019.1735

Cites

28 of 28 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Explanatory heat maps revealed that an AI model trained to detect skin cancer was focusing more on the surgical skin markings present on the images rather than the skin lesions.

    “An example of this use was when explanatory heat maps revealed that an AI model trained to detect skin cancer was focusing more on the surgical skin markings present on the images rather than the skin lesions.42”

    From The false hope of current approaches to explainable artificial intelligence in health care (2021) · cites this paper as [42]

  • An artificial intelligence system might learn to detect surgical skin markings to diagnose skin cancer, rather than looking for features related to the lesion itself.

    “For example, an artificial intelligence system might learn to detect surgical skin markings to diagnose skin cancer, rather than looking for features related to the lesion itself.9”

    From The medical algorithmic audit (2022) · cites this paper as [9]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 4

  1. [17]17. Sun J, Binder A. Comparison of deep learning architectures for H&E histopathology images. Paper presented at: IEEE Conference on Big Data and Analytics (ICBDA); November 16, 2017; Kuching, Malaysia. https://ieeexplore.ieee.org/document/ 8284105. Accessed November 7, 2018.
  2. [26]26. Mishra NK, Celebi ME. An overview of melanoma detection in dermoscopy images using image processing and machine learning. https://arxiv.org/ftp/arxiv/papers/1601/1601. 07843.pdf. Published January 27, 2016. Accessed November 7, 2018.
  3. [28]28. Salido JAA, Ruiz C Jr. Using deep learning to detect melanoma in dermoscopy images. Int J Mach Learn Comput. 2018;8(1):61-68.
  4. [30]30. Sultana NN, Puhan N. Recent deep learning methods for melanoma detection: a review. In:

Cited by

Sorted by citations

Related

Authors
Winkler, J. K., Fink, C., Toberer, F., Enk, A., Deinlein, T., Hofmann-Wellenhof, R., Thomas, L., Lallas, A., Blum, A., Stolz, W., & Haenssle, H. A.
Affiliations
  • Heidelberg University (DE)
  • Medical University of Graz (AT)
  • Department of Dermatology and Venerology, Medical University of Graz, Graz, Austria
  • Hospices Civils de Lyon (FR)
  • Aristotle University of Thessaloniki (GR)
  • Public, Private, and Teaching Practice, Konstanz, Germany
  • München Klinik (DE)
Corresponding author
Haenssle, Holger A.
ORCID
Venue
JAMA Dermatology
Publisher
American Medical Association (AMA)
Published
2019-10-01
Volume
155
Issue
10
Pages
1135
ISSN
2168-6068 (ISSN-L)
DOI
10.1001/jamadermatol.2019.1735
PMC
PMC6694463
OpenAlex
W2969096242
Semantic Scholar
68d1001637ed1c78fc1cc0b7d54fdc15c05eb004
Cited by
365
References
27
Citations by year
2026: 36 · 2025: 53 · 2024: 55 · 2023: 58 · 2022: 44 · 2021: 64 · 2020: 48 · 2019: 7
FWCI
17.03
Type
journal article
Language
en
Open access
Licence
other-oa
Locations
  • JAMA Dermatology (publishedVersion, other-oa)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Life Sciences › Biochemistry, Genetics and Molecular Biology › Cell Biology › melanin and skin pigmentation
Keywords
melanoma, dermoscopy, convolutional neural networks, crystal violet, image cropping, diagnostic accuracy, benign nevi, computer-aided diagnosis, false positives, dermatologist comparison, skin lesion classification, medical device validation, skin cancer screening, melanocytic lesions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review