CoChat

A pretrained neural network shows similar diagnostic accuracy to medical students in categorizing dermatoscopic images after comparable training conditions

Tschandl, P., Kittler, H., Argenziano, G. · 2017 · British Journal of Dermatology
Cited by 25

The DOI matches its Crossref record; 1 of the 1 references it cites were found in a registry or an index.

Abstract

Dear Editor, Automated image classification by machine vision is evolving, largely because of the increasing efficiency of neural networks with special architectures such as convolutional neural networks (CNNs).1 A recent study comparing board‐certified dermatologists with CNNs suggested that CNNs may achieve diagnostic accuracies similar to those of human experts.2 We demonstrated that medical students without prior knowledge of dermatoscopy learn equally well from an analytical or heuristic teaching approach.3 During heuristic teaching, representative images of each class are presented together with the correct diagnosis without any further explanations, mimicking training of a neural network. In this experiment we trained a CNN (pretrained for general‐purpose image classification) and a group of previously untrained medical students to diagnose pigmented skin lesions on dermatoscopic images, and compared their diagnostic performance after training. We invited final‐year medical students without prior knowledge of dermatoscopy to participate in a 1‐h training session.3 During this training, the students saw dermatoscopic images of 298 pigmented lesions: 40 basal cell carcinomas (BCCs), seven dermatofibromas, 62 melanomas, 129 melanocytic naevi, 38 seborrhoeic keratoses and 22 vascular lesions. The instructor mentioned the diagnosis but did not explain any diagnostic features. After the training session the students had to rate a test set of 50 randomly selected pigmented lesions including 10 melanomas, 10 BCCs, 14 naevi, nine seborrhoeic keratoses and five vascular lesions or dermatofibromas. The students were asked to make a specific diagnosis and to indicate the probability of malignancy (scale from 1 to 5). No additional information (age, site or medical history) was provided. All images from the students' training session were also used to retrain the last layer of the ‘GoogLeNet Inception v3' neural network,4 without any kind of test‐set augmentation (4000 epochs, learning rate 0·001, batch size 50). We used the retrained network on the same test set as the students. The diagnosis was considered to indicate a malignant lesion if the top class of the softmax outputs was BCC or melanoma. The softmax outputs indicate probabilities for each class and were used to calculate receiver operating characteristics (ROC) curves. We calculated the areas under the curves (AUCs) and their confidence intervals (CIs) with the package pROC,5 and used epiR (in R, https://www.r-project.org) for the calculation of diagnostic values. We assessed the inter‐rater agreement by calculating Cohen's kappa for each class and each reader pair using the irr package in R. The study was approved by the institutional review board of the Medical University of Vienna (no. 1285/2014). Twenty‐seven students (mean age 24·9 ± 2·3 years, 74% female) completed the training session and evaluated the test set. The retrained neural network (R‐NN) correctly identified malignant lesions (BCC and melanoma) with a sensitivity of 90% (95% CI 68–99) and a specificity of 71% (95% CI 51–87). The sensitivity was slightly higher than that achieved by the students (mean 86%, 95% CI 83–88) but the R‐NN specificity was slightly lower than that of the students (mean 79%, 95% CI 74–83). For the R‐NN, the AUC for correctly predicting malignancy was 0·91 (95% CI 0·81–0·97), which was similar to the AUC of the pooled student ratings of 0·85 (95% CI 0·87–0·91, P = 0·64; Fig. 1a). (a) Diagnostic accuracy. The retrained neural network (R‐NN) and students (pooled ratings) showed similar diagnostic accuracy in detecting skin cancer (basal cell carcinoma and melanoma) as measured by the areas under the receiver operating characteristic curves. (b) Misclassified lesions. Correct (black) and incorrect (red) specific diagnoses of every case (rows 1–48) by every student (columns 1–37) and the R‐NN, and the mean correct ratings of the students. A striking similarity is seen between misdiagnosed lesions of the students and the R‐NN. The R‐NN made a correct specific diagnosis in 69% of lesions. Two of 10 melanomas were misclassified as naevi and two as BCC. Cases misdiagnosed by the R‐NN were also misdiagnosed by 46% of students. Of the 13 cases that were misdiagnosed by more than half of the students, nine were also misclassified by the R‐NN (Fig. 1b). Agreement on specific diagnoses between the R‐NN and students was moderate (mean Cohen's kappa 0·57, 95% CI 0·52–0·61) and similar to the average agreement between students. We show that an R‐NN achieves a diagnostic accuracy similar to that of medical students under comparable training conditions. The neural network (GoogLeNet Inception v3)4 was pretrained on more than 1 million images for the purpose of general image classification but was not pretrained to diagnose skin cancer. During training, students and the R‐NN had to extract features that discriminate benign from malignant pigmented skin lesions. Given the small trai

Overview

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/bjd.15695

Cites

1 of 1 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 1 references

Cited by

See all 25

Metadata

Authors
Tschandl, P., Kittler, H., & Argenziano, G.
Venue
British Journal of Dermatology
Published
2017-07-19
Volume
177
Issue
3
Pages
867-869
ISSN
  • 0007-0963 (ISSN-L)
  • 1365-2133
OpenAlex
W2621161783
Cited by
25
References
1
FWCI
1.29
Type
journal article
Language
en
Open access
closed

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: A pretrained neural network shows similar diagnostic accuracy to medical students in categorizing dermatoscopic images after comparable training conditions. https://app.cochat.ai/papers/10.1111/bjd.15695
The paper
Tschandl, P., Kittler, H., & Argenziano, G. (2017). A pretrained neural network shows similar diagnostic accuracy to medical students in categorizing dermatoscopic images after comparable training conditions. British Journal of Dermatology. https://doi.org/10.1111/bjd.15695

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.