Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat
Sign Up
Cited by 354

The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.

Superior skin cancer classification by the combination of human and artificial intelligence

Open access
Hekler, Achim, Utikal, Jochen S., Enk, Alexander H., Hauschild, Axel, Weichenthal, Michael, Maron, Roman C., Berking, Carola, Haferkamp, Sebastian, Klode, Joachim, Schadendorf, Dirk, Schilling, Bastian, Holland-Letz, Tim, Izar, Benjamin, von Kalle, Christof, Fröhling, Stefan, Brinker, Titus J., Schmitt, Laurenz, Peitsch, Wiebke K., Hoffmann, Friederike, Becker, Jürgen C., Drusio, Christina, Jansen, Philipp, Klode, Joachim, Lodde, Georg, Sammet, Stefanie, Schadendorf, Dirk, Sondermann, Wiebke, Ugurel, Selma, Zader, Jeannine, Enk, Alexander, Salzmann, Martin, Schäfer, Sarah, Schäkel, Knut, Winkler, Julia, Wölbing, Priscilla, Asper, Hiba, Bohne, Ann-Sophie, Brown, Victoria, Burba, Bianca, Deffaa, Sophia, Dietrich, Cecilia, Dietrich, Matthias, Drerup, Katharina Antonia, Egberts, Friederike, Erkens, Anna-Sophie, Greven, Salim, Harde, Viola, Jost, Marion, Kaeding, Merit, Kosova, Katharina, Lischner, Stephan, Maagk, Maria, Messinger, Anna Laetitia, Metzner, Malte, Motamedi, Rogina, Rosenthal, Ann-Christine, Seidl, Ulrich, Stemmermann, Jana, Torz, Kaspar, Velez, Juliana Giraldo, Haiduk, Jennifer, Alter, Mareike, Bär, Claudia, Bergenthal, Paul, Gerlach, Anne, Holtorf, Christian, Karoglan, Ante, Kindermann, Sophie, Kraas, Luise, Felcht, Moritz, Gaiser, Maria R., Klemke, Claus-Detlev, Kurzen, Hjalmar, Leibing, Thomas, Müller, Verena, Reinhard, Raphael R., Utikal, Jochen, Winter, Franziska, Berking, Carola, Eicher, Laurie, Hartmann, Daniela, Heppt, Markus, Kilian, Katharina, Krammer, Sebastian, Lill, Diana, Niesert, Anne-Charlotte, Oppel, Eva, Sattler, Elke, Senner, Sonja, Wallmichrath, Jens, Wolff, Hans, Gesierich, Anja, Giner, Tina, Glutsch, Valerie, Kerstan, Andreas, Presser, Dagmar, Schrüfer, Philipp, Schummer, Patrick, Stolze, Ina, Weber, Judith, Drexler, Konstantin, Haferkamp, Sebastian, Mickler, Marion, Stauner, Camila Toledo, Thiem, Alexander·2019·European Journal of Cancer
DOI 10.1016/j.ejca.2019.07.019
Open this paper in CoChat — free account

Abstract

BACKGROUND: In recent studies, convolutional neural networks (CNNs) outperformed dermatologists in distinguishing dermoscopic images of melanoma and nevi. In these studies, dermatologists and artificial intelligence were considered as opponents. However, the combination of classifiers frequently yields superior results, both in machine learning and among humans. In this study, we investigated the potential benefit of combining human and artificial intelligence for skin cancer classification. METHODS: Using 11,444 dermoscopic images, which were divided into five diagnostic categories, novel deep learning techniques were used to train a single CNN. Then, both 112 dermatologists of 13 German university hospitals and the trained CNN independently classified a set of 300 biopsy-verified skin lesions into those five classes. Taking into account the certainty of the decisions, the two independently determined diagnoses were combined to a new classifier with the help of a gradient boosting method. The primary end-point of the study was the correct classification of the images into five designated categories, whereas the secondary end-point was the correct classification of lesions as either benign or malignant (binary classification). FINDINGS: Regarding the multiclass task, the combination of man and machine achieved an accuracy of 82.95%. This was 1.36% higher than the best of the two individual classifiers (81.59% achieved by the CNN). Owing to the class imbalance in the binary problem, sensitivity, but not accuracy, was examined and demonstrated to be superior (89%) to the best individual classifier (CNN with 86.1%). The specificity in the combined classifier decreased from 89.2% to 84%. However, at an equal sensitivity of 89%, the CNN achieved a specificity of only 81.5% INTERPRETATION: Our findings indicate that the combination of human and artificial intelligence achieves superior results over the independent results of both of these systems.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.ejca.2019.07.019

Cites

9 of 9 references verified0 citation errors · 0 not checkable

Statistics

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 9 references

Cited by

Metadata

Authors
Hekler, A., Utikal, J. S., Enk, A. H., Hauschild, A., Weichenthal, M., Maron, R. C., Berking, C., Haferkamp, S., Klode, J., Schadendorf, D., Schilling, B., Holland-Letz, T., Izar, B., von Kalle, C., Fröhling, S., Brinker, T. J., Schmitt, L., Peitsch, W. K., Hoffmann, F., Becker, J. C., Drusio, C., Jansen, P., Klode, J., Lodde, G., Sammet, S., Schadendorf, D., Sondermann, W., Ugurel, S., Zader, J., Enk, A., Salzmann, M., Schäfer, S., Schäkel, K., Winkler, J., Wölbing, P., Asper, H., Bohne, A.-S., Brown, V., Burba, B., Deffaa, S., Dietrich, C., Dietrich, M., Drerup, K. A., Egberts, F., Erkens, A.-S., Greven, S., Harde, V., Jost, M., Kaeding, M., Kosova, K., Lischner, S., Maagk, M., Messinger, A. L., Metzner, M., Motamedi, R., Rosenthal, A.-C., Seidl, U., Stemmermann, J., Torz, K., Velez, J. G., Haiduk, J., Alter, M., Bär, C., Bergenthal, P., Gerlach, A., Holtorf, C., Karoglan, A., Kindermann, S., Kraas, L., Felcht, M., Gaiser, M. R., Klemke, C.-D., Kurzen, H., Leibing, T., Müller, V., Reinhard, R. R., Utikal, J., Winter, F., Berking, C., Eicher, L., Hartmann, D., Heppt, M., Kilian, K., Krammer, S., Lill, D., Niesert, A.-C., Oppel, E., Sattler, E., Senner, S., Wallmichrath, J., Wolff, H., Gesierich, A., Giner, T., Glutsch, V., Kerstan, A., Presser, D., Schrüfer, P., Schummer, P., Stolze, I., Weber, J., Drexler, K., Haferkamp, S., Mickler, M., Stauner, C. T., & Thiem, A.
Venue
European Journal of Cancer
Publisher
Elsevier BV
Published
2019-10
Volume
120
Pages
114-121
ISSN
0959-8049 (ISSN-L)
DOI
10.1016/j.ejca.2019.07.019
OpenAlex
W2972588473
Cited by
354
References
9
FWCI
14.96
Type
journal article
Language
en
Open access
hybrid

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.