The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Open accessAbstract
Importance: Convolutional neural networks (CNNs) achieve expert-level accuracy in the diagnosis of pigmented melanocytic lesions. However, the most common types of skin cancer are nonpigmented and nonmelanocytic, and are more difficult to diagnose. Objective: To compare the accuracy of a CNN-based classifier with that of physicians with different levels of experience. Design, Setting, and Participants: A CNN-based classification model was trained on 7895 dermoscopic and 5829 close-up images of lesions excised at a primary skin cancer clinic between January 1, 2008, and July 13, 2017, for a combined evaluation of both imaging methods. The combined CNN (cCNN) was tested on a set of 2072 unknown cases and compared with results from 95 human raters who were medical personnel, including 62 board-certified dermatologists, with different experience in dermoscopy. Main Outcomes and Measures: The proportions of correct specific diagnoses and the accuracy to differentiate between benign and malignant lesions measured as an area under the receiver operating characteristic curve served as main outcome measures. Results: Among 95 human raters (51.6% female; mean age, 43.4 years; 95% CI, 41.0-45.7 years), the participants were divided into 3 groups (according to years of experience with dermoscopy): beginner raters (<3 years), intermediate raters (3-10 years), or expert raters (>10 years). The area under the receiver operating characteristic curve of the trained cCNN was higher than human ratings (0.742; 95% CI, 0.729-0.755 vs 0.695; 95% CI, 0.676-0.713; P < .001). The specificity was fixed at the mean level of human raters (51.3%), and therefore the sensitivity of the cCNN (80.5%; 95% CI, 79.0%-82.1%) was higher than that of human raters (77.6%; 95% CI, 74.7%-80.5%). The cCNN achieved a higher percentage of correct specific diagnoses compared with human raters (37.6%; 95% CI, 36.6%-38.4% vs 33.5%; 95% CI, 31.5%-35.6%; P = .001) but not compared with experts (37.3%; 95% CI, 35.7%-38.8% vs 40.0%; 95% CI, 37.0%-43.0%; P = .18). Conclusions and Relevance: Neural networks are able to classify dermoscopic and close-up images of nonpigmented lesions as accurately as human experts in an experimental setting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A. B. H., Thomas, L., Enk, A., et al. (2018) · Annals of Oncology · doi:10.1093/annonc/mdy166 · Cited by 1,571
Self-citationHan, S. S., Kim, M. S., Lim, W., Park, G. H., Park, I., & Chang, S. E. (2018) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2018.01.028 · Cited by 713
International Journal of Cancer
(2006) · International Journal of Cancer · doi:10.1002/(issn)1097-0215 · Cited by 665
Cited by
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Kumar, Y., Koul, A., Singla, R., & Ijaz, M. F. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03612-z · Cited by 1,040
RetractedGoyal, M., Knackstedt, T., Yan, S., & Hassanpour, S. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104065 · Cited by 428
Ali, M. S., Miah, M. S., Haque, J., Rahman, M. M., & Islam, M. K. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100036 · Cited by 415
Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3129 · Cited by 403
What is AI? Applications of artificial intelligence to dermatology
Du‐Harpur, X., Watt, F. M., Luscombe, N. M., & Lynch, M. D. (2020) · British Journal of Dermatology · doi:10.1111/bjd.18880 · Cited by 355
Superior skin cancer classification by the combination of human and artificial intelligence
Hekler, A., Utikal, J. S., Enk, A. H., Hauschild, A., Weichenthal, M., Maron, R. C., Berking, C., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.07.019 · Cited by 354
Computer-aided diagnosis of skin cancer based on soft computing techniques
Xu, Z., Sheykhahmad, F. R., Ghadimi, N., & Razmjooy, N. (2020) · Open Medicine · doi:10.1515/med-2020-0131 · Cited by 270
Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations
Chan, S., Reddy, V., Myers, B., Thibodeaux, Q., Brownstone, N., & Liao, W. (2020) · Dermatology and Therapy · doi:10.1007/s13555-020-00372-0 · Cited by 267
Haggenmüller, S., Maron, R. C., Hekler, A., Utikal, J. S., Barata, C., Barnhill, R. L., Beltraminelli, H., Berking, C., Betz-Stablein, B., Blum, A., et al. (2021) · European Journal of Cancer · doi:10.1016/j.ejca.2021.06.049 · Cited by 267
Artificial Intelligence in Dermatology: A Primer
Young, A. T., Xiong, M., Pfau, J., Keiser, M. J., & Wei, M. L. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.02.026 · Cited by 249
The Integration of Artificial Intelligence into Clinical Practice
Karalis, V. D. (2024) · Applied Biosciences · doi:10.3390/applbiosci3010002 · Cited by 255
Han, S. S., Park, I., Eun Chang, S., Lim, W., Kim, M. S., Park, G. H., Chae, J. B., Huh, C. H., & Na, J.-I. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.01.019 · Cited by 248
Characteristics of publicly available skin cancer image datasets: a systematic review
Wen, D., Khan, S. M., Ji Xu, A., Ibrahim, H., Smith, L., Caballero, J., Zepeda, L., de Blas Perez, C., Denniston, A. K., Liu, X., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00252-1 · Cited by 243
Maron, R. C., Weichenthal, M., Utikal, J. S., Hekler, A., Berking, C., Hauschild, A., Enk, A. H., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.06.013 · Cited by 219
Skin disease diagnosis with deep learning: a review
Li, H., Pan, Y., Zhao, J., & Zhang, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.096 · Cited by 178
Pre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4
Zhou, J., He, X., Sun, L., Xu, J., Chen, X., Chu, Y., Zhou, L., Liao, X., Zhang, B., Afvari, S., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-50043-3 · Cited by 189
Deep learning in dermatology: a systematic review of current approaches, outcomes, and limitations
Jeong, H. K., Park, C., Henao, R., & Kheterpal, M. (2023) · JID Innovations · doi:10.1016/j.xjidi.2022.100150 · Cited by 134
Accuracy of Computer-Aided Diagnosis of Melanoma: A Meta-analysis
Dick, V., Sinz, C., Mittlböck, M., Kittler, H., & Tschandl, P. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1375 · Cited by 125
Self-citationMultimodal Deep Learning Approaches for Precision Oncology: A Comprehensive Review
Yang, H., Yang, M., Chen, J., Yao, G., Zou, Q., & Jia, L. (2024) · Briefings in Bioinformatics · doi:10.1093/bib/bbae699 · Cited by 115
Malik, H., Farooq, M. S., Khelifi, A., Abid, A., Nasir Qureshi, J., & Hussain, M. (2020) · IEEE Access · doi:10.1109/access.2020.3004766 · Cited by 104
Pham, T.-C., Luong, C.-M., Hoang, V.-D., & Doucet, A. (2021) · Scientific Reports · doi:10.1038/s41598-021-96707-8 · Cited by 101
Fink, C., Blum, A., Buhl, T., Mitteldorf, C., Hofmann‐Wellenhof, R., Deinlein, T., Stolz, W., Trennheuser, L., Cussigh, C., Deltgen, D., et al. (2020) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.16165 · Cited by 71
Self-citationMachine learning applications in the evaluation and management of psoriasis: a systematic review
Yu, K., Syed, M. N., Bernardis, E., & Gelfand, J. M. (2020) · Journal of Psoriasis and Psoriatic Arthritis · doi:10.1177/2475530320950267 · Cited by 65
Effects of Label Noise on Deep Learning-Based Skin Cancer Classification
Hekler, A., Kather, J. N., Krieghoff-Henning, E., Utikal, J. S., Meier, F., Gellrich, F. F., Upmeier zu Belzen, J., French, L., Schlager, J. G., Ghoreschi, K., et al. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00177 · Cited by 58
Felmingham, C. M., Adler, N. R., Ge, Z., Morton, R. L., Janda, M., & Mar, V. J. (2021) · American Journal of Clinical Dermatology · doi:10.1007/s40257-020-00574-4 · Cited by 57
Background selection schema on deep learning-based classification of dermatological disease
Zhou, J., Wu, Z., Jiang, Z., Huang, K., Guo, K., & Zhao, S. (2022) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2022.105966 · Cited by 53
Zhong, F., He, K., Ji, M., Chen, J., Gao, T., Li, S., Zhang, J., & Li, C. (2024) · Scientific Reports · doi:10.1038/s41598-024-59436-2 · Cited by 37
Foltz, E. A., Witkowski, A., Becker, A. L., Latour, E., Lim, J. Y., Hamilton, A., & Ludzik, J. (2024) · Cancers · doi:10.3390/cancers16030629 · Cited by 28
Zhu, A.-Q., Wang, Q., Shi, Y.-L., Ren, W.-W., Cao, X., Ren, T.-T., Wang, J., Zhang, Y.-Q., Sun, Y.-K., Chen, X.-W., et al. (2024) · EClinicalMedicine · doi:10.1016/j.eclinm.2023.102391 · Cited by 25
Use of Artificial Intelligence in Skin Cancer Diagnosis and Management
Wada, M., Ge, Z., Gilmore, S. J., & Mar, V. J. (2020) · Medical Journal of Australia · doi:10.5694/mja2.50759 · Cited by 22
“6 Several studies have now shown that CNNs trained on retrospective image data collected at a single time point are capable of classifying skin cancer with sensitivities and specificities equal or superior to that of dermatologists (Box 1), 5,7–9 , 11 and clinicians with”
BackgroundAksoy, S., Demircioglu, P., & Bogrekci, I. (2024) · Dermato · doi:10.3390/dermato4040015 · Cited by 20
Artificial intelligence in dermatopathology: A systematic review
Lalmalani, R. M., Lim, C. X. Y., & Oh, C. C. (2025) · Clinical and Experimental Dermatology · doi:10.1093/ced/llae361 · Cited by 19
Artificial intelligence: its role in dermatopathology
Jartarkar, S. R. (2022) · Indian Journal of Dermatology Venereology and Leprology · doi:10.25259/ijdvl_725_2021 · Cited by 18
Artificial intelligence in Dermatopathology
Jartarkar, S. R., Cockerell, C. J., Patil, A., Kassir, M., Babaei, M., Weidenthaler‐Barth, B., Grabbe, S., & Goldust, M. (2023) · Journal of Cosmetic Dermatology · doi:10.1111/jocd.15565 · Cited by 17
“A recent study, done by Tschandl et al. 20 , used clinical and dermoscopic images of non-pigmented lesions and compared the classification of CNN with 95 human experts (62 of which were board certified dermatologists).”
BackgroundAlgorithmic bias, generalist models, and clinical medicine
Keeling, G. (2024) · AI and Ethics · doi:10.1007/s43681-023-00329-x · Cited by 5
Metadata
- Authors
- Tschandl, P., Rosendahl, C., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J., Lallas, A., Lapins, J., Marghoob, A., Menzies, S., Neuber, N. M., Paoli, J., Rabinovitz, H. S., Rinner, C., Scope, A., Soyer, H. P., Sinz, C., Thomas, L., Zalaudek, I., & Kittler, H.
- Venue
- JAMA Dermatology
- Publisher
- American Medical Association (AMA)
- Published
- 2019-01-01
- Volume
- 155
- Issue
- 1
- Pages
- 58
- ISSN
- 2168-6068 (ISSN-L)
- OpenAlex
- W2903060508
- Cited by
- 328
- References
- 37
- FWCI
- 10.11
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.