The DOI matches its Crossref record; 37 of the 37 references it cites were found in a registry or an index.
Expert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Open accessAbstract
Importance: Convolutional neural networks (CNNs) achieve expert-level accuracy in the diagnosis of pigmented melanocytic lesions. However, the most common types of skin cancer are nonpigmented and nonmelanocytic, and are more difficult to diagnose. Objective: To compare the accuracy of a CNN-based classifier with that of physicians with different levels of experience. Design, Setting, and Participants: A CNN-based classification model was trained on 7895 dermoscopic and 5829 close-up images of lesions excised at a primary skin cancer clinic between January 1, 2008, and July 13, 2017, for a combined evaluation of both imaging methods. The combined CNN (cCNN) was tested on a set of 2072 unknown cases and compared with results from 95 human raters who were medical personnel, including 62 board-certified dermatologists, with different experience in dermoscopy. Main Outcomes and Measures: The proportions of correct specific diagnoses and the accuracy to differentiate between benign and malignant lesions measured as an area under the receiver operating characteristic curve served as main outcome measures. Results: Among 95 human raters (51.6% female; mean age, 43.4 years; 95% CI, 41.0-45.7 years), the participants were divided into 3 groups (according to years of experience with dermoscopy): beginner raters (<3 years), intermediate raters (3-10 years), or expert raters (>10 years). The area under the receiver operating characteristic curve of the trained cCNN was higher than human ratings (0.742; 95% CI, 0.729-0.755 vs 0.695; 95% CI, 0.676-0.713; P < .001). The specificity was fixed at the mean level of human raters (51.3%), and therefore the sensitivity of the cCNN (80.5%; 95% CI, 79.0%-82.1%) was higher than that of human raters (77.6%; 95% CI, 74.7%-80.5%). The cCNN achieved a higher percentage of correct specific diagnoses compared with human raters (37.6%; 95% CI, 36.6%-38.4% vs 33.5%; 95% CI, 31.5%-35.6%; P = .001) but not compared with experts (37.3%; 95% CI, 35.7%-38.8% vs 40.0%; 95% CI, 37.0%-43.0%; P = .18). Conclusions and Relevance: Neural networks are able to classify dermoscopic and close-up images of nonpigmented lesions as accurately as human experts in an experimental setting.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing
Benjamini, Y., & Hochberg, Y. (1995) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1995.tb02031.x · Cited by 109,482
ImageNet Large Scale Visual Recognition Challenge
Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., et al. (2015) · International Journal of Computer Vision · doi:10.1007/s11263-015-0816-y · Cited by 39,380
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697
Advances in Neural Information Processing Systems 25
Zhang, Y., Sutton, C. A., Storkey, A., & Zoubin Ghahramani, A. (2012) · Cited by 12,653
XGBoost: A Scalable Tree Boosting System
Chen, T., & Guestrin, C. (2016) · arXiv · doi:10.48550/arxiv.1603.02754 · Cited by 7,104
Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A. B. H., Thomas, L., Enk, A., et al. (2018) · Annals of Oncology · doi:10.1093/annonc/mdy166 · Cited by 1,571
Self-citationHan, S. S., Kim, M. S., Lim, W., Park, G. H., Park, I., & Chang, S. E. (2018) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2018.01.028 · Cited by 713
International Journal of Cancer
(2006) · International Journal of Cancer · doi:10.1002/(issn)1097-0215 · Cited by 665
Seven-point checklist and skin lesion classification using multitask multimodal neural nets
Kawahara, J., Daneshvar, S., Argenziano, G., & Hamarneh, G. (2019) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2018.2824327 · Cited by 584
Self-citationReal-time Raman Spectroscopy for In Vivo Skin Cancer Diagnosis
Lui, H., Zhao, J., McLean, D., & Zeng, H. (2012) · Cancer Research · doi:10.1158/0008-5472.can-11-4061 · Cited by 429
Marchetti, M. A., Codella, N. C. F., Dusza, S. W., Gutman, D. A., Helba, B., Kalloo, A., Mishra, N., Carrera, C., Celebi, M. E., DeFazio, J. L., et al. (2018) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2017.08.016 · Cited by 330
Self-citationMultimodal skin lesion classification using deep learning
Yap, J., Yolland, W., & Tschandl, P. (2018) · Experimental Dermatology · doi:10.1111/exd.13777 · Cited by 326
Self-citationKittler, H., Marghoob, A. A., Argenziano, G., Carrera, C., Curiel-Lewandrowski, C., Hofmann-Wellenhof, R., Malvehy, J., Menzies, S., Puig, S., Rabinovitz, H., et al. (2016) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2015.12.038 · Cited by 319
Self-citationDermoscopic Evaluation of Amelanotic and Hypomelanotic Melanoma
Menzies, S. W., Kreusch, J., Byth, K., Pizzichetta, M. A., Marghoob, A., Braun, R., Malvehy, J., Puig, S., Argenziano, G., Zalaudek, I., et al. (2008) · Archives of Dermatology · doi:10.1001/archderm.144.9.1120 · Cited by 306
Self-citationBallerini, L., Fisher, R. B., Aldridge, B., & Rees, J. (2013) · Lecture Notes in Computational Vision and Biomechanics · doi:10.1007/978-94-007-5389-1_4 · Cited by 273
Diagnostic accuracy of dermatoscopy for melanocytic and nonmelanocytic pigmented lesions
Rosendahl, C., Tschandl, P., Cameron, A., & Kittler, H. (2011) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2010.03.039 · Cited by 213
Self-citationMalvehy, J., Hauschild, A., Curiel‐Lewandrowski, C., Mohr, P., Hofmann‐Wellenhof, R., Motley, R., Berking, C., Grossman, D., Paoli, J., Loquai, C., et al. (2014) · British Journal of Dermatology · doi:10.1111/bjd.13121 · Cited by 210
Self-citationCarli, P., de Giorgi, V., Chiarugi, A., Nardini, P., Weinstock, M. A., Crocetti, E., Stante, M., & Giannotti, B. (2004) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2003.09.009 · Cited by 207
Accuracy in melanoma detection: A 10-year multicenter survey
Argenziano, G., Cerroni, L., Zalaudek, I., Staibano, S., Hofmann-Wellenhof, R., Arpaia, N., Bakos, R. M., Balme, B., Bandic, J., Bandelloni, R., et al. (2012) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2011.07.019 · Cited by 207
Self-citationAutomated diagnosis of pigmented skin lesions
Rubegni, P., Cevenini, G., Burroni, M., Perotti, R., Dell'Eva, G., Sbano, P., Miracco, C., Luzi, P., Tosi, P., Barbini, P., et al. (2002) · International Journal of Cancer · doi:10.1002/ijc.10620 · Cited by 182
Management Reasoning: Beyond the Diagnosis
Cook, D. A., Sherbino, J., & Durning, S. J. (2018) · JAMA · doi:10.1001/jama.2018.4385 · Cited by 170
Menzies, S. W., Bischof, L., Talbot, H., Gutenev, A., Avramidis, M., Wong, L., Lo, S. K., Mackellar, G., Skladnev, V., McCarthy, W. H., et al. (2005) · Archives of Dermatology · doi:10.1001/archderm.141.11.1388 · Cited by 156
Self-citationGe, Z., Demyanov, S., Chakravorty, R., Bowling, A., & Garnavi, R. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7_29 · Cited by 145
Acral melanoma detection using a convolutional neural network for dermoscopy images
Yu, C., Yang, S., Kim, W. O., Jung, J. W., Chung, K. Y., Lee, S. W., & Oh, B. H. (2018) · PLoS ONE · doi:10.1371/journal.pone.0193321 · Cited by 133
Noninvasive genomic detection of melanoma
Wachsman, W., Morhenn, V. B., Palmer, T. J., Walls, L., Hata, T. R., Zalla, J. A., Scheinberg, R. S., Sofen, H. L., Mraz‐Gernhard, S., Gross, K., et al. (2011) · British Journal of Dermatology · doi:10.1111/j.1365-2133.2011.10239.x · Cited by 128
Self-citationAccuracy of dermatoscopy for the diagnosis of nonpigmented cancers of the skin
Sinz, C., Tschandl, P., Rosendahl, C., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J., et al. (2017) · Journal of the American Academy of Dermatology · doi:10.1016/j.jaad.2017.07.022 · Cited by 127
Self-citationBinder, M., Steiner, A., Schwarz, M., Knollmayer, S., Wolff, K., & Pehamberger, H. (1994) · British Journal of Dermatology · doi:10.1111/j.1365-2133.1994.tb03378.x · Cited by 110
Proposed Technical Guidelines for the Acquisition of Clinical Images of Skin-Related Conditions
Finnane, A., Curiel-Lewandrowski, C., Wimberley, G., Caffery, L., Katragadda, C., Halpern, A., Marghoob, A. A., Malvehy, J., Kittler, H., Hofmann-Wellenhof, R., et al. (2017) · JAMA Dermatology · doi:10.1001/jamadermatol.2016.6214 · Cited by 89
Self-citationCelebi, M., & Schaefer, G. (2012) · Lecture notes in computational vision and biomechanics · doi:10.1007/978-94-007-5389-1 · Cited by 85
Kharazmi, P., Kalia, S., Lui, H., Wang, Z. J., & Lee, T. K. (2018) · Skin Research and Technology · doi:10.1111/srt.12422 · Cited by 83
Binder, M., Kittler, H., Dreiseitl, S., Ganster, H., Wolff, K., & Pehamberger, H. (2000) · Melanoma Research · doi:10.1097/00008390-200012000-00007 · Cited by 62
Self-citationMedical Image Computing and Computer Assisted Intervention − MICCAI 2017
Descoteaux, M., Maier‐Hein, L., Franz, A. M., Jannin, P., Collins, D. L., & Duchesne, S. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-66179-7 · Cited by 59
Tschandl, P., Kittler, H., & Argenziano, G. (2017) · British Journal of Dermatology · doi:10.1111/bjd.15695 · Cited by 27
Self-citationApplying a decision support system in clinical practice: results from melanoma diagnosis.
Dreiseitl, S., Binder, M., Vinterbo, S. A., & Kittler, H. (2007) · PubMed · Cited by 13
Self-citationCarvalho, N. D., Welzel, J., Schuh, S., et al. (2018) · Experimental Dermatology · doi:10.1111/exd.2018.27.issue-11 · Cited by 9
Cited by
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Kumar, Y., Koul, A., Singla, R., & Ijaz, M. F. (2023) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-021-03612-z · Cited by 1,040
RetractedGoyal, M., Knackstedt, T., Yan, S., & Hassanpour, S. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104065 · Cited by 428
Ali, M. S., Miah, M. S., Haque, J., Rahman, M. M., & Islam, M. K. (2021) · Machine Learning with Applications · doi:10.1016/j.mlwa.2021.100036 · Cited by 415
Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3129 · Cited by 403
Metadata
- Authors
- Tschandl, P., Rosendahl, C., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J., Lallas, A., Lapins, J., Marghoob, A., Menzies, S., Neuber, N. M., Paoli, J., Rabinovitz, H. S., Rinner, C., Scope, A., Soyer, H. P., Sinz, C., Thomas, L., Zalaudek, I., & Kittler, H.
- Venue
- JAMA Dermatology
- Publisher
- American Medical Association (AMA)
- Published
- 2019-01-01
- Volume
- 155
- Issue
- 1
- Pages
- 58
- ISSN
- 2168-6068 (ISSN-L)
- OpenAlex
- W2903060508
- Cited by
- 328
- References
- 37
- FWCI
- 10.11
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.