Man against machine: diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists
Open accessThe DOI matches its Crossref record; 22 of the 22 references it cites were found in a registry or an index.
Abstract
Background Deep learning convolutional neural networks (CNN) may facilitate melanoma detection, but data comparing a CNN's diagnostic performance to larger groups of dermatologists are lacking. Methods Google's Inception v4 CNN architecture was trained and validated using dermoscopic images and corresponding diagnoses. In a comparative cross-sectional reader study a 100-image test-set was used (level-I: dermoscopy only; level-II: dermoscopy plus clinical information and images). Main outcome measures were sensitivity, specificity and area under the curve (AUC) of receiver operating characteristics (ROC) for diagnostic classification (dichotomous) of lesions by the CNN versus an international group of 58 dermatologists during level-I or -II of the reader study. Secondary end points included the dermatologists' diagnostic performance in their management decisions and differences in the diagnostic performance of dermatologists during level-I and -II of the reader study. Additionally, the CNN's performance was compared with the top-five algorithms of the 2016 International Symposium on Biomedical Imaging (ISBI) challenge. Results In level-I dermatologists achieved a mean (±standard deviation) sensitivity and specificity for lesion classification of 86.6% (±9.3%) and 71.3% (±11.2%), respectively. More clinical information (level-II) improved the sensitivity to 88.9% (±9.6%, P = 0.19) and specificity to 75.7% (±11.7%, P < 0.05). The CNN ROC curve revealed a higher specificity of 82.5% when compared with dermatologists in level-I (71.3%, P < 0.01) and level-II (75.7%, P < 0.01) at their sensitivities of 86.6% and 88.9%, respectively. The CNN ROC AUC was greater than the mean ROC area of dermatologists (0.86 versus 0.79, P < 0.01). The CNN scored results close to the top three algorithms of the ISBI 2016 challenge. Conclusions For the first time we compared a CNN's diagnostic performance with a large international group of 58 dermatologists, including 30 experts. Most dermatologists were outperformed by the CNN. Irrespective of any physicians' experience, they may benefit from assistance by a CNN's image classification. Clinical trial number This study was registered at the German Clinical Trial Register (DRKS-Study-ID: DRKS00013570; https://www.drks.de/drks_web/).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




![Figure 2 (A) ROC curve of the CNN in relation to the average ( 6 SD) sensitivity and specificity of all dermatologists [mean: green (online) circle; 6 SD: green (online) error bars] in set-100 (dichotomous classification, study level-I) and the dermatologists' mean sensitivity and specificity in relation to their level of experience. (B) ROC curve of the CNN in set-300.](https://figures.cochat.ai/paper-figures/76c9142373b44f08af897830b38f5f96/man-against-machine-diagnostic-performance-of-a-deep-learning-convolutional-figure-2.png)
![Figure 3 Comparison of ROC curves of the CNN described in this study (dark green (online) line) to the top-five ranked individual algorithms of the 2016 International Symposium on Biomedical Imaging (ISBI) challenge [18]. ROC AUCs in descending order were as follows: ISBI team-2: 0.7956; ISBI team-1: 0.7928; ISBI team-3: 0.7892; CNN of this study: 0.7868; ISBI team-4: 0.5460; ISBI team-5: 0.5324.](https://figures.cochat.ai/paper-figures/83ada374a42641bdae95a7d0d95bd1bc/man-against-machine-diagnostic-performance-of-a-deep-learning-convolutional-figure-3.png)
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“A series of works have demonstrated that CNNs can match the performance of board-certified dermatologists at classifying malignant skin lesions from benign ones 7,93,94 .”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [93]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“For instance, AI is shown to diagnose skin cancer more efficiently than dermatologists [54].”
From Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives (2020) · cites this paper as [54]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Several algorithms incorporating artificial intelligence (AI) have been developed to help interpret both clinical11,12,13,14,15 and dermoscopic16,17,18,19,20,21,22,23 images for a variety of skin conditions”
In the citing paper
“These studies have sequentially tested increasing numbers of dermatologists (25-7 57-93 , 157-94 ), consistently demonstrating a sensitivity and specificity in classification that matches or even exceeds physician levels.”
From Deep learning-enabled medical computer vision (2021) · cites this paper as [93]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The publisher hasn’t deposited this paper’s reference list.
Cited by
High-performance medicine: the convergence of human and artificial intelligence
Topol, E. J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0300-7 · Cited by 10,450
A guide to deep learning in healthcare
Esteva, A., Robicquet, A., Ramsundar, B., Kuleshov, V., DePristo, M., Chou, K., Cui, C., Corrado, G., Thrun, S., & Dean, J. (2019) · Nature Medicine · doi:10.1038/s41591-018-0316-z · Cited by 5,312
“…deep-learning models have achieved physician-level accuracy at a broad variety of diagnostic tasks, including identifying moles from melanomas 9,10 , diabetic retinopathy, cardiovascular risk, and referrals from fundus 15,16 and optical coherence tomography (OCT) 17 images of the eye, breast…”
BackgroundRevolutionizing healthcare: the role of artificial intelligence in clinical practice
Alowais, S. A., Alghamdi, S. S., Alsuhebany, N., Alqahtani, T., Alshaya, A. I., Almohareb, S. N., Aldairem, A., Alrashed, M., Bin Saleh, K., Badreldin, H. A., et al. (2023) · BMC Medical Education · doi:10.1186/s12909-023-04698-z · Cited by 3,146
“Furthermore, a study utilized deep learning to detect skin cancer which showed that an AI using CNN accurately diagnosed melanoma cases compared to derma-tologists and recommended treatment options [13, 14].”
BackgroundKey challenges for delivering clinical impact with artificial intelligence
Kelly, C. J., Karthikesalingam, A., Suleyman, M., Corrado, G., & King, D. (2019) · BMC Medicine · doi:10.1186/s12916-019-1426-2 · Cited by 3,136
Artificial intelligence in healthcare
Yu, K.-H., Beam, A. L., & Kohane, I. S. (2018) · Nature Biomedical Engineering · doi:10.1038/s41551-018-0305-z · Cited by 2,817
Resistance to Medical Artificial Intelligence
Longoni, C., Bonezzi, A., & Morewedge, C. K. (2019) · Journal of Consumer Research · doi:10.1093/jcr/ucz013 · Cited by 2,015
Liu, X., Faes, L., Kale, A. U., Wagner, S. K., Fu, D. J., Bruynseels, A., Mahendiran, T., Moraes, G., Shamdas, M., Kern, C., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30123-2 · Cited by 1,883
Artificial intelligence in digital pathology — new tools for diagnosis and precision oncology
Bera, K., Schalper, K. A., Rimm, D. L., Velcheti, V., & Madabhushi, A. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0252-y · Cited by 1,698
Deep learning-enabled medical computer vision
Esteva, A., Chou, K., Yeung, S., Naik, N., Madani, A., Mottaghi, A., Liu, Y., Topol, E., Dean, J., & Socher, R. (2021) · npj Digital Medicine · doi:10.1038/s41746-020-00376-2 · Cited by 1,406
Ahmed, Z., Mohamed, K., Zeeshan, S., & Dong, X. (2020) · Database · doi:10.1093/database/baaa010 · Cited by 1,035
Human-computer collaboration for skin cancer recognition
Tschandl, P., Rinner, C., Apalla, Z., Argenziano, G., Codella, N., Halpern, A., Janda, M., Lallas, A., Longo, C., Malvehy, J., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0942-0 · Cited by 1,024
Self-citationA deep learning system for differential diagnosis of skin diseases
Liu, Y., Jain, A., Eng, C., Way, D. H., Lee, K., Bui, P., Kanada, K., de Oliveira Marinho, G., Gallegos, J., Gabriele, S., et al. (2020) · Nature Medicine · doi:10.1038/s41591-020-0842-3 · Cited by 853
AI in Medical Imaging Informatics: Current Challenges and Future Directions
Panayides, A. S., Amini, A., Filipovic, N. D., Sharma, A., Tsaftaris, S. A., Young, A., Foran, D., Do, N., Golemati, S., Kurc, T., et al. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.2991043 · Cited by 713
Artificial intelligence in cancer diagnosis and prognosis: Opportunities and challenges
Huang, S., Yang, J., Fong, S., & Zhao, Q. (2020) · Cancer Letters · doi:10.1016/j.canlet.2019.12.007 · Cited by 672
Deep learning in cancer pathology: a new generation of clinical biomarkers
Echle, A., Rindtorff, N. T., Brinker, T. J., Luedde, T., Pearson, A. T., & Kather, J. N. (2021) · British Journal of Cancer · doi:10.1038/s41416-020-01122-x · Cited by 667
A survey on active learning and human-in-the-loop deep learning for medical image analysis
Budd, S., Robinson, E. C., & Kainz, B. (2021) · Medical Image Analysis · doi:10.1016/j.media.2021.102062 · Cited by 606
Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., Schilling, B., Haferkamp, S., Schadendorf, D., Holland-Letz, T., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.04.001 · Cited by 605
Self-citationFAT-Net: Feature adaptive transformers for automated skin lesion segmentation
Wu, H., Chen, S., Chen, G., Wang, W., Lei, B., & Wen, Z. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102327 · Cited by 590
Tschandl, P., Codella, N., Akay, B. N., Argenziano, G., Braun, R. P., Cabo, H., Gutman, D., Halpern, A., Helba, B., Hofmann-Wellenhof, R., et al. (2019) · The Lancet Oncology · doi:10.1016/s1470-2045(19)30333-x · Cited by 555
“Furthermore, Haenssle and colleagues 15 reported expert-level accuracy of algorithms for dermatoscopic images of melanocytic lesions.”
BackgroundResultSelf-citationDisparities in dermatology AI performance on a diverse, curated clinical image set
Daneshjou, R., Vodrahalli, K., Novoa, R. A., Jenkins, M., Liang, W., Rotemberg, V., Ko, J., Swetter, S. M., Bailey, E. E., Gevaert, O., et al. (2022) · Science Advances · doi:10.1126/sciadv.abq6147 · Cited by 544
Use of AI-based tools for healthcare purposes: a survey study from consumers’ perspectives
Esmaeilzadeh, P. (2020) · BMC Medical Informatics and Decision Making · doi:10.1186/s12911-020-01191-1 · Cited by 507
Skin Cancer Classification Using Convolutional Neural Networks: Systematic Review
Brinker, T. J., Hekler, A., Utikal, J. S., Grabe, N., Schadendorf, D., Klode, J., Berking, C., Steeb, T., Enk, A. H., & von Kalle, C. (2018) · Journal of Medical Internet Research · doi:10.2196/11936 · Cited by 500
Self-citationNagpal, K., Foote, D., Liu, Y., Chen, P.-H. C., Wulczyn, E., Tan, F., Olson, N., Smith, J. L., Mohtashamian, A., Wren, J. H., et al. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0112-2 · Cited by 495
Artificial intelligence in medical education
Masters, K. (2019) · Medical Teacher · doi:10.1080/0142159x.2019.1595557 · Cited by 469
“Already, computers can perform prediction and diagnosis better than humans (Bogu (cid:1) sevi (cid:1) cius et al. 2002; Esteva et al. 2017; Litjens et al. 2017; Mobadersany et al. 2017; Betancur et al. 2018; De Fauw et al. 2018; Haenssle et al. 2018).”
BackgroundRETRACTED ARTICLE: Artificial intelligence and machine learning in precision and genomic medicine
Quazi, S. (2022) · Medical Oncology · doi:10.1007/s12032-022-01711-1 · Cited by 432
RetractedGoyal, M., Knackstedt, T., Yan, S., & Hassanpour, S. (2020) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2020.104065 · Cited by 428
Cancer Diagnosis Using Deep Learning: A Bibliographic Review
Munir, K., Elahi, H., Ayub, A., Frezza, F., & Rizzi, A. (2019) · Cancers · doi:10.3390/cancers11091235 · Cited by 422
Deep neural networks are superior to dermatologists in melanoma image classification
Brinker, T. J., Hekler, A., Enk, A. H., Berking, C., Haferkamp, S., Hauschild, A., Weichenthal, M., Klode, J., Schadendorf, D., Holland-Letz, T., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.05.023 · Cited by 415
Self-citationHidden stratification causes clinically meaningful failures in machine learning for medical imaging
Oakden-Rayner, L., Dunnmon, J., Carneiro, G., & Re, C. (2020) · Proceedings of the ACM Conference on Health, Inference, and Learning · doi:10.1145/3368555.3384468 · Cited by 411
Daneshjou, R., Smith, M. P., Sun, M. D., Rotemberg, V., & Zou, J. (2021) · JAMA Dermatology · doi:10.1001/jamadermatol.2021.3129 · Cited by 403
Winkler, J. K., Fink, C., Toberer, F., Enk, A., Deinlein, T., Hofmann-Wellenhof, R., Thomas, L., Lallas, A., Blum, A., Stolz, W., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1735 · Cited by 365
Self-citationWhat is AI? Applications of artificial intelligence to dermatology
Du‐Harpur, X., Watt, F. M., Luscombe, N. M., & Lynch, M. D. (2020) · British Journal of Dermatology · doi:10.1111/bjd.18880 · Cited by 355
Superior skin cancer classification by the combination of human and artificial intelligence
Hekler, A., Utikal, J. S., Enk, A. H., Hauschild, A., Weichenthal, M., Maron, R. C., Berking, C., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.07.019 · Cited by 354
Self-citationExpert-Level Diagnosis of Nonpigmented Skin Cancer by Combined Convolutional Neural Networks
Tschandl, P., Rosendahl, C., Akay, B. N., Argenziano, G., Blum, A., Braun, R. P., Cabo, H., Gourhant, J.-Y., Kreusch, J., Lallas, A., et al. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2018.4378 · Cited by 328
Self-citationDesigning deep learning studies in cancer diagnostics
Kleppe, A., Skrede, O.-J., De Raedt, S., Liestøl, K., Kerr, D. J., & Danielsen, H. E. (2021) · Nature Reviews Cancer · doi:10.1038/s41568-020-00327-9 · Cited by 316
Artificial intelligence in diagnostic pathology
Shafi, S., & Parwani, A. V. (2023) · Diagnostic Pathology · doi:10.1186/s13000-023-01375-z · Cited by 315
Analysis of the ISIC image datasets: Usage, benchmarks and recommendations
Cassidy, B., Kendrick, C., Brodzicki, A., Jaworek-Korjakowska, J., & Yap, M. H. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102305 · Cited by 307
Xu, J., Yang, P., Xue, S., Sharma, B., Sanchez-Martin, M., Wang, F., Beaty, K. A., Dehan, E., & Parikh, B. (2019) · Human Genetics · doi:10.1007/s00439-019-01970-5 · Cited by 305
Brinker, T. J., Hekler, A., Enk, A. H., Klode, J., Hauschild, A., Berking, C., Schilling, B., Haferkamp, S., Schadendorf, D., Fröhling, S., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.02.005 · Cited by 302
Self-citationAI-based computer-aided diagnosis (AI-CAD): the latest review to read first
Fujita, H. (2020) · Radiological Physics and Technology · doi:10.1007/s12194-019-00552-4 · Cited by 300
Laï, M.-C., Brian, M., & Mamzer, M.-F. (2020) · Journal of Translational Medicine · doi:10.1186/s12967-019-02204-y · Cited by 294
Going Deep in Medical Image Analysis: Concepts, Methods, Challenges, and Future Directions
Altaf, F., Islam, S. M. S., Akhtar, N., & Janjua, N. K. (2019) · IEEE Access · doi:10.1109/access.2019.2929365 · Cited by 286
Hekler, A., Utikal, J. S., Enk, A. H., Solass, W., Schmitt, M., Klode, J., Schadendorf, D., Sondermann, W., Franklin, C., Bestvater, F., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.06.012 · Cited by 285
Artificial Intelligence in Medicine: Where Are We Now?
Kulkarni, S., Seneviratne, N., Baig, M. S., & Khan, A. H. A. (2020) · Academic Radiology · doi:10.1016/j.acra.2019.10.001 · Cited by 282
Machine learning in clinical decision making
Adlung, L., Cohen, Y., Mor, U., & Elinav, E. (2021) · Med · doi:10.1016/j.medj.2021.04.006 · Cited by 282
Nelson, C. A., Pérez-Chada, L. M., Creadore, A., Li, S. J., Lo, K., Manjaly, P., Pournamdari, A. B., Tkachenko, E., Barbieri, J. S., Ko, J. M., et al. (2020) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.5014 · Cited by 278
Scheetz, J., Rothschild, P., McGuinness, M., Hadoux, X., Soyer, H. P., Janda, M., Condon, J. J. J., Oakden-Rayner, L., Palmer, L. J., Keel, S., et al. (2021) · Scientific Reports · doi:10.1038/s41598-021-84698-5 · Cited by 273
Jones, O. T., Matin, R. N., van der Schaar, M., Prathivadi Bhayankaram, K., Ranmuthu, C. K. I., Islam, M. S., Behiyat, D., Boscott, R., Calanzani, N., Emery, J., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(22)00023-1 · Cited by 272
Algorithmic bias: Senses, sources, solutions
Fazelpour, S., & Danks, D. (2021) · Philosophy Compass · doi:10.1111/phc3.12760 · Cited by 272
Kassem, M. A., Hosny, K. M., Damaševičius, R., & Eltoukhy, M. M. (2021) · Diagnostics · doi:10.3390/diagnostics11081390 · Cited by 270
Machine Learning in Dermatology: Current Applications, Opportunities, and Limitations
Chan, S., Reddy, V., Myers, B., Thibodeaux, Q., Brownstone, N., & Liao, W. (2020) · Dermatology and Therapy · doi:10.1007/s13555-020-00372-0 · Cited by 267
Haggenmüller, S., Maron, R. C., Hekler, A., Utikal, J. S., Barata, C., Barnhill, R. L., Beltraminelli, H., Berking, C., Betz-Stablein, B., Blum, A., et al. (2021) · European Journal of Cancer · doi:10.1016/j.ejca.2021.06.049 · Cited by 267
Jinnai, S., Yamazaki, N., Hirano, Y., Sugawara, Y., Ohe, Y., & Hamamoto, R. (2020) · Biomolecules · doi:10.3390/biom10081123 · Cited by 266
Patients’ views of wearable devices and AI in healthcare: findings from the ComPaRe e-cohort
Tran, V.-T., Riveros, C., & Ravaud, P. (2019) · npj Digital Medicine · doi:10.1038/s41746-019-0132-y · Cited by 264
Pacheco, A. G. C., & Krohling, R. A. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3062002 · Cited by 257
Kadampur, M. A., & Al Riyaee, S. (2020) · Informatics in Medicine Unlocked · doi:10.1016/j.imu.2019.100282 · Cited by 256
Artificial Intelligence in Dermatology: A Primer
Young, A. T., Xiong, M., Pfau, J., Keiser, M. J., & Wei, M. L. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.02.026 · Cited by 249
Han, S. S., Park, I., Eun Chang, S., Lim, W., Kim, M. S., Park, G. H., Chae, J. B., Huh, C. H., & Na, J.-I. (2020) · Journal of Investigative Dermatology · doi:10.1016/j.jid.2020.01.019 · Cited by 248
Ten Ways Artificial Intelligence Will Transform Primary Care
Lin, S. Y., Mahoney, M. R., & Sinsky, C. A. (2019) · Journal of General Internal Medicine · doi:10.1007/s11606-019-05035-1 · Cited by 248
Characteristics of publicly available skin cancer image datasets: a systematic review
Wen, D., Khan, S. M., Ji Xu, A., Ibrahim, H., Smith, L., Caballero, J., Zepeda, L., de Blas Perez, C., Denniston, A. K., Liu, X., et al. (2022) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00252-1 · Cited by 243
Tahir, M., Naeem, A., Malik, H., Tanveer, J., Naqvi, R. A., & Lee, S.-W. (2023) · Cancers · doi:10.3390/cancers15072179 · Cited by 240
“In this study, we reduce the complexity of the model by decreasing the number of trainable parameters to obtain a significant classifier.”
MethodPathologist-level classification of histopathological melanoma images with deep neural networks
Hekler, A., Utikal, J. S., Enk, A. H., Berking, C., Klode, J., Schadendorf, D., Jansen, P., Franklin, C., Holland-Letz, T., Krahl, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.04.021 · Cited by 231
Self-citationMaron, R. C., Weichenthal, M., Utikal, J. S., Hekler, A., Berking, C., Hauschild, A., Enk, A. H., Haferkamp, S., Klode, J., Schadendorf, D., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2019.06.013 · Cited by 219
Self-citationPre-trained multimodal large language model enhances dermatological diagnosis using SkinGPT-4
Zhou, J., He, X., Sun, L., Xu, J., Chen, X., Chu, Y., Zhou, L., Liao, X., Zhang, B., Afvari, S., et al. (2024) · Nature Communications · doi:10.1038/s41467-024-50043-3 · Cited by 189
Naeem, A., Farooq, M. S., Khelifi, A., & Abid, A. (2020) · IEEE Access · doi:10.1109/access.2020.3001507 · Cited by 189
“The report of the World Health Organization (WHO) shows that cancer is one of the world’s leading causes of death [2].”
BackgroundKuziemsky, C., Maeder, A. J., John, O., Gogia, S. B., Basu, A., Meher, S., & Ito, M. (2019) · Yearbook of Medical Informatics · doi:10.1055/s-0039-1677897 · Cited by 183
Ellahham, S., Ellahham, N., & Simsekler, M. C. E. (2020) · American Journal of Medical Quality · doi:10.1177/1062860619878515 · Cited by 177
Skin disease diagnosis with deep learning: a review
Li, H., Pan, Y., Zhao, J., & Zhang, L. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.08.096 · Cited by 178
Deep Learning-Based Transfer Learning for Classification of Skin Cancer
Jain, S., Singhania, U., Tripathy, B., Nasr, E. A., Aboudaif, M. K., & Kamrani, A. K. (2021) · Sensors · doi:10.3390/s21238142 · Cited by 169
“Although the advantage of the above-mentioned research works is that they provide a straightforward algorithm approach and acceptable accuracy, most of them did not consider all types of legions and used relatively old datasets. squamous cell carcinoma, and melanoma.”
BackgroundJain, A., Way, D., Gupta, V., Gao, Y., de Oliveira Marinho, G., Hartford, J., Sayres, R., Kanada, K., Eng, C., Nagpal, K., et al. (2021) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.7249 · Cited by 164
Artificial Intelligence in Nephrology: Core Concepts, Clinical Applications, and Perspectives
Niel, O., & Bastard, P. (2019) · American Journal of Kidney Diseases · doi:10.1053/j.ajkd.2019.05.020 · Cited by 165
Brinker, T. J., Hekler, A., Hauschild, A., Berking, C., Schilling, B., Enk, A. H., Haferkamp, S., Karoglan, A., von Kalle, C., Weichenthal, M., et al. (2019) · European Journal of Cancer · doi:10.1016/j.ejca.2018.12.016 · Cited by 157
Self-citationWhen computers take the lead: The automation of leadership
Wesche, J. S., & Sonderegger, A. (2019) · Computers in Human Behavior · doi:10.1016/j.chb.2019.07.027 · Cited by 157
Melanoma detection using deep learning-based classifications
Alwakid, G., Gouda, W., Humayun, M., & Sama, N. U. (2022) · Healthcare · doi:10.3390/healthcare10122481 · Cited by 153
Human Factors Challenges for the Safe Use of Artificial Intelligence in Patient Care
Sujan, M., Furniss, D., Grundy, K., Grundy, H., Nelson, D., Elliott, M., White, S., Habli, I., & Reynolds, N. (2019) · BMJ Health & Care Informatics · doi:10.1136/bmjhci-2019-100081 · Cited by 144
Deep learning in dermatology: a systematic review of current approaches, outcomes, and limitations
Jeong, H. K., Park, C., Henao, R., & Kheterpal, M. (2023) · JID Innovations · doi:10.1016/j.xjidi.2022.100150 · Cited by 134
Human and artificial cognition
Siemens, G., Marmolejo-Ramos, F., Gabriel, F., Medeiros, K., Marrone, R., Joksimovic, S., & de Laat, M. (2022) · Computers and Education: Artificial Intelligence · doi:10.1016/j.caeai.2022.100107 · Cited by 133
Maassen, O., Fritsch, S., Palm, J., Deffge, S., Kunze, J., Marx, G., Riedel, M., Schuppert, A., & Bickenbach, J. (2021) · Journal of Medical Internet Research · doi:10.2196/26646 · Cited by 125
Accuracy of Computer-Aided Diagnosis of Melanoma: A Meta-analysis
Dick, V., Sinz, C., Mittlböck, M., Kittler, H., & Tschandl, P. (2019) · JAMA Dermatology · doi:10.1001/jamadermatol.2019.1375 · Cited by 125
Trust in artificial intelligence for medical diagnoses
(2020) · Progress in Brain Research · doi:10.1016/bs.pbr.2020.06.006 · Cited by 116
Biopsy-Free in Vivo Virtual Histology of Skin Using Deep Learning
Li, J., Garfinkel, J., Zhang, X., Wu, D., Zhang, Y., de Haan, K., Wang, H., Liu, T., Bai, B., Rivenson, Y., et al. (2021) · Light: Science & Applications · doi:10.1038/s41377-021-00674-8 · Cited by 109
Höhn, J., Krieghoff-Henning, E., Jutzi, T. B., von Kalle, C., Utikal, J. S., Meier, F., Gellrich, F. F., Hobelsberger, S., Hauschild, A., Schlager, J. G., et al. (2021) · European Journal of Cancer · doi:10.1016/j.ejca.2021.02.032 · Cited by 106
Schlicker, N., Langer, M., Ötting, S. K., Baum, K., König, C. J., & Wallach, D. (2021) · Computers in Human Behavior · doi:10.1016/j.chb.2021.106837 · Cited by 106
Malik, H., Farooq, M. S., Khelifi, A., Abid, A., Nasir Qureshi, J., & Hussain, M. (2020) · IEEE Access · doi:10.1109/access.2020.3004766 · Cited by 104
Accuracy of a smartphone application for triage of skin lesions based on machine learning algorithms
Udrea, A., Mitra, G. D., Costea, D., Noels, E. C., Wakkee, M., Siegel, D. M., de Carvalho, T. M., & Nijsten, T. E. C. (2020) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.15935 · Cited by 96
Artificial intelligence in food science and nutrition: A narrative review
Miyazawa, T., Hiratsuka, Y., Toda, M., Hatakeyama, N., Ozawa, H., Abe, C., Cheng, T.-Y., Matsushima, Y., Miyawaki, Y., Ashida, K., et al. (2022) · Nutrition Reviews · doi:10.1093/nutrit/nuac033 · Cited by 91
Kempt, H., & Nagel, S. K. (2022) · Journal of Medical Ethics · doi:10.1136/medethics-2021-107440 · Cited by 87
“The anticipated superiority of machines in several diagnoses such as melanoma 20 or early breast cancer detection 21 22 (for an overview of studies on how physicians fare with AI diagnoses, see Topol 1 ) suggest that if we rejected their use, we would prefer worse outcomes in diagnostic precision…”
BackgroundFairLens: Auditing black-box clinical decision support systems
Panigutti, C., Perotti, A., Panisson, A., Bajardi, P., & Pedreschi, D. (2021) · Information Processing & Management · doi:10.1016/j.ipm.2021.102657 · Cited by 86
Explaining Machine Learning Models for Clinical Gait Analysis
Slijepcevic, D., Horst, F., Lapuschkin, S., Horsak, B., Raberger, A.-M., Kranzl, A., Samek, W., Breiteneder, C., Schöllhorn, W. I., & Zeppelzauer, M. (2022) · ACM Transactions on Computing for Healthcare · doi:10.1145/3474121 · Cited by 82
“Impressively, there are certain medical tasks, such as the detection of skin or breast cancer, that ML approaches have already been able to solve more efficiently and effectively than humans [16, 21, 42].”
BackgroundFink, C., Blum, A., Buhl, T., Mitteldorf, C., Hofmann‐Wellenhof, R., Deinlein, T., Stolz, W., Trennheuser, L., Cussigh, C., Deltgen, D., et al. (2020) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.16165 · Cited by 71
Self-citationCapable but Amoral? Comparing AI and Human Expert Collaboration in Ethical Decision Making
Tolmeijer, S., Christen, M., Kandul, S., Kneer, M., & Bernstein, A. (2022) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3491102.3517732 · Cited by 70
Muñoz‐López, C., Ramírez‐Cornejo, C., Marchetti, M. A., Han, S. S., Del Barrio‐Díaz, P., Jaque, A., Uribe, P., Majerson, D., Curi, M., Del Puerto, C., et al. (2021) · Journal of the European Academy of Dermatology and Venereology · doi:10.1111/jdv.16979 · Cited by 70
The Next Generation of Medical Decision Support: A Roadmap Toward Transparent Expert Companions
Bruckert, S., Finzel, B., & Schmid, U. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.507973 · Cited by 66
“Several recent studies show that ML can help to increase the accuracy of diagnosis (Weng et al., 2017; Haenssle et al., 2018; Hu et al., 2019).”
BackgroundMachine learning applications in the evaluation and management of psoriasis: a systematic review
Yu, K., Syed, M. N., Bernardis, E., & Gelfand, J. M. (2020) · Journal of Psoriasis and Psoriatic Arthritis · doi:10.1177/2475530320950267 · Cited by 65
Guo, L., Yang, Y., Ding, H., Zheng, H., Yang, H., Xie, J., Li, Y., Lin, T., & Ge, Y. (2022) · Annals of Translational Medicine · doi:10.21037/atm-22-1738 · Cited by 62
“Recent DCNN architectures have further demonstrated a comparable or even superior competence to dermatologists in disease classification, differentiation, and dermatopathological analysis (27-30).”
Cerminara, S. E., Cheng, P., Kostner, L., Huber, S., Kunz, M., Maul, J.-T., Böhm, J. S., Dettwiler, C. F., Geser, A., Jakopović, C., et al. (2023) · European Journal of Cancer · doi:10.1016/j.ejca.2023.112954 · Cited by 62
The role of technology in melanoma screening and diagnosis
Young, A. T., Vora, N. B., Cortez, J., Tam, A., Yeniay, Y., Afifi, L., Yan, D., Nosrati, A., Wong, A., Johal, A., et al. (2021) · Pigment Cell & Melanoma Research · doi:10.1111/pcmr.12907 · Cited by 60
Shaw, K., Henning, M. A., & Webster, C. S. (2025) · Medical Science Educator · doi:10.1007/s40670-025-02373-0 · Cited by 58
Effects of Label Noise on Deep Learning-Based Skin Cancer Classification
Hekler, A., Kather, J. N., Krieghoff-Henning, E., Utikal, J. S., Meier, F., Gellrich, F. F., Upmeier zu Belzen, J., French, L., Schlager, J. G., Ghoreschi, K., et al. (2020) · Frontiers in Medicine · doi:10.3389/fmed.2020.00177 · Cited by 58
Felmingham, C. M., Adler, N. R., Ge, Z., Morton, R. L., Janda, M., & Mar, V. J. (2021) · American Journal of Clinical Dermatology · doi:10.1007/s40257-020-00574-4 · Cited by 57
Metadata
- Authors
- Haenssle, H. A., Fink, C., Schneiderbauer, R., Toberer, F., Buhl, T., Blum, A., Kalloo, A., Hassen, A. B. H., Thomas, L., Enk, A., Uhlmann, L., Alt, C., Arenbergerova, M., Bakos, R., Baltzer, A., Bertlich, I., Blum, A., Bokor-Billmann, T., Bowling, J., Braghiroli, N., Braun, R., Buder-Bakhaya, K., Buhl, T., Cabo, H., Cabrijan, L., Cevic, N., Classen, A., Deltgen, D., Fink, C., Georgieva, I., Hakim-Meibodi, L.-E., Hanner, S., Hartmann, F., Hartmann, J., Haus, G., Hoxha, E., Karls, R., Koga, H., Kreusch, J., Lallas, A., Majenka, P., Marghoob, A., Massone, C., Mekokishvili, L., Mestel, D., Meyer, V., Neuberger, A., Nielsen, K., Oliviero, M., Pampena, R., Paoli, J., Pawlik, E., Rao, B., Rendon, A., Russo, T., Sadek, A., Samhaber, K., Schneiderbauer, R., Schweizer, A., Toberer, F., Trennheuser, L., Vlahova, L., Wald, A., Winkler, J., Wölbing, P., & Zalaudek, I.
- Affiliations
- Heidelberg University (DE)
- Memorial Sloan Kettering Cancer Center (US)
- University of Passau (DE)
- Lyon 1 Université (FR)
- Corresponding author
- Haenssle, H.A.
- ORCID
- Venue
- Annals of Oncology
- Publisher
- Elsevier BV
- Published
- 2018-08
- Volume
- 29
- Issue
- 8
- Pages
- 1836-1842
- ISSN
- 0923-7534 (ISSN-L)
- OpenAlex
- W2806853752
- Cited by
- 1,571
- References
- 0
- Citations by year
- 2026: 117 · 2025: 202 · 2024: 224 · 2023: 217 · 2022: 185 · 2021: 215 · 2020: 187 · 2019: 185 · 2018: 36
- FWCI
- 58.54
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by-nc-nd
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- www.elsevier.com/open-access/userlicense/1.0/ (vor, 450 d)
- Locations
- Annals of Oncology (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- ArTS Archivio della ricerca di Trieste (University of Trieste https://www.units.it/) (submittedVersion, other-oa)
- Lund University Publications (Lund University) (submittedVersion)
- GoeScholar The Publication Server of the Georg-August-Universität Göttingen (Georg-August-Universität Göttingen) (submittedVersion, other-oa)
- Topics
- Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- melanoma, dermoscopy, convolutional neural networks, Inception-v4, computer-aided diagnosis, skin lesion classification, sensitivity and specificity, clinical decision support, artificial intelligence in dermatology
- MeSH terms
- Dermatologists, Deep Learning, Clinical Competence, Cross-Sectional Studies, Humans, Image Processing, Computer-Assisted, International Cooperation, Melanoma, Retrospective Studies, ROC Curve, Skin, Skin Neoplasms, Dermoscopy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.