CoChat
Sign Up
Cited by 70Cutaneous Melanoma Detection and ManagementArtificial Intelligence in Healthcare and EducationAI in cancer detection

Performance of a deep neural network in teledermatology: a single‐centre prospective diagnostic study

Open access
Muñoz‐López, C., Ramírez‐Cornejo, C., Marchetti, M.A., Han, S. S., Del Barrio‐Díaz, P., Jaque, A., Uribe, P., Majerson, D. +12 more·2021·Journal of the European Academy of Dermatology and Venereology
DOI 10.1111/jdv.16979PMC PMC8274350
Open this paper in CoChat — free account

Abstract

Background: The use of artificial intelligence (AI) algorithms for the diagnosis of skin diseases has shown promise in experimental settings but has not been yet tested in real‐life conditions. Objective: To assess the diagnostic performance and potential clinical utility of a 174‐multiclass AI algorithm in a real‐life telemedicine setting. Methods: Prospective, diagnostic accuracy study including consecutive patients who submitted images for teledermatology evaluation. The treating dermatologist chose a single image to upload to a web application during teleconsultation. A follow‐up reader study including nine healthcare providers (3 dermatologists, 3 dermatology residents and 3 general practitioners) was performed. Results: A total of 340 cases from 281 patients met study inclusion criteria. The mean (SD) age of patients was 33.7 (17.5) years; 63% (n = 177) were female. Exposure to the AI algorithm results was considered useful in 11.8% of visits (n = 40) and the teledermatologist correctly modified the real‐time diagnosis in 0.6% (n = 2) of cases. The overall top‐1 accuracy of the algorithm (41.2%) was lower than that of the dermatologists (60.1%), residents (57.8%) and general practitioners (49.3%) (all comparisons P < 0.05, in the reader study). When the analysis was limited to the diagnoses on which the algorithm had been explicitly trained, the balanced top‐1 accuracy of the algorithm (47.6%) was comparable to the dermatologists (49.7%) and residents (47.7%) but superior to the general practitioners (39.7%;P = 0.049). Algorithm performance was associated with patient skin type and image quality. Conclusions: A 174‐disease class AI algorithm appears to be a promising tool in the triage and evaluation of lesions with patient‐taken photographs via telemedicine.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1111/jdv.16979

Cites

8 of 8 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 17

  1. [2]Aractingi S, Pellacani G. Computational neural network in melanocytic lesions diagnosis: artificial intelligence to improve diagnosis in dermatology? Eur J Dermatol 2019; 29: 4– 7. 31017580
  2. [3]Dick V, Sinz C, Mittlböck M, Kittler H, Tschandl P. Accuracy of computer-aided diagnosis of melanoma: a meta-analysis. JAMA Dermatology. 2019; 155: 1291– 1299. 31215969 PMC6584889
  3. [4]Brinker TJ, Hekler A, Enk AH, Klode J, Hauschild A, Berking C A convolutional neural network trained with dermoscopic images performed on par with 145 dermatologists in a clinical melanoma image classification task. Eur J Cancer 2019; 111: 148– 154. 30852421
  4. [5]Dreiseitl S, Binder M, Hable K, Kittler H. Computer versus human diagnosis of melanoma: evaluation of the feasibility of an automated diagnostic system in a prospective clinical trial. Melanoma Res 2009; 19: 180– 184. 19369900
  5. [6]Smith AC, Thomas E, Snoswell CL, Haydon H, Mehrotra A, Clemensen J. Telehealth for global emergencies: Implications for coronavirus disease 2019 (COVID-19). J Telemed Telecare 2019; 2020: 2019. PMC7140977 32196391
  6. [8]Ohannessian R, Duong TA, Odone A. Global telemedicine implementation and integration within health systems to fight the COVID-19 pandemic: a call to action. JMIR Public Heal Surveill 2020; 6: e18810. PMC7124951 32238336
  7. [9]Lee JJ, English JC. Teledermatology: a review and update. Am J Clin Dermatol 2018; 19: 253– 260. 28871562
  8. [13]Liu Y, Jain A, Eng C, Way DH, Lee K, Bui P A deep learning system for differential diagnosis of skin diseases. Nat Med 2020; 26(6): 900– 908. http://www.nature.com/articles/s41591-020-0842-3 32424212
  9. [14]Tschandl P, Kittler H, Argenziano G. A pretrained neural network shows similar diagnostic accuracy to medical students in categorizing dermatoscopic images after comparable training conditions. Br J Dermatol 2017; 177: 867– 869. 28569993
  10. [15]Haenssle HA, Fink C, Schneiderbauer R, Toberer F, Buhl T, Blum A Man against Machine: Diagnostic performance of a deep learning convolutional neural network for dermoscopic melanoma recognition in comparison to 58 dermatologists. Ann Oncol 2018; 29: 1836– 1842. 29846502
  11. [16]Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM Dermatologist-level classification of skin cancer with deep neural networks. Nature 2017; 542: 115– 118. 28117445 PMC8382232
  12. [17]Marchetti MA, Codella NCF, Dusza SW, Gutman DA, Helba B, Kalloo A Results of the 2016 International Skin Imaging Collaboration International Symposium on Biomedical Imaging challenge: Comparison of the accuracy of computer algorithms to dermatologists for the diagnosis of melanoma from dermoscopic images. J Am Acad Dermatol 2018; 78: 270– 277.e1. 28969863 PMC5768444
  13. [18]Tschandl P, Rinner C, Apalla Z, Argenziano G, Codella N. Human – computer collaboration for skin cancer recognition. Nat Med 2020; 26(8): 1229– 1234. 10.1038/s41591-020-0942-0 32572267 · DOI · Paper page
  14. [20]Hulstaert E, Hulstaert L. Artificial intelligence in dermato-oncology: a joint clinical and data science perspective. Int J Dermatol 2019; 58: 989– 990. 31149729
  15. [23]Navarrete-Dechent C, Liopyris K, Molenda MA Human surface anatomy terminology for dermatology: a Delphi consensus from the International Skin Imaging Collaboration. J Eur Acad Dermatol Venereol 2020; 34: 2659– 2663. 32770737 PMC8236240
  16. [24]Hollander JE, Carr BG. Virtually Perfect? Telemedicine for Covid-19. N Engl J Med 2020; 382 18: 1679– 1681. 32160451
  17. [25]Webster P. Virtual health care in the era of COVID-19. The Lancet 2020; 395: 1180– 1181. PMC7146660 32278374

Cited by

Sorted by citations

Authors
Muñoz‐López, C., Ramírez‐Cornejo, C., Marchetti, M. A., Han, S. S., Del Barrio‐Díaz, P., Jaque, A., Uribe, P., Majerson, D., Curi, M., Del Puerto, C., Reyes‐Baraona, F., Meza‐Romero, R., Parra‐Cares, J., Araneda‐Ortega, P., Guzmán, M., Millán‐Apablaza, R., Nuñez‐Mora, M., Liopyris, K., Vera‐Kellet, C., & Navarrete‐Dechent, C.
Affiliations
  • Department of Dermatology Escuela de Medicina Pontificia Universidad Católica de Chile Santiago Chile
  • Pontificia Universidad Católica de Chile (CL)
  • Memorial Sloan Kettering Cancer Center (US)
  • Dermatology Clinic Seoul Korea
  • National and Kapodistrian University of Athens (GR)
Corresponding author
Navarrete‐Dechent, C.
ORCID
Venue
Journal of the European Academy of Dermatology and Venereology
Publisher
Wiley
Published
2020-11-22
Volume
35
Issue
2
Pages
546-553
ISSN
  • 0926-9959 (ISSN-L)
  • 1468-3083
DOI
10.1111/jdv.16979
PMC
PMC8274350
OpenAlex
W3092516577
Cited by
70
References
8
Citations by year
2026: 9 · 2025: 13 · 2024: 15 · 2023: 13 · 2022: 12 · 2021: 7 · 2020: 1
FWCI
2.98
Type
journal article
Language
en
Open access
Licence
onlinelibrary.wiley.com/termsAndConditions#vor (vor)
Locations
  • Journal of the European Academy of Dermatology and Venereology (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Cancer Research Institute (P30 CA008748)
Topics
  • Health Sciences › Medicine › Oncology › Cutaneous Melanoma Detection and Management
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
Keywords
teledermatology, deep neural network, dermatological diagnosis, artificial intelligence, telemedicine, diagnostic accuracy, dermatologist comparison, general practitioners, skin type, image quality
MeSH terms
Adult, Artificial Intelligence, Dermatology, Female, Humans, Male, Prospective Studies, Skin Diseases, Neural Networks, Computer, Telemedicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review