CoChat
Sign Up
Cited by 310Nail Diseases and TreatmentsAdvanced Neural Network ApplicationsArtificial Intelligence in Healthcare and Education

The DOI matches its Crossref record; 19 of the 19 references it cites were found in a registry or an index.

Deep neural networks show an equivalent and often superior performance to dermatologists in onychomycosis diagnosis: automatic construction of onychomycosis datasets by region-based convolutional deep neural network.

Open access
Han, Seung Seog, Park, Gyeong Hun, Lim, Woohyung, Kim, Myoung Shin, Na, Jung Im, Park, Ilwoo, Chang, Sung Eun·2018·PLoS ONE
DOI 10.1371/journal.pone.0191493PMC PMC5774804
Open this paper in CoChat — free accountOpen PDF

Abstract

Although there have been reports of the successful diagnosis of skin disorders using deep learning, unrealistically large clinical image datasets are required for artificial intelligence (AI) training. We created datasets of standardized nail images using a region-based convolutional neural network (R-CNN) trained to distinguish the nail from the background. We used R-CNN to generate training datasets of 49,567 images, which we then used to fine-tune the ResNet-152 and VGG-19 models. The validation datasets comprised 100 and 194 images from Inje University (B1 and B2 datasets, respectively), 125 images from Hallym University (C dataset), and 939 images from Seoul National University (D dataset). The AI (ensemble model; ResNet-152 + VGG-19 + feedforward neural networks) results showed test sensitivity/specificity/ area under the curve values of (96.0 / 94.7 / 0.98), (82.7 / 96.7 / 0.95), (92.3 / 79.3 / 0.93), (87.7 / 69.3 / 0.82) for the B1, B2, C, and D datasets. With a combination of the B1 and C datasets, the AI Youden index was significantly (p = 0.01) higher than that of 42 dermatologists doing the same assessment manually. For B1+C and B2+ D dataset combinations, almost none of the dermatologists performed as well as the AI. By training with a dataset comprising 49,567 images, we achieved a diagnostic accuracy for onychomycosis using deep learning that was superior to that of most of the dermatologists who participated in this study.

Overview

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1371/journal.pone.0191493

Cites

19 of 19 references verified0 citation errors · 0 not checkable

Statistics

Figures

Fig 1 Automated establishment of the A1 dataset using R-CNN and two CNNs. 1) CNN hand & foot image selector which automatically detects and selects hand and foot images from the entire dataset. 2) R-CNN nail part extractor which identifies and crops part of the nail from individual clinical images by differentiating the nail plate from the background. 3) CNN fine image selector which rules out photographs with inadequate composition or focus.
Fig 1 Automated establishment of the A1 dataset using R-CNN and two CNNs. 1) CNN hand & foot image selector which automatically detects and selects hand and foot images from the entire dataset. 2) R-CNN nail part extractor which identifies and crops part of the nail from individual clinical images by differentiating the nail plate from the background. 3) CNN fine image selector which rules out photographs with inadequate composition or focus.CC BY · PubMed Central
Fig 2 Examples of adequate, inadequate, and wrong images generated by R-CNN. (A) Adequate onychomycosis images; a nail plate occupies around 50~80% of the whole picture. (B) Inadequate nail plate images which include non-standardized nail images such as 1) a partially cut nail, 2) tilted nail, 3) a nail with a foreign substance, 4) unfocussed nail, 5) nail plates that are too small or too large to identify surrounding skin, and 6) image containing two or more nail plates. (C) Wrong images that included many general objects detected by R-CNN based on the difference between the nail and the background, but not corresponding to either. Objects that had a similar shape to a nail, such as teeth or warts were also generated. A large number of fingers and toes without a nail plate also came out.
Fig 2 Examples of adequate, inadequate, and wrong images generated by R-CNN. (A) Adequate onychomycosis images; a nail plate occupies around 50~80% of the whole picture. (B) Inadequate nail plate images which include non-standardized nail images such as 1) a partially cut nail, 2) tilted nail, 3) a nail with a foreign substance, 4) unfocussed nail, 5) nail plates that are too small or too large to identify surrounding skin, and 6) image containing two or more nail plates. (C) Wrong images that included many general objects detected by R-CNN based on the difference between the nail and the background, but not corresponding to either. Objects that had a similar shape to a nail, such as teeth or warts were also generated. A large number of fingers and toes without a nail plate also came out.CC BY · PubMed Central
Fig 3 The representation of model architecture image for ResNet-152, VGG-19, and two-layered feedforward neural networks. 1) VGG-19 is a CNN model that have shown the importance of depth of the network in model performance. Using the simplest 3x3 convolution kernel throughout the whole network, VGG-19 won the ILSVRC in 2014. 2) ResNet-152 introduced the concept of residual learning in which the subtraction of feature is learned from the input of that layer by using shortcut connections (directly connecting input of (n)th layer to some (n+x)th layer, which is shown as curved arrow). It has proven that the residual learning can improve the performance of model training, especially when the model has deep network with more than 20 layers, and also revolve the problem of degrading accuracy in deep networks. 3) A total of 12 outputs from ResNet-152 and VGG-19 were used as input in a two-hidden-layered feedforward neural network. The final outputs (onychomycosis or not) were computed through the two hidden layers. ; fc is an abbreviation for fully connected. The numbers at each layer represent the number of nodes at each layer. ; Fig 3 was created based on the figure from the Ref #20. [20].
Fig 3 The representation of model architecture image for ResNet-152, VGG-19, and two-layered feedforward neural networks. 1) VGG-19 is a CNN model that have shown the importance of depth of the network in model performance. Using the simplest 3x3 convolution kernel throughout the whole network, VGG-19 won the ILSVRC in 2014. 2) ResNet-152 introduced the concept of residual learning in which the subtraction of feature is learned from the input of that layer by using shortcut connections (directly connecting input of (n)th layer to some (n+x)th layer, which is shown as curved arrow). It has proven that the residual learning can improve the performance of model training, especially when the model has deep network with more than 20 layers, and also revolve the problem of degrading accuracy in deep networks. 3) A total of 12 outputs from ResNet-152 and VGG-19 were used as input in a two-hidden-layered feedforward neural network. The final outputs (onychomycosis or not) were computed through the two hidden layers. ; fc is an abbreviation for fully connected. The numbers at each layer represent the number of nodes at each layer. ; Fig 3 was created based on the figure from the Ref #20. [20].CC BY · PubMed Central
Fig 4 Evaluation of sensitivity/specificity in diagnosing onychomycosis and nail dystrophy for the B1 and C test datasets. (A) B1 test dataset: 100 images (50 onychomycosis and 50 nail dystrophy) (B) C test dataset: 125 images (57 onychomycosis and 68 nail dystrophy) (C) B1 + C test dataset: 225 images (107 onychomycosis and 118 nail dystrophy) Black curve: Ensemble model (ResNet-152 + VGG-19 + feedforward neural networks) trained with the A1 dataset; Gray curve: ResNet-152 trained with the A1 dataset; Red curve: ResNet-152 trained with the A2 dataset; Green curve: VGG-19 trained with the A1 dataset; Yellow curve: ResNet-152 trained with the E dataset; Red dot: 34 dermatologists (16 professors and 18 clinicians); Green dot: 8 dermatologists (8 residents); Yellow dot: 11 general practitioners, 13 medical students, 15 nurses in the dermatology department, and 18 non-medical personnel; Blue line: Youden index (sensitivity + specificity − 1) of 42 dermatologists with 99% confidence interval (CI); Blue dotted line: Youden index of 42 dermatologists with a 95% CI.
Fig 4 Evaluation of sensitivity/specificity in diagnosing onychomycosis and nail dystrophy for the B1 and C test datasets. (A) B1 test dataset: 100 images (50 onychomycosis and 50 nail dystrophy) (B) C test dataset: 125 images (57 onychomycosis and 68 nail dystrophy) (C) B1 + C test dataset: 225 images (107 onychomycosis and 118 nail dystrophy) Black curve: Ensemble model (ResNet-152 + VGG-19 + feedforward neural networks) trained with the A1 dataset; Gray curve: ResNet-152 trained with the A1 dataset; Red curve: ResNet-152 trained with the A2 dataset; Green curve: VGG-19 trained with the A1 dataset; Yellow curve: ResNet-152 trained with the E dataset; Red dot: 34 dermatologists (16 professors and 18 clinicians); Green dot: 8 dermatologists (8 residents); Yellow dot: 11 general practitioners, 13 medical students, 15 nurses in the dermatology department, and 18 non-medical personnel; Blue line: Youden index (sensitivity + specificity − 1) of 42 dermatologists with 99% confidence interval (CI); Blue dotted line: Youden index of 42 dermatologists with a 95% CI.CC BY · PubMed Central
Fig 5 Evaluation of sensitivity/specificity in diagnosing onychomycosis and nail dystrophy for the B2 and D test datasets. (A) B2 test dataset: 194 images (173 onychomycosis and 21 nail dystrophy) (B) D test dataset: 939 images (500 onychomycosis and 439 nail dystrophy) (C) B2 + D test dataset: 1133 images (673 onychomycosis and 460 nail dystrophy) Black curve: Ensemble model (ResNet-152 + VGG-19 + feedforward neural networks) trained with the A1 dataset; Gray curve: ResNet-152 trained with the A1 dataset; Red curve: ResNet-152 trained with the A2 dataset; Green curve: VGG-19 trained with the A1 dataset; Yellow curve: ResNet-152 trained with the E dataset.
Fig 5 Evaluation of sensitivity/specificity in diagnosing onychomycosis and nail dystrophy for the B2 and D test datasets. (A) B2 test dataset: 194 images (173 onychomycosis and 21 nail dystrophy) (B) D test dataset: 939 images (500 onychomycosis and 439 nail dystrophy) (C) B2 + D test dataset: 1133 images (673 onychomycosis and 460 nail dystrophy) Black curve: Ensemble model (ResNet-152 + VGG-19 + feedforward neural networks) trained with the A1 dataset; Gray curve: ResNet-152 trained with the A1 dataset; Red curve: ResNet-152 trained with the A2 dataset; Green curve: VGG-19 trained with the A1 dataset; Yellow curve: ResNet-152 trained with the E dataset.CC BY · PubMed Central
Fig 6 An example of nail images with the adjusted brightness and noise in 10 levels. (A) Noise (Gaussian); (B) Noise (Speckle); (C) Brightness (Black); (D) Brightness (White).
Fig 6 An example of nail images with the adjusted brightness and noise in 10 levels. (A) Noise (Gaussian); (B) Noise (Speckle); (C) Brightness (Black); (D) Brightness (White).CC BY · PubMed Central
Fig 7 The change in the AUC of onychomycosis classification (ensemble model; test = B1 + C dataset) and the adequate value of the fine image selector. Fine image selector is a CNN model (ResNet-152) trained to analyze photos, categorize into adequate, inadequate, or wrong image, and produce an adequate value as an output. As the amount of added brightness and noise increased, the AUC (Test = B1 and C dataset), which represents the classification accuracy, and the adequate value of the fine image selector decreased.
Fig 7 The change in the AUC of onychomycosis classification (ensemble model; test = B1 + C dataset) and the adequate value of the fine image selector. Fine image selector is a CNN model (ResNet-152) trained to analyze photos, categorize into adequate, inadequate, or wrong image, and produce an adequate value as an output. As the amount of added brightness and noise increased, the AUC (Test = B1 and C dataset), which represents the classification accuracy, and the adequate value of the fine image selector decreased.CC BY · PubMed Central

Verification

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Citations

Cites

Show all 18 references

Research works printed but not linked: 1

  1. He K, Zhang X, Ren S, Sun J, editors. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition; 2016. · DOI · Paper page

Cited by

Sorted by citations

Related

See all

Metadata

Authors
Han, S. S., Park, G. H., Lim, W., Kim, M. S., Na, J. I., Park, I., & Chang, S. E.
Affiliations
  • Hallym University (KR)
  • Inje University (KR)
  • Seoul National University (KR)
  • Chonnam National University (KR)
  • Asan Medical Center (KR)
Corresponding author
Chang, Sung Eun
ORCID
Venue
PLoS ONE
Publisher
Public Library of Science (PLoS)
Published
2018-01-19
Volume
13
Issue
1
Pages
e0191493
ISSN
1932-6203 (ISSN-L)
DOI
10.1371/journal.pone.0191493
PMC
PMC5774804
OpenAlex
W2785200097
Cited by
305
References
18
Citations by year
2026: 24 · 2025: 32 · 2024: 40 · 2023: 33 · 2022: 58 · 2021: 37 · 2020: 49 · 2019: 24 · 2018: 7
FWCI
22.75
Type
journal article
Language
en
Open access
Licence
Locations
  • PLoS ONE (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
Topics
  • Health Sciences › Medicine › Dermatology › Nail Diseases and Treatments
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
onychomycosis, automated diagnosis, dermatologist comparison, R-CNN, ResNet-152, VGG19, ensemble learning, dermatological image analysis
MeSH terms
Machine Learning, Dermatologists, Adult, Aged, Algorithms, Artificial Intelligence, Diagnosis, Computer-Assisted, Female, Foot Dermatoses, Hand Dermatoses, Humans, Image Interpretation, Computer-Assisted, Male, Middle Aged, Onychomycosis, Databases, Factual, Neural Networks, Computer, Area Under Curve, Young Adult

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review