Cited by 5,759AI in cancer detectionAdvanced Neural Network ApplicationsRadiomics and Machine Learning in Medical Imaging

The DOI matches its Crossref record; 78 of the 104 references it cites were found in a registry or an index; none of the 71 sources its claims cite is checked for support yet.

Deep Convolutional Neural Networks for Computer-Aided Detection: CNN Architectures, Dataset Characteristics and Transfer Learning

Open access

Abstract

Remarkable progress has been made in image recognition, primarily due to the availability of large-scale annotated datasets and deep convolutional neural networks (CNNs). CNNs enable learning data-driven, highly representative, hierarchical image features from sufficient training data. However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge. There are currently three major techniques that successfully employ CNNs to medical image classification: training the CNN from scratch, using off-the-shelf pre-trained CNN features, and conducting unsupervised CNN pre-training with supervised fine-tuning. Another effective method is transfer learning, i.e., fine-tuning CNN models pre-trained from natural image dataset to medical image tasks. In this paper, we exploit three important, but previously understudied factors of employing deep convolutional neural networks to computer-aided detection problems. We first explore and evaluate different CNN architectures. The studied models contain 5 thousand to 160 million parameters, and vary in numbers of layers. We then evaluate the influence of dataset scale and spatial image context on performance. Finally, we examine when and why transfer learning from pre-trained ImageNet (via fine-tuning) can be useful. We study two specific computer-aided detection (CADe) problems, namely thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. We achieve the state-of-the-art performance on the mediastinal LN detection, and report the first five-fold cross-validation classification results on predicting axial CT slices with ILD categories. Our extensive empirical evaluation, CNN model analysis and valuable insights can be extended to the design of high performance CAD systems for other medical imaging tasks.

Overview

AI snapshot

Written from the paper’s full text.

TL;DR

This paper empirically evaluates CNN architectures, dataset characteristics, and ImageNet transfer learning for two computer-aided detection tasks: thoraco-abdominal lymph node (LN) detection and interstitial lung disease (ILD) classification. It reports state-of-the-art mediastinal LN detection and the first five-fold cross-validation ILD slice classification results, finding that transfer learning and end-to-end training generally outperform off-the-shelf CNN features.

Key findings

  • For mediastinal LN detection, GoogLeNet-RI-L achieved AUC = 0.95 and 0.85 TPR/3FP, surpassing prior results of AUC = 0.92 and 0.70 TPR/3FP [22] and 0.78 TPR/3FP [36].
  • Transfer learning from ImageNet consistently improved performance: GoogLeNet-TL-H versus GoogLeNet-RI-H gave 0.81 versus 0.61 TPR/3FP in mediastinum and 0.70 versus 0.48 TPR/3FP in abdomen; AlexNet-TL-H versus AlexNet-RI-H gave 0.81 versus 0.79 TPR/3FP in mediastinum and 0.69 versus 0.67 TPR/3FP in abdomen.
  • For ILD slice-level five-fold CV, GoogLeNet-TL reached 0.57 accuracy versus GoogLeNet-RI 0.41, and AlexNet-TL reached 0.46 versus AlexNet-RI 0.44; patch-level ILD classification was less challenging, with five deep models in the range [0.74, 0.76] and an averaged model at 0.79.
  • CNNs trained from scratch or fine-tuned from ImageNet consistently outperformed CNNs using only off-the-shelf CNN features in both LN and ILD problems.

Methods

  • The study evaluated CifarNet, AlexNet, and GoogLeNet architectures (later also OverFeat and VGGNet) on two CADe tasks: thoraco-abdominal LN detection using 388 mediastinal LNs in 90 patient CT scans and 595 abdominal LNs in 86 patient CT scans, and ILD classification using 905 annotated image slices from 120 patients with six ILD labels.
  • LN detection used randomly sampled 2.5D views with fields-of-view spanning 35–128 mm, rescaled to 64×64 pixels, and three-fold cross-validation with FROC AUC and TPR/3FP metrics; ILD classification used slice-level and patch-level tasks under patient-level five-fold cross-validation, with CT windowing encoded into RGB channels and data augmentation.
  • Training used the Caffe framework and NVidia K40 GPUs; models trained from scratch used random Gaussian initialization for 30 epochs with mini-batch size 50, while transfer learning fine-tuned all layers except the last at a learning rate 10 times smaller than default, with the last fully connected layer randomly initialized and trained at 0.01.

Limitations

  • The paper states that the ILD dataset is partially labeled, with very few CT slices labeled as normal or healthy, and that this is one of its main limitations; it also states that the dataset’s limited patient population of 120 patients may not adequately represent ILD disease CT lung imaging patterns.
  • The paper states that slice-level CNN learning is restricted to only 905 CT image slices, that no existing work has reached performance requirements for a realistic clinical setting, and that the most critical bottlenecks for ILD classification are cross-dataset learning and limited patient population size.
  • Evident from the reported results: ILD slice-level CV5 accuracies were low (0.41–0.57), and LOO validation produced much higher accuracies than CV5 (e.g., AlexNet-TL 0.867 versus 0.46; GoogLeNet-TL 0.902 versus 0.57), indicating sensitivity to evaluation protocol; also, for abdominal LN detection the best reported CNN result (0.70 TPR/3FP) did not surpass the prior [22] result of 0.83 TPR/3FP.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.1109/tmi.2016.2528162

Cites

78 of 78 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Fig. 1. Some examples of abdominal and mediastinal lymph nodes sampled on axial (ax), coronal (co), and sagittal (sa) views, with four different fields-of-views (30 mm: orange; 45 mm: red; 85 mm: green; 128 mm: blue) surrounding lymph nodes.
Fig. 1. Some examples of abdominal and mediastinal lymph nodes sampled on axial (ax), coronal (co), and sagittal (sa) views, with four different fields-of-views (30 mm: orange; 45 mm: red; 85 mm: green; 128 mm: blue) surrounding lymph nodes.CC BY · PubMed Central
Fig. 2. Some examples of CT image slices with six lung tissue types in the ILD dataset [37]. Disease tissue types are located with dark orange arrows. (a): healthy; (b): emphysema; (c): ground glass; (d): fibrosis; (e): micronodules; (f): consolidation.
Fig. 2. Some examples of CT image slices with six lung tissue types in the ILD dataset [37]. Disease tissue types are located with dark orange arrows. (a): healthy; (b): emphysema; (c): ground glass; (d): fibrosis; (e): micronodules; (f): consolidation.CC BY · PubMed Central
Fig. 3. Some examples of }{}$64\times 64~{\rm pixel}$ CT image patches for (a) Nm, (b) Em, (c) Gg, (d) Fb, (e) MN (f) CD.
Fig. 3. Some examples of }{}$64\times 64~{\rm pixel}$ CT image patches for (a) Nm, (b) Em, (c) Gg, (d) Fb, (e) MN (f) CD.CC BY · PubMed Central
Fig. 4. An example of lung/high-attenuation/low-attenuation CT windowing for an axis lung CT slice. We encode the lung/high-attenuation/low-attenuation CT windowing into red/green/blue channels.
Fig. 4. An example of lung/high-attenuation/low-attenuation CT windowing for an axis lung CT slice. We encode the lung/high-attenuation/low-attenuation CT windowing into red/green/blue channels.CC BY · PubMed Central
Fig. 5. A simplified illustration of the CNN architectures used. Googlenet [33] contains two convolution layers, three pooling layers, and nine inception layers. Each of the inception layer of googlenet consists of six convolution layers and one pooling layer.
Fig. 5. A simplified illustration of the CNN architectures used. Googlenet [33] contains two convolution layers, three pooling layers, and nine inception layers. Each of the inception layer of googlenet consists of six convolution layers and one pooling layer.CC BY · PubMed Central
Fig. 6. Illustration of }{}${\tt inception3a}$ layer of googlenet. Inception layers of googlenet consist of six convolution layers with different kernel sizes and one pooling layer.
Fig. 6. Illustration of }{}${\tt inception3a}$ layer of googlenet. Inception layers of googlenet consist of six convolution layers with different kernel sizes and one pooling layer.CC BY · PubMed Central
Fig. 7. Some examples of cifar10 dataset and some images of “tennis ball” class from imagenet dataset. Images of cifar10 dataset are small (32×32) images with object of the image class category in the center. Images of imagenet dataset are larger (256×256), where object of the image class category can be small, obscure, partial, and sometimes in a cluttered environment.
Fig. 7. Some examples of cifar10 dataset and some images of “tennis ball” class from imagenet dataset. Images of cifar10 dataset are small (32×32) images with object of the image class category in the center. Images of imagenet dataset are larger (256×256), where object of the image class category can be small, obscure, partial, and sometimes in a cluttered environment.CC BY · PubMed Central
Fig. 8. FROC curves averaged on three-fold CV for the abdominal (left) and mediastinal (right) lymph nodes using different CNN models.
Fig. 8. FROC curves averaged on three-fold CV for the abdominal (left) and mediastinal (right) lymph nodes using different CNN models.CC BY · PubMed Central
Fig. 9. Examples of misclassified lymph nodes (in axial view) of both false negatives (left) and false positives (right). Mediastinal LN examples are shown in the upper row, and abdominal LN examples in the bottom row.
Fig. 9. Examples of misclassified lymph nodes (in axial view) of both false negatives (left) and false positives (right). Mediastinal LN examples are shown in the upper row, and abdominal LN examples in the bottom row.CC BY · PubMed Central
Fig. 10. Visual examples of misclassified ILD 64×64 patches (in axial view), with their ground truth labels and inaccurately classified labels.
Fig. 10. Visual examples of misclassified ILD 64×64 patches (in axial view), with their ground truth labels and inaccurately classified labels.CC BY · PubMed Central
Fig. 11. Traces of training and validation loss (blue and green lines) and validation accuracy (orange lines) during (a) training alexnet from random initialization and (b) fine-tuning from imagenet pre-trained cnn, for ILD classification.
Fig. 11. Traces of training and validation loss (blue and green lines) and validation accuracy (orange lines) during (a) training alexnet from random initialization and (b) fine-tuning from imagenet pre-trained cnn, for ILD classification.CC BY · PubMed Central
Fig. 12. Visualization of first layer convolution filters of CNNs trained on abdominal and mediastinal LNs in RGB color, from random initialization (alexnet-RI (256×256), alexnet-RI (64×64), googlenet-RI (256×256) and googlenet-RI (64×64)) and with transfer learning (alexnet-TL (256×256)).
Fig. 12. Visualization of first layer convolution filters of CNNs trained on abdominal and mediastinal LNs in RGB color, from random initialization (alexnet-RI (256×256), alexnet-RI (64×64), googlenet-RI (256×256) and googlenet-RI (64×64)) and with transfer learning (alexnet-TL (256×256)).CC BY · PubMed Central
Fig. 13. Visualization of the last pooling layer (pool-5) activations (top). Pooling units where the relative image location of the disease region is located in the image are highlighted with green boxes. The original images reconstructed from the units are shown in the bottom [72]. The examples in (a) and (b) are computed from the input ILD images in Figs. 2(b) and 2(c), respectively.
Fig. 13. Visualization of the last pooling layer (pool-5) activations (top). Pooling units where the relative image location of the disease region is located in the image are highlighted with green boxes. The original images reconstructed from the units are shown in the bottom [72]. The examples in (a) and (b) are computed from the input ILD images in Figs. 2(b) and 2(c), respectively.CC BY · PubMed Central

Verification

Claims the paper makes

49 Not checked yet · 31 Own result · 25 Own interpretation · 14 No basis given · 5 Sources not found · 2 Own method

Verify all 64 references — free account

Show all 72 claims

  • It may state two separate claims as one. 32
  • Probably not a substantial claim. 28
  • The quote only partly states this claim. 3
  • Borderline: one of the tests for a substantial claim is unclear. 1
54 of 54
  • The paper relies on this without support.

    Fine-tuning an ImageNet pre-trained CNN model on medical image datasets had not yet been exploited at the time of the study.

    “However, the fine-tuning of an ImageNet pre-trained CNN model on medical image datasets has not yet been exploited.”
  • The paper relies on this without support.

    Obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge.

    “However, obtaining datasets as comprehensively annotated as ImageNet in the medical imaging domain remains a challenge.”
  • The paper relies on this without support.

    Some areas outside the lung appear in both healthy or diseased images.

    “Some areas outside of the lung appear in both healthy or diseased images.”
  • The paper relies on this without support.

    The low-attenuation CT window is useful for visualizing certain texture patterns of lung diseases, especially emphysema.

    “The low-attenuation CT window is useful for visualizing certain texture patterns of lung diseases (especially emphysema).”
  • The paper relies on this without support.

    Collecting and annotating large numbers of medical images still poses significant challenges.

    “Collecting and annotating large numbers of medical images still poses significant challenges.”
  • The authors state this as an assumption.

    OverFeat and VGGNet may generate quantitative performance results ranked between AlexNet's and GoogLeNet's.

    “Likewise, a reasonable assumption is that OverFeat and VGGNet may generate quantitative performance results ranked between AlexNet's and GoogLeNet's.”
  • The authors state this as an assumption.

    OverFeat's and VGGNet's performance levels fall between AlexNet's and GoogLeNet's.

    “The reported results validate our assumption that OverFeat's and VGGNet's performance levels fall between AlexNet's and GoogLeNet's (this observation is consistent with the computer vision findings).”
  • The paper relies on this without support.

    Both the ILD and ImageNet datasets use full-size images.

    “Indeed, both datasets use full-size images.”
  • The paper relies on this without support.

    CifarNet is designed for images with smaller dimensions (32×32 images), and thus is not catered to classification tasks involving 256×256 images.

    “CifarNet is designed for images with smaller dimensions (32×32 images), and thus is not catered to classification tasks involving 256×256 images.”
  • The paper relies on this without support.

    Medical datasets are often biased, in that the number of healthy samples is much larger than the number of diseased instances, or that the numbers of images per class are uneven.

    “Medical datasets are often “biased”, in that the number of healthy samples is much larger than the number of diseased instances, or that the numbers of images per class are uneven.”
10 of 10

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

4 of 4

Citations

Cites

Show all 104 references

Research works printed but not linked: 3

  1. seff, 2015, Leveraging mid-level semantic boundary cues for computer-aided lymph node detection, Proc MICCAI
  2. kim, 2014, Performance assessment of retroperitoneal lymph node computer-assisted detection using random forest and deep convolutional neural network learning algorithms in tandem, 102nd Annu Meet Radiol Soc N Am
  3. sermanet, 2014, Overfeat: Integrated recognition, localization and detection using convolutional networks, Proc ICLR

Cited by

Sorted by citations

Items 201–300 of 1,020

Metadata

Authors
Shin, H.-C., Roth, H. R., Gao, M., Lu, L., Xu, Z., Nogues, I., Yao, J., Mollura, D., & Summers, R. M.
Affiliations
National Institutes of Health (US)
Corresponding author
Shin, Hoo-Chang
Venue
IEEE Transactions on Medical Imaging
Published
2016-05
Volume
35
Issue
5
Pages
1285-1298
ISSN
  • 0278-0062 (ISSN-L)
  • 1558-254X
OpenAlex
W2253429366
Cited by
5,759
References
104
Citations by year
2026: 279 · 2025: 480 · 2024: 572 · 2023: 614 · 2022: 725 · 2021: 872 · 2020: 808 · 2019: 712 · 2018: 426 · 2017: 223 · 2016: 44
FWCI
385.74
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Medical Imaging (publishedVersion)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
  • arXiv (Cornell University) (submittedVersion)
  • arXiv (Cornell University)
Funders
  • National Institutes of Health
  • NIH Clinical Center (Z01CL040004)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
computer-aided detection, deep convolutional neural networks, transfer learning, CNN architecture comparison, medical image classification, ImageNet pre-training, fine-tuning, lymph node detection, interstitial lung disease classification, computed tomography
MeSH terms
Diagnosis, Computer-Assisted, Humans, Image Interpretation, Computer-Assisted, Lymph Nodes, Reproducibility of Results, Databases, Factual, Neural Networks, Computer, Lung Diseases, Interstitial

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review