The Performance of Deep Learning Algorithms on Automatic Pulmonary Nodule Detection and Classification Tested on Different Datasets That Are Not Derived from LIDC-IDRI: A Systematic Review
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
The aim of this study was to systematically review the performance of deep learning technology in detecting and classifying pulmonary nodules on computed tomography (CT) scans that were not from the Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) database. Furthermore, we explored the difference in performance when the deep learning technology was applied to test datasets different from the training datasets. Only peer-reviewed, original research articles utilizing deep learning technology were included in this study, and only results from testing on datasets other than the LIDC-IDRI were included. We searched a total of six databases: EMBASE, PubMed, Cochrane Library, the Institute of Electrical and Electronics Engineers, Inc. (IEEE), Scopus, and Web of Science. This resulted in 1782 studies after duplicates were removed, and a total of 26 studies were included in this systematic review. Three studies explored the performance of pulmonary nodule detection only, 16 studies explored the performance of pulmonary nodule classification only, and 7 studies had reports of both pulmonary nodule detection and classification. Three different deep learning architectures were mentioned amongst the included studies: convolutional neural network (CNN), massive training artificial neural network (MTANN), and deep stacked denoising autoencoder extreme learning machine (SDAE-ELM). The studies reached a classification accuracy between 68–99.6% and a detection accuracy between 80.6–94%. Performance of deep learning technology in studies using different test and training datasets was comparable to studies using same type of test and training datasets. In conclusion, deep learning was able to achieve high levels of accuracy, sensitivity, and/or specificity in detecting and/or classifying nodules when applied to pulmonary CT scans not from the LIDC-IDRI database.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277 · cited 1×
“…of this tendency, studies have had success with training their algorithms through pre-training [55], transfer learning [56], and / or fine-tuning [57] to bypass the problem of a small training dataset, in addition to developing variations of algorithms that are based on other deep learning…”
BackgroundLearning and Transferring Mid-level Image Representations Using Convolutional Neural Networks
Oquab, M., Bottou, L., Laptev, I., & Sivic, J. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.222 · Cited by 3,150 · cited 1×
“Because of this tendency, studies have had success with training their algorithms through pre-training [55], transfer learning [56], and/or fine-tuning [57] to bypass the problem of a small training dataset, in addition to developing variations of algorithms that are based on other deep learning technologies besides t…”
BackgroundArmato, S. G., McLennan, G., Bidaut, L., McNitt‐Gray, M. F., Meyer, C. R., Reeves, A. P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al. (2011) · Medical Physics · doi:10.1118/1.3528204 · Cited by 2,887 · cited 2×
“Most studies have trained and tested their algorithms on the large and publicly available Lung Image Database Consortium and Image Database Resource Initiative (LIDC-IDRI) dataset, which makes the studies homogenous [16].”
BackgroundSetio, A. A. A., Ciompi, F., Litjens, G., Gerke, P., Jacobs, C., van Riel, S. J., Wille, M. M. W., Naqibullah, M., Sanchez, C. I., & van Ginneken, B. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2536809 · Cited by 1,258
Masood, A., Sheng, B., Li, P., Hou, X., Wei, X., Qin, J., & Feng, D. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.01.005 · Cited by 292
Ciompi, F., de Hoop, B., van Riel, S. J., Chung, K., Scholten, E. T., Oudkerk, M., de Jong, P. A., Prokop, M., & Ginneken, B. V. (2015) · Medical Image Analysis · doi:10.1016/j.media.2015.08.001 · Cited by 289 · cited 3×
“(2015) [28] tested on CT scans from the Dutch–Belgian Randomized Lung”
BackgroundMethodInfluentialCho, J., Lee, K., Shin, E., Choy, G., & Do, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06348 · Cited by 268
Zhang, C., Sun, X., Dang, K., Li, K., Guo, X.-W., Chang, J., Yu, Z.-Q., Huang, F.-Y., Wu, Y.-S., Liang, Z., et al. (2019) · The Oncologist · doi:10.1634/theoncologist.2018-0908 · Cited by 168
Cited by
Adams, S. J., Stone, E., Baldwin, D. R., Vliegenthart, R., Lee, P., & Fintelmann, F. J. (2023) · The Lancet · doi:10.1016/s0140-6736(22)01694-4 · Cited by 467
Artificial intelligence and machine learning in cancer imaging
Koh, D.-M., Papanikolaou, N., Bick, U., Illing, R., Kahn, C. E., Kalpathi-Cramer, J., Matos, C., Martí-Bonmatí, L., Miles, A., Mun, S. K., et al. (2022) · Communications Medicine · doi:10.1038/s43856-022-00199-0 · Cited by 311
Application of Artificial Intelligence in Lung Cancer
Chiu, H.-Y., Chao, H.-S., & Chen, Y.-M. (2022) · Cancers · doi:10.3390/cancers14061370 · Cited by 138
“Many studies have used AI algorithms to detect lung nodules in chest CT images [101,102].”
Jayakumar, S., Sounderajah, V., Normahani, P., Harling, L., Markar, S. R., Ashrafian, H., & Darzi, A. (2022) · npj Digital Medicine · doi:10.1038/s41746-021-00544-y · Cited by 110
Morone, G., De Angelis, L., Martino Cinnera, A., Carbonetti, R., Bisirri, A., Ciancarelli, I., Iosa, M., Negrini, S., Kiekens, C., & Negrini, F. (2025) · Frontiers in Digital Health · doi:10.3389/fdgth.2025.1550731 · Cited by 41
Metadata
- Authors
- Li, D., Mikela Vilmun, B., Frederik Carlsen, J., Albrecht-Beste, E., Ammitzbøl Lauridsen, C., Bachmann Nielsen, M., & Lindskov Hansen, K.
- Venue
- Diagnostics
- Publisher
- MDPI AG
- Published
- 2019-11-29
- Volume
- 9
- Issue
- 4
- Pages
- 207
- ISSN
- 2075-4418 (ISSN-L)
- OpenAlex
- W2990375292
- Cited by
- 66
- References
- 9
- FWCI
- 1.23
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Performance of Deep Learning Algorithms on Automatic Pulmonary Nodule Detection and Classification Tested on Different Datasets That Are Not Derived from LIDC-IDRI: A Systematic Review. https://app.cochat.ai/papers/10.3390/diagnostics9040207
- The paper
- Li, D., Mikela Vilmun, B., Frederik Carlsen, J., Albrecht-Beste, E., Ammitzbøl Lauridsen, C., Bachmann Nielsen, M., & Lindskov Hansen, K. (2019). The Performance of Deep Learning Algorithms on Automatic Pulmonary Nodule Detection and Classification Tested on Different Datasets That Are Not Derived from LIDC-IDRI: A Systematic Review. Diagnostics. https://doi.org/10.3390/diagnostics9040207
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.