Prior CT Improves Deep Learning for Malignancy Risk Estimation of Screening-detected Pulmonary Nodules
Open accessAbstract
Background Prior chest CT provides valuable temporal information (eg, changes in nodule size or appearance) to accurately estimate malignancy risk. Purpose To develop a deep learning (DL) algorithm that uses a current and prior low-dose CT examination to estimate 3-year malignancy risk of pulmonary nodules. Materials and Methods In this retrospective study, the algorithm was trained using National Lung Screening Trial data (collected from 2002 to 2004), wherein patients were imaged at most 2 years apart, and evaluated with two external test sets from the Danish Lung Cancer Screening Trial (DLCST) and the Multicentric Italian Lung Detection Trial (MILD), collected in 2004–2010 and 2005–2014, respectively. Performance was evaluated using area under the receiver operating characteristic curve (AUC) on cancer-enriched subsets with size-matched benign nodules imaged 1 and 2 years apart from DLCST and MILD, respectively. The algorithm was compared with a validated DL algorithm that only processed a single CT examination and the Pan-Canadian Early Lung Cancer Detection Study (PanCan) model. Results The training set included 10 508 nodules (422 malignant) in 4902 trial participants (mean age, 64 years ± 5 [SD]; 2778 men). The size-matched external test sets included 129 nodules (43 malignant) and 126 nodules (42 malignant). The algorithm achieved AUCs of 0.91 (95% CI: 0.85, 0.97) and 0.94 (95% CI: 0.89, 0.98). It significantly outperformed the DL algorithm that only processed a single CT examination (AUC, 0.85 [95% CI: 0.78, 0.92; P = .002]; and AUC, 0.89 [95% CI: 0.84, 0.95; P = .01]) and the PanCan model (AUC, 0.64 [95% CI: 0.53, 0.74; P < .001]; and AUC, 0.63 [95% CI: 0.52, 0.74; P < .001]). Conclusion A DL algorithm using current and prior low-dose CT examinations was more effective at estimating 3-year malignancy risk of pulmonary nodules than established models that only use a single CT examination. Clinical trial registration nos. NCT00047385, NCT00496977, NCT02837809 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Horst and Nishino in this issue.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In oncology, longitudinal information is important to assess slowly progressive forms of cancer or cancers with unclear dignity, as well as in evaluation of treatment response.
“For example, in oncology, longitudinal information is important to assess slowly progressive forms of cancer or cancers with yet unclear dignity18 (benign vs. malignant), as well as in the evaluation of treatment response.”
From Orchestrating explainable artificial intelligence for multimodal and longitudinal data in medical imaging (2024) · cites this paper as [18]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988) · Biometrics · doi:10.2307/2531595 · Cited by 22,649 · see citing works
Siegel, R. L., Miller, K. D., & Jemal, A. (2020) · CA: A Cancer Journal for Clinicians · doi:10.3322/caac.21590 · Cited by 20,444 · see citing works
pROC: an open-source package for R and S+ to analyze and compare ROC curves
Robin, X., Turck, N., Hainard, A., Tiberti, N., Lisacek, F., Sanchez, J.-C., & Müller, M. (2011) · BMC Bioinformatics · doi:10.1186/1471-2105-12-77 · Cited by 13,697 · see citing works
Reduced Lung-Cancer Mortality with Low-Dose Computed Tomographic Screening
The National Lung Screening Trial Research Team (2011) · New England Journal of Medicine · doi:10.1056/nejmoa1102873 · Cited by 11,064 · see citing works
nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation
Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2021) · Nature Methods · doi:10.1038/s41592-020-01008-z · Cited by 10,477 · see citing works
Reduced Lung-Cancer Mortality with Volume CT Screening in a Randomized Trial
de Koning, H. J., van der Aalst, C. M., de Jong, P. A., Scholten, E. T., Nackaerts, K., Heuvelmans, M. A., Lammers, J.-W. J., Weenink, C., Yousaf-Khan, U., Horeweg, N., et al. (2020) · New England Journal of Medicine · doi:10.1056/nejmoa1911793 · Cited by 3,624 · see citing works
Self-citationArmato, S. G., McLennan, G., Bidaut, L., McNitt‐Gray, M. F., Meyer, C. R., Reeves, A. P., Zhao, B., Aberle, D. R., Henschke, C. I., Hoffman, E. A., et al. (2011) · Medical Physics · doi:10.1118/1.3528204 · Cited by 2,887 · see citing works
Ardila, D., Kiraly, A. P., Bharadwaj, S., Choi, B., Reicher, J. J., Peng, L., Tse, D., Etemadi, M., Ye, W., Corrado, G., et al. (2019) · Nature Medicine · doi:10.1038/s41591-019-0447-x · Cited by 2,167 · see citing works
Probability of Cancer in Pulmonary Nodules Detected on First Screening CT
McWilliams, A. M., Tammemägi, M. C., Mayo, J. R., Roberts, H. C., Liu, G., Soghrati, K., Yasufuku, K., Martel, S., Laberge, F., Gingras, M., et al. (2013) · New England Journal of Medicine · doi:10.1056/nejmoa1214726 · Cited by 1,425 · see citing works
Callister, M. E. J., Baldwin, D., Akram, A. R., Barnard, S., Cane, P., Draffan, J., Franks, K., Gleeson, F. V., Graham, R., Malhotra, P. S., et al. (2015) · Thorax · doi:10.1136/thoraxjnl-2015-207168 · Cited by 970 · see citing works
Self-citationEuropean position statement on lung cancer screening
Oudkerk, M., Devaraj, A., Vliegenthart, R., Henzler, T., Prosch, H., Heußel, C. P., Bastarrika, G., Sverzellati, N., Mascalchi, M., Delorme, S., et al. (2017) · The Lancet Oncology · doi:10.1016/s1470-2045(17)30861-6 · Cited by 590 · see citing works
Self-citationPastorino, U., Silva, M., Sestini, S., Sabia, F., Boeri, M., Cantarutti, A., Sverzellati, N., Sozzi, G., Corrao, G., & Marchianò, A. V. (2019) · Annals of Oncology · doi:10.1093/annonc/mdz117 · Cited by 531 · see citing works
Self-citationHoreweg, N., van Rosmalen, J., Heuvelmans, M. A., van der Aalst, C. M., Vliegenthart, R., Scholten, E. T., ten Haaf, K., Nackaerts, K., Lammers, J. J., Weenink, C., et al. (2014) · The Lancet Oncology · doi:10.1016/s1470-2045(14)70389-4 · Cited by 525 · see citing works
Self-citationEvaluation of Registration Methods on Thoracic CT: The EMPIRE10 Challenge
Murphy, K., van Ginneken, B., Reinhardt, J. M., Kabus, S., Ding, K., Deng, X. H., Cao, K., Du, K., Christensen, G. E., Garcia, V., et al. (2011) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2011.2158349 · Cited by 441 · see citing works
Pedersen, J. H., Ashraf, H., Dirksen, A., Bach, K., Hansen, H., Toennesen, P., Thorsen, H., Brodersen, J., Skov, B. G., Døssing, M., et al. (2009) · Journal of Thoracic Oncology · doi:10.1097/jto.0b013e3181a0d98f · Cited by 432 · see citing works
Adams, S. J., Stone, E., Baldwin, D. R., Vliegenthart, R., Lee, P., & Fintelmann, F. J. (2023) · The Lancet · doi:10.1016/s0140-6736(22)01694-4 · Cited by 428 · see citing works
Results of the Randomized Danish Lung Cancer Screening Trial with Focus on High-Risk Profiling
Wille, M. M. W., Dirksen, A., Ashraf, H., Saghir, Z., Bach, K., Brodersen, J., Clementsen, P. F., Hansen, H., Larsen, K. R., Mortensen, J., et al. (2015) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201505-1040oc · Cited by 341 · see citing works
Self-citationKuhnigk, J., Dicken, V., Bornemann, L., Bakai, A., Wormanns, D., Krass, S., & Peitgen, H. (2006) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2006.871547 · Cited by 287 · see citing works
Lung cancer prediction by Deep Learning to identify benign lung nodules
Heuvelmans, M. A., van Ooijen, P. M. A., Ather, S., Silva, C. F., Han, D., Heußel, C. P., Hickes, W., Kauczor, H., Novotný, P., Peschl, H., et al. (2021) · Lung Cancer · doi:10.1016/j.lungcan.2021.01.027 · Cited by 214 · see citing works
Baldwin, D., Gustafson, J. K., Pickup, L., Arteta, C., Novotný, P., Declerck, J. M., Kadir, T., Figueiras, C., Sterba, A., Exell, A., et al. (2020) · Thorax · doi:10.1136/thoraxjnl-2019-214104 · Cited by 211 · see citing works
Assessing the accuracy of a deep learning method to risk stratify indeterminate pulmonary nodules
Massion, P. P., Antic, S., Ather, S., Arteta, C., Brabec, J., Chen, H., Declerck, J., Dufek, D., Hickes, W., Kadir, T., et al. (2020) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.201903-0505oc · Cited by 190 · see citing works
Deep Learning for Malignancy Risk Estimation of Pulmonary Nodules Detected at Low-Dose Screening CT
Venkadesh, K. V., Setio, A. A. A., Schreuder, A., Scholten, E. T., Chung, K., Wille, M. M. W., Saghir, Z., van Ginneken, B., Prokop, M., & Jacobs, C. (2021) · Radiology · doi:10.1148/radiol.2021204433 · Cited by 181 · see citing works
Self-citationHuang, P., Lin, C. T., Li, Y., Tammemägi, M. C., Brock, M. V., Atkar-Khattra, S., Xu, Y., Hu, P., Mayo, J. R., Schmidt, H., et al. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30159-1 · Cited by 151 · see citing works
Protocol and Rationale for the International Lung Screening Trial
Lim, K. P., Marshall, H., Tammemägi, M. C., Brims, F., McWilliams, A. M., Stone, E. C. A., Manser, R. L., Canfell, K., Weber, M. F., Connelly, L. B., et al. (2020) · Annals of the American Thoracic Society · doi:10.1513/annalsats.201902-102oc · Cited by 92 · see citing works
Schreuder, A., Scholten, E. T., van Ginneken, B., & Jacobs, C. (2021) · Translational Lung Cancer Research · doi:10.21037/tlcr-2020-lcs-06 · Cited by 91 · see citing works
Self-citationiW-Net: an automatic and minimalistic interactive lung nodule segmentation deep network
Aresta, G., Jacobs, C., Araújo, T., Cunha, A. M. T. D. S., Ramos, I. M., van Ginneken, B., & Campilho, A. (2019) · Scientific Reports · doi:10.1038/s41598-019-48004-8 · Cited by 87 · see citing works
Self-citationSilva, M., Prokop, M., Jacobs, C., Capretti, G., Sverzellati, N., Ciompi, F., van Ginneken, B., Schaefer‐Prokop, C. M., Galeone, C., Marchianò, A. V., et al. (2018) · Journal of Thoracic Oncology · doi:10.1016/j.jtho.2018.06.013 · Cited by 85 · see citing works
Self-citationArtificial Intelligence Tool for Assessment of Indeterminate Pulmonary Nodules Detected with CT
Kim, R. Y., Oke, J. L., Pickup, L., Munden, R. F., Dotson, T. L., Bellinger, C., Cohen, A., Simoff, M., Massion, P. P., Filippini, C., et al. (2022) · Radiology · doi:10.1148/radiol.212182 · Cited by 85 · see citing works
Wille, M. M. W., van Riel, S. J., Saghir, Z., Dirksen, A., Pedersen, J. H., Jacobs, C., Thomsen, L. H., Scholten, E. T., Skovgaard, L. T., & van Ginneken, B. (2015) · European Radiology · doi:10.1007/s00330-015-3689-0 · Cited by 83 · see citing works
Self-citationGoogle’s lung cancer AI: a promising tool that needs further validation
Jacobs, C., & van Ginneken, B. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0248-7 · Cited by 39 · see citing works
Self-citationvan Riel, S. J., Ciompi, F., Wille, M. M. W., et al. (2017) · PLoS ONE · doi:10.1371/journal.pone.0185032 · Cited by 36 · see citing works
The Need for Medical Artificial Intelligence That Incorporates Prior Images
Acosta, J., Falcone, G. J., & Rajpurkar, P. (2022) · Radiology · doi:10.1148/radiol.212830 · Cited by 36 · see citing works
Jacobs, C., Schreuder, A., van Riel, S. J., Scholten, E. T., Wittenberg, R., Wille, M. M. W., de Hoop, B., Sprengers, R. W., Mets, O. M., Geurts, B. H. J., et al. (2021) · Radiology Imaging Cancer · doi:10.1148/rycan.2021200160 · Cited by 32 · see citing works
Self-citationMILD trial, strong confirmation of lung cancer screening efficacy
Schabath, M. B., & Aberle, D. R. (2019) · Nature Reviews Clinical Oncology · doi:10.1038/s41571-019-0231-3 · Cited by 9 · see citing works
Deep learning to stratify lung nodules on annual follow-up CT
Heuvelmans, M. A., & Oudkerk, M. (2019) · The Lancet Digital Health · doi:10.1016/s2589-7500(19)30156-6 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Tammemägi MC. Lung Cancer Screening with CT: A Few Steps on a Long Journey. Radiology 2021;300(2):448-449.
- He K, Zhang X, Ren S, Sun J. Deep Residual Learning for Image Rec- ognition. 2016 IEEE Conf Comput Vis Pattern Recognit CVPR. IEEE, 2016; 770-778.
- Platt J. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Adv Large Margin Classif 1999;10(3):61-74. https://scholar.google.com/citations?view_op=view_citation&hl=en&user= rtWKzFwAAAAJ&citation_for_view=rtWKzFwAAAAJ:u-x6o8ySG0sC.
Cited by
Sorted by citations
Pahud de Mortanges, A., Luo, H., Shu, S. Z., Kamath, A., Suter, Y., Shelan, M., Pöllinger, A., & Reyes, M. (2024) · npj Digital Medicine · doi:10.1038/s41746-024-01190-w · Cited by 93 · see citing works
- Authors
- Vaidhya Venkadesh, K., Aleef, T. A., Scholten, E. T., Saghir, Z., Silva, M., Sverzellati, N., Pastorino, U., van Ginneken, B., Prokop, M., & Jacobs, C.
- Affiliations
- University of Copenhagen (DK)
- ORCID
- Venue
- Radiology
- Published
- 2023-08-01
- Volume
- 308
- Issue
- 2
- Pages
- e223308-e223308
- ISSN
- 0033-8419 (ISSN-L)
- 1527-1315
- OpenAlex
- W4385441689
- Cited by
- 32
- References
- 35
- Citations by year
- 2026: 8 · 2025: 12 · 2024: 10 · 2023: 2
- FWCI
- 5.90
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Radiology (publishedVersion)
- PubMed (publishedVersion)
- Research at the University of Copenhagen (University of Copenhagen) (publishedVersion)
- Radboud Repository (Radboud University) (submittedVersion)
- University of Groningen research database (University of Groningen / Centre for Information Technology) (submittedVersion, other-oa)
- Topics
- Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Lung Cancer Diagnosis and Treatment
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- pulmonary nodules, low-dose CT, lung cancer screening, deep learning, nodule growth, National Lung Screening Trial, nodule characterization, benign versus malignant nodules, external validation, model comparison
- MeSH terms
- Deep Learning, Canada, Humans, Lung Neoplasms, Male, Middle Aged, Retrospective Studies, Tomography, X-Ray Computed, Early Detection of Cancer, Multiple Pulmonary Nodules
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
