CoChat
Sign Up
Cited by 123Lung Cancer Diagnosis and TreatmentAI in cancer detectionRadiomics and Machine Learning in Medical Imaging

Deep Learning Using Chest Radiographs to Identify High-Risk Smokers for Lung Cancer Screening Computed Tomography: Development and Validation of a Prediction Model

Open access
Lu, Michael T., Raghu, Vineet K., Mayrhofer, Thomas, Aerts, Hugo J.W.L., Hoffmann, Udo·2020·Annals of Internal Medicine
DOI 10.7326/m20-1868PMC PMC9200444
Open this paper in CoChat — free account

Abstract

Background: Lung cancer screening with chest computed tomography (CT) reduces lung cancer death. Centers for Medicare & Medicaid Services (CMS) eligibility criteria for lung cancer screening with CT require detailed smoking information and miss many incident lung cancers. An automated deep-learning approach based on chest radiograph images may identify more smokers at high risk for lung cancer who could benefit from screening with CT. Objective: To develop and validate a convolutional neural network (CXR-LC) that predicts long-term incident lung cancer using data commonly available in the electronic medical record (EMR) (chest radiograph, age, sex, and whether currently smoking). Design: Risk prediction study. Setting: U.S. lung cancer screening trials. Participants: The CXR-LC model was developed in the PLCO (Prostate, Lung, Colorectal, and Ovarian) Cancer Screening Trial (n = 41 856). The final CXR-LC model was validated in additional PLCO smokers (n = 5615, 12-year follow-up) and NLST (National Lung Screening Trial) heavy smokers (n = 5493, 6-year follow-up). Results are reported for validation data sets only. Measurements: Up to 12-year lung cancer incidence predicted by CXR-LC. Results: The CXR-LC model had better discrimination (area under the receiver-operating characteristic curve [AUC]) for incident lung cancer than CMS eligibility (PLCO AUC, 0.755 vs. 0.634; P < 0.001). The CXR-LC model's performance was similar to that of PLCOM2012, a state-of-the-art risk score with 11 inputs, in both the PLCO data set (CXR-LC AUC of 0.755 vs. PLCOM2012 AUC of 0.751) and the NLST data set (0.659 vs. 0.650). When compared in equal-sized screening populations, CXR-LC was more sensitive than CMS eligibility in the PLCO data set (74.9% vs. 63.8%; P = 0.012) and missed 30.7% fewer incident lung cancers. On decision curve analysis, CXR-LC had higher net benefit than CMS eligibility and similar benefit to PLCOM2012. Limitation: Validation in lung cancer screening trials and not a clinical setting. Conclusion: The CXR-LC model identified smokers at high risk for incident lung cancer, beyond CMS eligibility and using information commonly available in the EMR. Primary Funding Source: None.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.7326/m20-1868

Cites

3 of 3 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Deep learning can capture imaging features beyond traditional findings (e.g. enlarged heart silhouette, lung nodules) with valuable prognostic and potentially actionable information.

    “the results of this and previously published studies(10–12) suggest that deep learning can capture imaging features beyond traditional findings (e.g. enlarged heart silhouette, lung nodules) with valuable prognostic and potentially actionable information.”

    From Deep Learning to Estimate Cardiovascular Risk From Chest Radiographs (2024) · cites this paper as [11]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 50

  1. [5]Tammemagi MC. Selecting lung cancer screenees using risk prediction models-where do we go from here. Transl Lung Cancer Res. 2018; 7(3): 243– 53. 30050763 PMC6037970
  2. [6]Wang GX, Baggett TP, Pandharipande PV, Park ER, Percac-Lima S, Shepard JO, Barriers to Lung Cancer Screening Engagement from the Patient and Provider Perspective. Radiology. 2019; 290(2): 278– 87. 30620258
  3. [7]Wang Y, Midthun DE, Wampfler JA, Deng B, Stoddard SM, Zhang S, Trends in the proportion of patients with lung cancer meeting screening criteria. JAMA. 2015; 313(8): 853– 5. 25710663 PMC4493881
  4. [8]Jemal A, Fedewa SA. Lung Cancer Screening With Low-Dose Computed Tomography in the United States-2010 to 2015. JAMA Oncol. 2017; 3(9): 1278– 81. 28152136 PMC5824282
  5. [9]Richards TB, Doria-Rose VP, Soman A, Klabunde CN, Caraballo RS, Gray SC, Lung Cancer Screening Inconsistent With U.S. Preventive Services Task Force Recommendations. Am J Prev Med. 2019; 56(1): 66– 73. 30467092 PMC6319382
  6. [10]Smith RA, Andrews KS, Brooks D, Fedewa SA, Manassaram-Baptiste D, Saslow D, Cancer screening in the United States, 2019: A review of current American Cancer Society guidelines and current issues in cancer screening. CA Cancer J Clin. 2019; 69(3): 184– 210. 30875085
  7. [13]Melzer AC, Golden SE, Ono SS, Datta S, Triplette M, Slatore CG. "We Just Never Have Enough Time": Clinician Views of Lung Cancer Screening Processes and Implementation. Ann Am Thorac Soc. 2020. 32497437
  8. [14]Triplette M, Kross EK, Mann BA, Elmore JG, Slatore CG, Shahrir S, An Assessment of Primary Care and Pulmonary Provider Perspectives on Lung Cancer Screening. Ann Am Thorac Soc. 2018; 15(1): 69– 75. 28933940 PMC5822418
  9. [15]Detterbeck FC, Mazzone PJ, Naidich DP, Bach PB. Screening for lung cancer: Diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest. 2013; 143(5 Suppl): e78S– e92S. 23649455 PMC3749713
  10. [16]Kinsinger LS, Anderson C, Kim J, Larson M, Chan SH, King HA, Implementation of Lung Cancer Screening in the Veterans Health Administration. JAMA Intern Med. 2017; 177(3): 399– 406. 28135352
  11. [17]Modin HE, Fathi JT, Gilbert CR, Wilshire CL, Wilson AK, Aye RW, Pack-Year Cigarette Smoking History for Determination of Lung Cancer Screening Eligibility. Comparison of the Electronic Medical Record versus a Shared Decision-making Conversation. Ann Am Thorac Soc. 2017; 14(8): 1320– 5. 28406708
  12. [18]Tammemagi MC, Katki HA, Hocking WG, Church TR, Caporaso N, Kvale PA, Selection criteria for lung-cancer screening. N Engl J Med. 2013; 368(8): 728– 36. 23425165 PMC3929969
  13. [19]Katki HA, Kovalchik SA, Petito LC, Cheung LC, Jacobs E, Jemal A, Implications of Nine Risk Prediction Models for Selecting Ever-Smokers for Computed Tomography Lung Cancer Screening. Ann Intern Med. 2018; 169(1): 10– 9. 29800127 PMC6557386
  14. [20]Ron E Cancer risks from medical radiation. Health Phys. 2003; 85(1): 47– 59. 12852471
  15. [21]Oken MM, Hocking WG, Kvale PA, Andriole GL, Buys SS, Church TR, Screening by chest radiograph and lung cancer mortality: the Prostate, Lung, Colorectal, and Ovarian (PLCO) randomized trial. JAMA. 2011; 306(17): 1865– 73. 22031728
  16. [22]Lu MT, Ivanov A, Mayrhofer T, Hosny A, Aerts H, Hoffmann U. Deep Learning to Assess Long-term Mortality From Chest Radiographs. JAMA Netw Open. 2019; 2(7): e197416. 31322692 PMC6646994
  17. [23]Prorok PC, Andriole GL, Bresalier RS, Buys SS, Chia D, Crawford ED, Design of the Prostate, Lung, Colorectal and Ovarian (PLCO) Cancer Screening Trial. Control Clin Trials. 2000; 21(6 Suppl): 273S– 309S. 11189684
  18. [25]Aberle DR, Adams AM, Berg CD, Black WC, Clapp JD, Fagerstrom RM, Reduced lung-cancer mortality with low-dose computed tomographic screening. N Engl J Med. 2011; 365(5): 395– 409. 21714641 PMC4356534
  19. [26]Tammemagi MC, Freedman MT, Pinsky PF, Oken MM, Hu P, Riley TL, Prediction of true positive lung cancers in individuals with abnormal suspicious chest radiographs: a prostate, lung, colorectal, and ovarian cancer screening trial study. J Thorac Oncol. 2009; 4(6): 710– 21. 19404219
  20. [27]Chin J, Syrek Jensen T, Ashby L, Hermansen J, Hutter JD, Conway PH. Screening for lung cancer with low-dose CT--translating science into Medicare coverage policy. N Engl J Med. 2015; 372(22): 2083– 5. 26017822
  21. [28]Moyer VA. Screening for lung cancer: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med. 2014; 160(5): 330– 8. 24378917
  22. [29]Liu Y, Chen PC, Krause J, Peng L. How to Read Articles That Use Machine Learning: Users' Guides to the Medical Literature. JAMA. 2019; 322(18): 1806– 16. 31714992
  23. [30]Szegedy C, Ioffe S, Vanhoucke V, Alemi A. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. 2016. Accessed at https://arxiv.org/abs/1602.07261 on April 4, 2018. · arXiv
  24. [31]Tammemagi MC, Church TR, Hocking WG, Silvestri GA, Kvale PA, Riley TL, Evaluation of the lung cancer risks at which to screen ever- and never-smokers: screening rules applied to the PLCO and NLST cohorts. PLoS Med. 2014; 11(12): e1001764. 25460915 PMC4251899
  25. [32]Platt J Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in Large Margin Classifiers. 1999; 10(3): 61– 74.
  26. [33]Sun X, Xu W. Fast implementation of DeLong’s algorithm for comparing the areas under correlated receiver operating characteristic curves. IEEE Signal Processing Letters. 2014; 21(11): 1389– 93.
  27. [34]Ten Haaf K, Jeon J, Tammemagi MC, Han SS, Kong CY, Plevritis SK, Risk prediction models for selection of lung cancer screening candidates: A retrospective validation study. PLoS Med. 2017; 14(4): e1002277. 28376113 PMC5380315
  28. [37]Vickers AJ, Van Calster B, Steyerberg EW. Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests. BMJ. 2016; 352: i6. 26810254 PMC4724785
  29. [38]Vickers AJ, van Calster B, Steyerberg EW. A simple, step-by-step guide to interpreting decision curve analysis. Diagn Progn Res. 2019; 3: 18. 31592444 PMC6777022
  30. [39]Raji OY, Duffy SW, Agbaje OF, Baker SG, Christiani DC, Cassidy A, Predictive accuracy of the Liverpool Lung Project risk model for stratifying patients for computed tomography screening for lung cancer: a case-control and cohort validation study. Ann Intern Med. 2012; 157(4): 242– 50. 22910935 PMC3723683
  31. [40]Localio AR, Goodman S. Beyond the usual prediction accuracy metrics: reporting results for clinical decision making. Ann Intern Med. 2012; 157(4): 294– 5. 22910942
  32. [41]Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization. arXiv e-prints; 2016.
  33. [42]Wood DE, Kazerooni EA, Baum SL, Eapen GA, Ettinger DS, Hou L, Lung Cancer Screening, Version 3.2018, NCCN Clinical Practice Guidelines in Oncology. J Natl Compr Canc Netw. 2018; 16(4): 412– 41. 29632061 PMC6476336
  34. [43]Tape TG, Mushlin AI. The utility of routine chest radiographs. Ann Intern Med. 1986; 104(5): 663– 70. 3516043
  35. [44]Polubriaginof F, Salmasian H, Albert DA, Vawdrey DK. Challenges with Collecting Smoking Status in Electronic Health Records. AMIA Annu Symp Proc. 2017; 2017: 1392– 400. 29854208 PMC5977725
  36. [45]Uzuner O, Goldstein I, Luo Y, Kohane I. Identifying patient smoking status from medical discharge records. J Am Med Inform Assoc. 2008; 15(1): 14– 24. 17947624 PMC2274873
  37. [46]Dikici E, Bigelow M, Prevedello LM, White RD, Erdal BS. Integrating AI into radiology workflow: levels of research, production, and feedback maturity. J Med Imaging (Bellingham). 2020; 7(1): 016502. 32064302 PMC7012173
  38. [47]Fortmann SP, Bailey SR, Brooks NB, Hitsman B, Rittner SS, Gillespie SE, Trends in smoking documentation rates in safety net clinics. Health Serv Res. 2020; 55(2): 170– 7. 31930738 PMC7080378
  39. [48]Simpson CR, Hippisley-Cox J, Sheikh A. Trends in the epidemiology of smoking recorded in UK general practice. Br J Gen Pract. 2010; 60(572): e121– 7. 20202355 PMC2828860
  40. [52]Sohn JH, Chillakuru YR, Lee S, Lee AY, Kelil T, Hess CP, An Open-Source, Vender Agnostic Hardware and Software Pipeline for Integration of Artificial Intelligence in Radiology Workflow. J Digit Imaging. 2020. PMC7522128 32468486
  41. [53]Lam S, Myers R, Ruparel M, Atkar-Khattra SK, Stone E, Manser R, Lung Cancer Screenee Selection by USPSTF Versus PLCOm2012 Criteria – Interim ILST Findings. J Thorac Oncol. 2019; 14(10): S4– S5.
  42. [54]O'Dowd EL, Ten Haaf K. Lung cancer screening: enhancing risk stratification and minimising harms by incorporating information from screening results. Thorax. 2019; 74(9): 825– 7. 31363021
  43. [55]Tammemagi MC, Ten Haaf K, Toumazis I, Kong CY, Han SS, Jeon J, Development and Validation of a Multivariable Lung Cancer Risk Prediction Model That Includes Low-Dose Computed Tomography Screening Results: A Secondary Analysis of Data From the National Lung Screening Trial. JAMA Netw Open. 2019; 2(3): e190204. 30821827 PMC6484623
  44. [56]Henschke CI, Yip R, Boffetta P, Markowitz S, Miller A, Hanaoka T, CT screening for lung cancer: Importance of emphysema for never smokers and smokers. Lung Cancer. 2015; 88(1): 42– 7. 25698134
  45. [57]Yong PC, Sigel K, de-Torres JP, Mhango G, Kale M, Kong CY, The effect of radiographic emphysema in assessing lung cancer risk. Thorax. 2019. 30723183
  46. [58]Nalichowski R, Keogh D, Chueh HC, Murphy SN. Calculating the benefits of a Research Patient Data Repository. AMIA Annu Symp Proc. 2006: 1044. 17238663 PMC1839563
  47. [59]Aldrich MC, Mercaldo SF, Sandler KL, Blot WJ, Grogan EL, Blume JD. Evaluation of USPSTF Lung Cancer Screening Guidelines Among African American Adult Smokers. JAMA Oncol. 2019. PMC6604090 31246249
  48. [60]Lu MT, Onuma OK, Massaro JM, D'Agostino RB Sr, O'Donnell CJ, Hoffmann U. Lung Cancer Screening Eligibility in the Community: Cardiovascular Risk Factors, Coronary Artery Calcification, and Cardiovascular Events. Circulation. 2016; 134(12): 897– 9. 27647299 PMC5031246
  49. [61]Baltruschat IM, Nickisch H, Grass M, Knopp T, Saalbach A. Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classification. Sci Rep. 2019; 9(1): 6381. 31011155 PMC6476887
  50. [62]Avati A, Duan T, Jung K, Shah NH, Ng A. Countdown Regression: Sharp and Calibrated Survival Predictions. arXiv e-prints; 2018.

Cited by

Sorted by citations

Authors
Lu, M. T., Raghu, V. K., Mayrhofer, T., Aerts, H. J. W. L., & Hoffmann, U.
Affiliations
  • Brigham and Women's Hospital (US)
  • Harvard University (US)
ORCID
Venue
Annals of Internal Medicine
Publisher
American College of Physicians
Published
2020-11-03
Volume
173
Issue
9
Pages
704-713
ISSN
  • 0003-4819 (ISSN-L)
  • 1539-3704
DOI
10.7326/m20-1868
PMC
PMC9200444
OpenAlex
W3082622980
Cited by
123
References
3
Citations by year
2026: 8 · 2025: 22 · 2024: 29 · 2023: 28 · 2022: 20 · 2021: 14 · 2020: 2
FWCI
8.85
Type
journal article
Language
en
Open access
Locations
  • Annals of Internal Medicine (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
National Heart, Lung, and Blood Institute (T32HL076136)
Topics
  • Health Sciences › Medicine › Pulmonary and Respiratory Medicine › Lung Cancer Diagnosis and Treatment
  • Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
  • Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
Keywords
lung cancer screening, chest radiograph, deep learning, convolutional neural networks, smoking, high-risk smokers, lung cancer risk prediction, lung cancer incidence, PLCO Cancer Screening Trial, National Lung Screening Trial, electronic health records, automated risk assessment, decision curve analysis, net benefit, lung cancer mortality reduction
MeSH terms
Deep Learning, Aged, Decision Support Techniques, Female, Humans, Lung, Lung Neoplasms, Male, Middle Aged, Risk Factors, Sensitivity and Specificity, Smoking, Tomography, X-Ray Computed, Risk Assessment, Early Detection of Cancer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review