Prediction of premature all-cause mortality: A prospective general population cohort study comparing machine-learning and standard epidemiological approaches
Open accessAbstract
BACKGROUND: Prognostic modelling using standard methods is well-established, particularly for predicting risk of single diseases. Machine-learning may offer potential to explore outcomes of even greater complexity, such as premature death. This study aimed to develop novel prediction algorithms using machine-learning, in addition to standard survival modelling, to predict premature all-cause mortality. METHODS: A prospective population cohort of 502,628 participants aged 40-69 years were recruited to the UK Biobank from 2006-2010 and followed-up until 2016. Participants were assessed on a range of demographic, biometric, clinical and lifestyle factors. Mortality data by ICD-10 were obtained from linkage to Office of National Statistics. Models were developed using deep learning, random forest and Cox regression. Calibration was assessed by comparing observed to predicted risks; and discrimination by area under the 'receiver operating curve' (AUC). FINDINGS: 14,418 deaths (2.9%) occurred over a total follow-up time of 3,508,454 person-years. A simple age and gender Cox model was the least predictive (AUC 0.689, 95% CI 0.681-0.699). A multivariate Cox regression model significantly improved discrimination by 6.2% (AUC 0.751, 95% CI 0.748-0.767). The application of machine-learning algorithms further improved discrimination by 3.2% using random forest (AUC 0.783, 95% CI 0.776-0.791) and 3.9% using deep learning (AUC 0.790, 95% CI 0.783-0.797). These ML algorithms improved discrimination by 9.4% and 10.1% respectively from a simple age and gender Cox regression model. Random forest and deep learning achieved similar levels of discrimination with no significant difference. Machine-learning algorithms were well-calibrated, while Cox regression models consistently over-predicted risk. CONCLUSIONS: Machine-learning significantly improved accuracy of prediction of premature all-cause mortality in this middle-aged population, compared to standard methods. This study illustrates the value of machine-learning for risk prediction within a traditional epidemiological study design, and how this approach might be reported to assist scientific verification.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
AI is expected to augment healthcare workflow through automated triage, improve the productivity of individual physicians, reduce human errors, discover better patterns of patient care, defray medical costs, perform minimally invasive surgery, and reduce mortality rates.
“AI is expected to augment healthcare workflow through automated triage, improve the productivity of individual physicians, reduce human errors, discover better patterns of patient care, defray medical costs, perform minimally invasive surgery, and reduce mortality rates.8–14”
From Educating Future Physicians in Artificial Intelligence (AI): An Integrative Review and Proposed Changes (2021) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Multilayer feedforward networks are universal approximators
Hornik, K., Stinchcombe, M., & White, H. (1989) · Neural Networks · doi:10.1016/0893-6080(89)90020-8 · Cited by 21,728 · see citing works
Dermatologist-level classification of skin cancer with deep neural networks
Esteva, A., Kuprel, B., Novoa, R. A., Ko, J., Swetter, S. M., Blau, H. M., & Thrun, S. (2017) · Nature · doi:10.1038/nature21056 · Cited by 14,362 · see citing works
Multilayer feedforward networks are universal approximators
HornikK., StinchcombeM., & WhiteH. (1989) · Neural Networks · doi:10.5555/70405.70408 · Cited by 9,266 · see citing works
Gulshan, V., Peng, L., Coram, M., Stumpe, M. C., Wu, D., Narayanaswamy, A., Venugopalan, S., Widner, K., Madams, T., Cuadros, J., et al. (2016) · JAMA · doi:10.1001/jama.2016.17216 · Cited by 7,697 · see citing works
General Cardiovascular Risk Profile for Use in Primary Care: The Framingham Heart Study
D’Agostino, R. B., Vasan, R. S., Pencina, M. J., Wolf, P. A., Cobain, M., Massaro, J. M., & Kannel, W. B. (2008) · Circulation · doi:10.1161/circulationaha.107.699579 · Cited by 7,491 · see citing works
Hagan, M., Demuth, H. B., Beale, M. H., & Jes?s, O. D. (2016) · Institutional Repository of the National University Open and Distance UNAD (Universidad Nacional Abierta y a Distancia) · Cited by 5,133 · see citing works
Fry, A., Littlejohns, T. J., Sudlow, C., Doherty, N., Adamska, L., Sprosen, T., Collins, R., & Allen, N. E. (2017) · American Journal of Epidemiology · doi:10.1093/aje/kwx246 · Cited by 3,979 · see citing works
Bias in random forest variable importance measures: Illustrations, sources and a solution
Strobl, C., Boulesteix, A., Zeileis, A., & Hothorn, T. (2007) · BMC Bioinformatics · doi:10.1186/1471-2105-8-25 · Cited by 3,755 · see citing works
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506 · see citing works
Practical Recommendations for Gradient-Based Training of Deep Architectures
Bengio, Y. (2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8_26 · Cited by 2,259 · see citing works
Logistic regression and Artificial Neural Network Classification models: A methodology review
Dreiseitl, S., & Ohno-Machado, L. (2002) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(03)00034-0 · Cited by 2,177 · see citing works
Neural Networks: Tricks of the Trade: Second Edition
(2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8 · Cited by 2,116 · see citing works
Collins, G. S., Reitsma, J. B., Altman, D. G., & Moons, K. (2015) · BMC Medicine · doi:10.1186/s12916-014-0241-z · Cited by 1,927 · see citing works
Hippisley-Cox, J., Coupland, C., & Brindle, P. (2017) · BMJ · doi:10.1136/bmj.j2099 · Cited by 1,703 · see citing works
Health and Deprivation: Inequality and the North
Townsend, P. D., Phillimore, P., & Beattie, A. (1987) · DOAJ (DOAJ: Directory of Open Access Journals) · Cited by 1,385 · see citing works
Can machine-learning improve cardiovascular risk prediction using routine clinical data?
Weng, S. F., Reps, J., Kai, J., Garibaldi, J. M., & Qureshi, N. (2017) · PLOS ONE · doi:10.1371/journal.pone.0174944 · Cited by 1,346 · see citing works
Self-citationOlden, J. D., & Jackson, D. A. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00064-9 · Cited by 1,224 · see citing works
Multiple Imputation of Missing Values: Update of Ice
Royston, P. J. (2005) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0500500404 · Cited by 1,038 · see citing works
Yu, K.-H., Zhang, C., Berry, G. J., Altman, R. B., Ré, C., Rubin, D. L., & Snyder, M. (2016) · Nature Communications · doi:10.1038/ncomms12474 · Cited by 972 · see citing works
Prediction models for cardiovascular disease risk in the general population: systematic review
Damen, J. A. A. G., Hooft, L., Schuit, E., Debray, T. P. A., Collins, G. S., Tzoulaki, I., Lassale, C. M., Siontis, G. C. M., Chiocchia, V., Roberts, C., et al. (2016) · BMJ · doi:10.1136/bmj.i2416 · Cited by 935 · see citing works
Health and Deprivation: Inequality and the North
Blane, D., Townsend, P. D., Phillimore, P., & Beattie, A. (1989) · British Journal of Sociology · doi:10.2307/590279 · Cited by 756 · see citing works
Collins, G. S., de Groot, J. A., Dutton, S., Omar, O., Shanyinde, M., Tajar, A., Voysey, M., Wharton, R., Yu, L.-M., Moons, K. G., et al. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-40 · Cited by 729 · see citing works
Little’s test of missing completely at random
Li, C. (2013) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x1301300407 · Cited by 608 · see citing works
Onukwugha, E., Yong, C., Hussain, A., Seal, B., & Mullins, C. D. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-1 · Cited by 589 · see citing works
Westreich, D. J., Lessler, J., & Funk, M. J. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2009.11.020 · Cited by 505 · see citing works
Comparing the Predictive Powers of Survival Models Using Harrell's C or Somers’ D
Newson, R. B. (2010) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x1001000303 · Cited by 422 · see citing works
Voets, M., Møllersen, K., & Bongo, L. A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0217541 · Cited by 312 · see citing works
Confidence Intervals for Rank Statistics: Somers’ D and Extensions
Newson, R. B. (2006) · The Stata Journal Promoting communications on statistics and Stata · doi:10.1177/1536867x0600600302 · Cited by 267 · see citing works
Practical recommendations for gradient-based training of deep architecturesPreprint
Bengio, Y. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5533 · Cited by 266 · see citing works
Particulate air pollution and mortality in 38 of China’s largest cities: time series analysis
Yin, P., He, G., Fan, M., Chiu, K. Y., Fan, M., Liu, C., Xue, A., Liu, T., Pan, Y., Mu, Q., et al. (2017) · BMJ · doi:10.1136/bmj.j667 · Cited by 140 · see citing works
Cohort profile of the UK Biobank: diagnosis and characteristics of cerebrovascular disease
Hewitt, J., Walters, J., Padmanabhan, S., & Dawson, J. (2016) · BMJ Open · doi:10.1136/bmjopen-2015-009161 · Cited by 92 · see citing works
The color of death: race, observed skin tone, and all-cause mortality in the United States
Stewart, Q. T., Cobb, R. J., & Keith, V. M. (2018) · Ethnicity and Health · doi:10.1080/13557858.2018.1469735 · Cited by 37 · see citing works
Physical occupational exposures and health expectancies in a French occupational cohort
Platts, L. G., Head, J. A., Stenholm, S., et al. (2016) · Occupational and Environmental Medicine · doi:10.1136/oemed-2016-103804 · Cited by 24 · see citing works
Using machine learning to predict outcomes in psychosis
Young, J., Kempton, M. J., & McGuire, P. K. (2016) · The Lancet Psychiatry · doi:10.1016/s2215-0366(16)30218-8 · Cited by 21 · see citing works
Flagged first, then by citations
Research works printed but not linked: 1
- [23]Voets M, Møllersen K, LA B. Replication study: Development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. arXiv. 2018;abs/1803.04337. PMC6553744 31170223
Cited by
Sorted by citations
Introducing artificial intelligence training in medical education
Paranjape, K., Schinkel, M., Nannan Panday, R., Car, J., & Nanayakkara, P. (2019) · JMIR Medical Education · doi:10.2196/16048 · Cited by 634 · see citing works
Grunhut, J., Wyatt, A. T., & Marques, O. (2021) · Journal of Medical Education and Curricular Development · doi:10.1177/23821205211036836 · Cited by 161 · see citing works
AI ethics: Algorithmic determinism or self-determination? The GPDR approach
Milossi, M., Alexandropoulou-Egyptiadou, E., & Psannis, K. E. (2021) · IEEE Access · doi:10.1109/access.2021.3072782 · Cited by 46 · see citing works
Bergquist, T., Schaffter, T., Yan, Y., Yu, T., Prosser, J., Gao, J., Chen, G., Charzewski, Ł., Nawalany, Z., Brugere, I., et al. (2023) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocad159 · Cited by 8 · see citing works
- Authors
- Weng, S. F., Vaz, L., Qureshi, N., & Kai, J.
- Affiliations
- University of Nottingham (GB)
- Corresponding author
- Weng, Stephen F.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2019-03-27
- Volume
- 14
- Issue
- 3
- Pages
- e0214365
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC6436798
- OpenAlex
- W2923291095
- Cited by
- 118
- References
- 35
- Citations by year
- 2026: 9 · 2025: 14 · 2024: 17 · 2023: 11 · 2022: 16 · 2021: 27 · 2020: 20 · 2019: 4
- FWCI
- 18.26
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Repository@Nottingham (University of Nottingham) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Figshare (submittedVersion, cc-by)
- Funders
- National Institute for Health and Care Research
- NIHR School for Primary Care Research
- Topics
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- mortality prediction, machine learning, random forest, deep learning, Cox proportional hazards model, risk prediction model, model discrimination, model calibration, middle-aged adults, UK Biobank
- MeSH terms
- Deep Learning, Adult, Aged, Female, United Kingdom, Humans, Male, Middle Aged, Prognosis, Prospective Studies, ROC Curve, Logistic Models, Biological Specimen Banks, Mortality, Premature
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review