Survival Prediction Models since Liver Transplantation - Comparisons between Cox Models and Machine Learning Techniques
Open accessPublished version of a preprint
Abstract
BACKGROUND: Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it has greater potential than traditional regression models when dealing with complex data. Criticism to ML is related to unsuitable performance measures and lack of interpretability which is important for clinicians. METHODS: In this paper, ML techniques such as random forests and neural networks are applied to large data of 62294 patients from the United States with 97 predictors selected on clinical/statistical grounds, over more than 600, to predict survival from transplantation. Of particular interest is also the identification of potential risk factors. A comparison is performed between 3 different Cox models (with all variables, backward selection and LASSO) and 3 machine learning techniques: a random survival forest and 2 partial logistic artificial neural networks (PLANNs). For PLANNs, novel extensions to their original specification are tested. Emphasis is given on the advantages and pitfalls of each method and on the interpretability of the ML techniques. RESULTS: Well-established predictive measures are employed from the survival field (C-index, Brier score and Integrated Brier Score) and the strongest prognostic factors are identified for each model. Clinical endpoint is overall graft-survival defined as the time between transplantation and the date of graft-failure or death. The random survival forest shows slightly better predictive performance than Cox models based on the C-index. Neural networks show better performance than both Cox models and random survival forest based on the Integrated Brier Score at 10 years. CONCLUSION: In this work, it is shown that machine learning techniques can be a useful tool for both prediction and interpretation in the survival context. From the ML techniques examined here, PLANN with 1 hidden layer predicts survival probabilities the most accurately, being as calibrated as the Cox model with all variables. TRIAL REGISTRATION: Retrospective data were provided by the Scientific Registry of Transplant Recipients under Data Use Agreement number 9477 for analysis of risk factors after liver transplantation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In some cases, the use of AI cannot provide significant improvements over current models.
“Secondly, in some cases, the use of AI cannot provide significant improvements over current models (58–60).”
From Artificial Intelligence: Present and Future Potential for Solid Organ Transplantation (2022) · cites this paper as [59]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
R: A Language and Environment for Statistical Computing
R Core Team (2000) · doi:10.32614/r.manuals · Cited by 355,706 · see citing works
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538 · see citing works
Nonparametric Estimation from Incomplete Observations
Kaplan, E. L., & Meier, P. (1992) · Springer series in statistics · doi:10.1007/978-1-4612-4380-9_25 · Cited by 45,232 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1972) · Journal of the Royal Statistical Society Series B: Statistical Methodology · doi:10.1111/j.2517-6161.1972.tb00899.x · Cited by 39,793 · see citing works
Nonparametric Estimation from Incomplete Observations
Kaplan, E. L., & Meier, P. (1958) · Journal of the American Statistical Association · doi:10.1080/01621459.1958.10501452 · Cited by 39,158 · see citing works
Regression Models and Life-Tables
Cox, D. R. (1992) · Springer Series in Statistics · doi:10.1007/978-1-4612-4380-9_37 · Cited by 26,297 · see citing works
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970 · see citing works
MissForest—Non-parametric missing value imputation for mixed-type data
Stekhoven, D. J., & Bühlmann, P. (2012) · Bioinformatics · doi:10.1093/bioinformatics/btr597 · Cited by 6,156 · see citing works
Journal of the American Statistical Association
Koehler, A. B. (1995) · International Journal of Forecasting · doi:10.1016/0169-2070(95)90067-5 · Cited by 5,359 · see citing works
Deo, R. C. (2015) · Circulation · doi:10.1161/circulationaha.115.001593 · Cited by 3,466 · see citing works
Ishwaran, H., Kogalur, U. B., Blackstone, E. H., & Lauer, M. S. (2008) · The Annals of Applied Statistics · doi:10.1214/08-aoas169 · Cited by 2,714 · see citing works
A note on quantifying follow-up in studies of failure time
SCHEMPER, M., & Smith, T. L. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(96)00075-x · Cited by 2,592 · see citing works
Interpreting neural-network connection weights
Garson, G. D. (1991) · AI Expert archive · doi:10.5555/129449.129452 · Cited by 1,498 · see citing works
Ishwaran, H., Kogalur, U. B., Blackstone, E. H., & Lauer, M. S. (2008) · Cited by 1,310 · see citing works
Olden, J. D., & Jackson, D. A. (2002) · Ecological Modelling · doi:10.1016/s0304-3800(02)00064-9 · Cited by 1,224 · see citing works
Back-propagation neural networks for modeling complex systems
Goh, A. T. C. (1995) · Artificial Intelligence in Engineering · doi:10.1016/0954-1810(94)00011-s · Cited by 1,085 · see citing works
Linear regression with censored data
Buckley, J. D., & JAMES, I. R. (1979) · Biometrika · doi:10.1093/biomet/66.3.429 · Cited by 1,032 · see citing works
SPLIT SELECTION METHODS FOR CLASSIFICATION TREES
Loh, W., & Shih, Y. (1997) · Cited by 933 · see citing works
The Survival Benefit of Liver Transplantation
Merion, R. M., Schaubel, D. E., Dykstra, D. M., Freeman, R. B., Port, F. K., & Wolfe, R. A. (2005) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2004.00703.x · Cited by 868 · see citing works
On the exact distribution of maximally selected rank statistics
Hothorn, T., & Lausen, B. (2003) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(02)00225-6 · Cited by 694 · see citing works
Computational statistics & data analysis
ScienceDirect (1983) · Cited by 637 · see citing works
Predictive value of statistical models.
Van Houwelingen, J. C., & Le Cessie, S. (1990) · Statistics in Medicine · doi:10.1002/sim.4780091109 · Cited by 621 · see citing works
Regression Trees for Censored Data
Segal, M. R. (1988) · Biometrics · doi:10.2307/2531894 · Cited by 497 · see citing works
Machine Learning for Survival Analysis
Wang, P., Li, Y., & Reddy, C. K. (2019) · ACM Computing Surveys · doi:10.1145/3214306 · Cited by 473 · see citing works
A neural network model for survival data
Faraggi, D., & Simon, R. M. (1995) · Statistics in Medicine · doi:10.1002/sim.4780140108 · Cited by 467 · see citing works
High-Dimensional Variable Selection for Survival Data
Ishwaran, H., Kogalur, U. B., Gorodeski, E. Z., Minn, A. J., & Lauer, M. S. (2010) · Journal of the American Statistical Association · doi:10.1198/jasa.2009.tm08622 · Cited by 439 · see citing works
Cox-nnet: An artificial neural network method for prognosis prediction of high-throughput omics data
Ching, T. H., Zhu, X., & Garmire, L. X. (2018) · PLoS Computational Biology · doi:10.1371/journal.pcbi.1006076 · Cited by 403 · see citing works
Machine Learning in Healthcare: A Review
Shailaja, K., Seetharamulu, B., & Jabbar, M. A. (2018) · 2018 Second International Conference on Electronics, Communication and Aerospace Technology (ICECA) · doi:10.1109/iceca.2018.8474918 · Cited by 372 · see citing works
Desai, R. J., Wang, S. V., Vaduganathan, M., Evers, T., & Schneeweiß, S. (2020) · JAMA Network Open · doi:10.1001/jamanetworkopen.2019.18962 · Cited by 294 · see citing works
Editorial: Artificial Neural Networks as Models of Neural Information Processing
van Gerven, M. A. J., & Bohté, S. M. (2017) · Frontiers in Computational Neuroscience · doi:10.3389/fncom.2017.00114 · Cited by 279 · see citing works
Biganzoli, E. M., Boracchi, P., Mariani, L., & Marubini, E. (1998) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19980530)17:10 · Cited by 274 · see citing works
Cross‐validation in survival analysis
Verweij, P. J. M., & van Houwelingen, H. C. (1993) · Statistics in Medicine · doi:10.1002/sim.4780122407 · Cited by 267 · see citing works
Ishwaran, H., & Lü, M. (2018) · Statistics in Medicine · doi:10.1002/sim.7803 · Cited by 261 · see citing works
Efficient Screening of Nonnormal Regression Models
Lawless, J. F., & Singhal, K. (1978) · Biometrics · doi:10.2307/2530022 · Cited by 243 · see citing works
Machine-Learning Algorithms Predict Graft Failure After Liver Transplantation
Lau, L., Kankanige, Y., Rubinstein, B. I. P., Jones, R. M., Christophi, C., Muralidharan, V., & Bailey, J. (2016) · Transplantation · doi:10.1097/tp.0000000000001600 · Cited by 170 · see citing works
Why Is Organ Transplantation Clinically Important?Preprint
Grinyo, J. M. (2013) · Cold Spring Harbor Perspectives in Medicine · doi:10.1101/cshperspect.a014985 · Cited by 159 · see citing works
Briceño, J., Cruz-Ramírez, M., Prieto, M., Navasa, M., Ortiz de Urbina, J., Orti, R., Gómez-Bravo, M.-Á., Otero, A., Varo, E., Tomé, S., et al. (2014) · Journal of Hepatology · doi:10.1016/j.jhep.2014.05.039 · Cited by 152 · see citing works
Lisboa, P. J. G., Wong, H., Harris, P., & Swindell, R. (2003) · Artificial Intelligence in Medicine · doi:10.1016/s0933-3657(03)00033-2 · Cited by 149 · see citing works
Survival analysis and neural nets
Liestbl, K., Andersen, P. K., & Andersen, U. (1994) · Statistics in Medicine · doi:10.1002/sim.4780131202 · Cited by 114 · see citing works
Xiaowei, S., Mitnitski, A., Cox, J. L., et al. (2004) · Studies in health technology and informatics · doi:10.3233/978-1-60750-949-3-736 · Cited by 105 · see citing works
Use of Neural Networks in Predicting the Risk of Coronary Artery Disease
Lapuerta, P., Azen, S. P., & LABREE, L. D. (1995) · Computers and Biomedical Research · doi:10.1006/cbmr.1995.1004 · Cited by 102 · see citing works
Journal of Probability and Statistics
(2009) · Journal of Probability and Statistics · doi:10.1155/jps · Cited by 90 · see citing works
Xiang, A. H., Lapuerta, P., Ryutov, A., Buckley, J. D., & Azen, S. P. (2000) · Computational Statistics & Data Analysis · doi:10.1016/s0167-9473(99)00098-5 · Cited by 86 · see citing works
A general framework for neural network models on censored survival data
Biganzoli, E. M., Boracchi, P., & Marubini, E. (2002) · Neural Networks · doi:10.1016/s0893-6080(01)00131-9 · Cited by 85 · see citing works
Machine learning in clinical practice: prospects and pitfalls
Scott, I. A., Cook, D., Coiera, E. W., & Richards, B. (2019) · The Medical Journal of Australia · doi:10.5694/mja2.50294 · Cited by 50 · see citing works
Predictive Capacity of Risk Models in Liver Transplantation
de Boer, J. D., Putter, H., Blok, J. J., Alwayn, I. P. J., van Hoek, B., & Braat, A. E. (2019) · Transplantation Direct · doi:10.1097/txd.0000000000000896 · Cited by 30 · see citing works
Self-citationDouble-blind evaluation and benchmarking of survival models in a multi-centre study
Taktak, A. F. G., Antolini, L., Aung, M. H., Boracchi, P., Campbell, I., Damato, B. E., Ifeachor, E., Lama, N., Lisboa, P. J. G., Setzkorn, C., et al. (2006) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2006.10.001 · Cited by 25 · see citing works
Jones, A. S., Taktak, A. F. G., Helliwell, T. R., et al. (2006) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-006-0021-2 · Cited by 21 · see citing works
Blok, J. J., Putter, H., Metselaar, H. J., Porte, R. J., Gonella, F., de Jonge, J., van den Berg, A. P., van der Zande, J., de Boer, J. D., van Hoek, B., et al. (2018) · Transplantation Direct · doi:10.1097/txd.0000000000000822 · Cited by 14 · see citing works
Self-citationNonlinear Survival Regression Using Artificial Neural Network
Biglarian, A., Bakhshi, E., Baghestani, A. R., et al. (2013) · Journal of Probability and Statistics · doi:10.1155/2013/753930 · Cited by 12 · see citing works
Multiple imputation of missing blood pressure covariates in survival analysis
van Buuren, S., Boshuizen, H. C., & Knook, D. L. (1999) · Statistics in Medicine · doi:10.1002/(sici)1097-0258(19990330)18:6
Harrell, F., Jr. (1996) · doi:10.1002/(sici)1097-0258(19960229)15:4
Tibshirani (1997) · doi:10.1002/(sici)1097-0258(19970228)16:4
Flagged first, then by citations
Research works printed but not linked: 3
- [23]Kantidakis G, Lancia C, Fiocco M. Prediction Models for Liver Transplantation - Comparisons Between Cox Models and Machine Learning Techniques [abstract OC30-4]: 40th Annual Conference of the International Society for Clinical Biostatistics; 2019, pp. 343–4. https://kuleuvencongres.be/iscb40/images/iscb40-2019-e-versie.pdf.
- [35]Minsky M Papert S Perceptrons; an Introduction to Computational Geometry. (Book edition 1) 1969 Cambridge MIT Press
- [41]Houwelingen JCv Putter H Dynamic Prediction in Clinical Survival Analysis. (Book edition 1) 2012 Boca, Raton CRC Press
Cited by
Sorted by citations
Artificial Intelligence: Present and Future Potential for Solid Organ Transplantation
Peloso, A., Moeckli, B., Delaune, V., Oldani, G., Andres, A., & Compagnon, P. (2022) · Transplant International · doi:10.3389/ti.2022.10640 · Cited by 67 · see citing works
Machine Learning Applications in Solid Organ Transplantation and Related Complications
Balch, J. A., Delitto, D., Tighe, P. J., Zarrinpar, A., Efron, P. A., Rashidi, P., Upchurch, G. R., Bihorac, A., & Loftus, T. J. (2021) · Frontiers in Immunology · doi:10.3389/fimmu.2021.739728 · Cited by 33 · see citing works
- Authors
- Kantidakis, G., Putter, H., Lancia, C., Boer, J. D., Braat, A. E., & Fiocco, M.
- Affiliations
- Leiden University (NL)
- Leiden University Medical Center (NL)
- Corresponding author
- Kantidakis, Georgios
- Venue
- BMC Medical Research Methodology
- Published
- 2020-11-16
- Volume
- 20
- Issue
- 1
- Pages
- 277-277
- ISSN
- 1471-2288 (ISSN-L)
- PMC
- PMC7667810
- OpenAlex
- W3105277854
- Cited by
- 75
- References
- 54
- Citations by year
- 2026: 10 · 2025: 16 · 2024: 17 · 2023: 13 · 2022: 12 · 2021: 7
- FWCI
- 6.98
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Research Methodology (publishedVersion, cc-by)
- Research Square (acceptedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Leiden Repository (Leiden University) (submittedVersion, cc-by)
- Data Archiving and Networked Services (DANS) (submittedVersion, cc-by)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- European Organisation for Research and Treatment of Cancer
- Universiteit Leiden
- Topics
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Medicine › Surgery › Organ Transplantation Techniques and Outcomes
- Keywords
- liver transplantation, graft survival prediction, machine learning, Cox proportional hazards model, random survival forest, LASSO, backward selection, prognostic factors, model interpretability, calibration, Scientific Registry of Transplant Recipients
- MeSH terms
- Machine Learning, Humans, Retrospective Studies, Proportional Hazards Models, Liver Transplantation, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
