Long-term Mortality Risk Stratification of Liver Transplant Recipients: Real-Time Application of Deep Learning Algorithms on Longitudinal Data
Open accessAbstract
BACKGROUND: Survival of liver transplant recipients beyond 1 year since transplantation is compromised by an increased risk of cancer, cardiovascular events, infection, and graft failure. Few clinical tools are available to identify patients at risk of these complications, which would flag them for screening tests and potentially life-saving interventions. In this retrospective analysis, we aimed to assess the ability of deep learning algorithms of longitudinal data from two prospective cohorts to predict complications resulting in death after liver transplantation over multiple timeframes, compared with logistic regression models. METHODS: In this machine learning analysis, model development was done on a set of 42 146 liver transplant recipients (mean age 48·6 years [SD 17·3]; 17 196 [40·8%] women) from the Scientific Registry of Transplant Recipients (SRTR) in the USA. Transferability of the model was further evaluated by fine-tuning on a dataset from the University Health Network (UHN) in Canada (n=3269; mean age 52·5 years [11·1]; 1079 [33·0%] women). The primary outcome was cause of death, as recorded in the databases, due to cardiovascular causes, infection, graft failure, or cancer, within 1 year and 5 years of each follow-up examination after transplantation. We compared the performance of four deep learning models against logistic regression, assessing performance using the area under the receiver operating characteristic curve (AUROC). FINDINGS: In both datasets, deep learning models outperformed logistic regression, with the Transformer model achieving the highest AUROCs in both datasets (p<0·0001). The AUROC for the Transformer model across all outcomes in the SRTR dataset was 0·804 (99% CI 0·795-0·854) for 1-year predictions and 0·733 (0·729-0·769) for 5-year predictions. In the UHN dataset, the AUROC for the top-performing deep learning model was 0·807 (0·795-0·842) for 1-year predictions and 0·722 (0·705-0·764) for 5-year predictions. AUROCs ranged from 0·695 (0·680-0·713) for prediction of death from infection within 5 years to 0·859 (0·847-0·871) for prediction of death by graft failure within 1 year. INTERPRETATION: Deep learning algorithms can incorporate longitudinal information to continuously predict long-term outcomes after liver transplantation, outperforming logistic regression models. Physicians could use these algorithms at routine follow-up visits to identify liver transplant recipients at risk for adverse outcomes and prevent these complications by modifying management based on ranked features. FUNDING: Canadian Donation and Transplant Research Program, CIFAR AI Chairs Program.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Nitski et al. analyzed large retrospective datasets with machine learning algorithms to predict mortality in liver transplant patients.
“Taking a more general approach, Nitski et al. analyzed large retrospective datasets with machine learning algorithms to predict mortality in liver transplant patients (34).”
From Artificial Intelligence: Present and Future Potential for Solid Organ Transplantation (2022) · cites this paper as [34]
The Nitski model provided meaningful predictions based on readily available data such as graft age, blood values, donor age, and postoperative complications.
“Interestingly, the model provided meaningful predictions based on readily available data such as graft age, blood values, donor age, and postoperative complications, making a potential clinical implementation relatively straightforward.”
From Artificial Intelligence: Present and Future Potential for Solid Organ Transplantation (2022) · cites this paper as [34]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404 · see citing works
Jakubův, J., Chvalovský, K., Goertzel, Z., et al. (2023) · DROPS (Schloss Dagstuhl – Leibniz Center for Informatics) · doi:10.4230/lipics.itp.2023.19 · Cited by 76,648 · see citing works
The use of the area under the ROC curve in the evaluation of machine learning algorithms
Bradley, A. P. (1997) · Pattern Recognition · doi:10.1016/s0031-3203(96)00142-2 · Cited by 7,217 · see citing works
Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network
Sherstinsky, A. (2020) · Physica D: Nonlinear Phenomena · doi:10.1016/j.physd.2019.132306 · Cited by 5,680 · see citing works
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652 · see citing works
Big Data and Machine Learning in Health Care
Beam, A. L., & Kohane, I. S. (2018) · JAMA · doi:10.1001/jama.2017.18391 · Cited by 2,506 · see citing works
Characteristics Associated with Liver Graft Failure: The Concept of a Donor Risk Index
Feng, S., Goodrich, N. P., Bragg‐Gresham, J. L., Dykstra, D. M., Punch, J. D., DebRoy, M. A., Greenstein, S., & Merion, R. M. (2006) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2006.01242.x · Cited by 1,959 · see citing works
Watt, K. D. S., Pedersen, R. A., Kremers, W. K., Heimbach, J. K., & Charlton, M. (2010) · American Journal of Transplantation · doi:10.1111/j.1600-6143.2010.03126.x · Cited by 784 · see citing works
Self-citationYounossi, Z. M., Stepanova, M., Ong, J. P., Trimble, G., Alqahtani, S. A., Younossi, I. M., Ahmed, A., Racila, A., & Henry, L. L. (2020) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2020.05.064 · Cited by 587 · see citing works
The expressive power of neural networks: a view from the width
Zhou, L., Pu, H., Wang, F., Hu, Z., & Wang, L. (2017) · neural information processing systems · Cited by 516 · see citing works
Thorsen-Meyer, H.-C., Nielsen, A. B., Nielsen, A. P., Kaas-Hansen, B. S., Toft, P., Schierbeck, J., Strøm, T., Chmura, P. J., Heimann, M., Dybdahl, L., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30018-2 · Cited by 336 · see citing works
Charlton, M., Levitsky, J., Aqel, B. A., O’Grady, J., Hemibach, J., Rinella, M. E., Fung, J. J., Ghabril, M., Thomason, R., Burra, P., et al. (2018) · Transplantation · doi:10.1097/tp.0000000000002147 · Cited by 311 · see citing works
Self-citationRodríguez‐Perálvarez, M., Tsochatzis, E. A., Naveas, M. C., Pieri, G., García-Caparrós, C., O’Beirne, J. P., Poyato, A., Ferrín, G., Montero-Álvarez, J. L., Patch, D., et al. (2013) · Journal of Hepatology · doi:10.1016/j.jhep.2013.07.012 · Cited by 264 · see citing works
Acute Rejection Increases Risk of Graft Failure and Death in Recent Liver Transplant Recipients
Levitsky, J., Goldberg, D., Smith, A. R., Mansfield, S. A., Gillespie, B. W., Merion, R. M., Lok, A. S. F., Levy, G., Kulik, L., Abecassis, M., et al. (2017) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2016.07.035 · Cited by 236 · see citing works
Self-citationApplying Machine Learning in Liver Disease and Transplantation: A Comprehensive Review
Spann, A., Yasodhara, A., Kang, J., Watt, K. D. S., Wang, B., Goldenberg, A., & Bhat, M. (2020) · Hepatology · doi:10.1002/hep.31103 · Cited by 202 · see citing works
Self-citationMachine-Learning Algorithms Predict Graft Failure After Liver Transplantation
Lau, L., Kankanige, Y., Rubinstein, B. I. P., Jones, R. M., Christophi, C., Muralidharan, V., & Bailey, J. (2016) · Transplantation · doi:10.1097/tp.0000000000001600 · Cited by 170 · see citing works
Kuo, H., Sampaio, M. S., Ye, X., Reddy, P. N., Martin, P., & Bunnapradist, S. (2010) · Transplantation · doi:10.1097/tp.0b013e3181d2fec1 · Cited by 154 · see citing works
Briceño, J., Cruz-Ramírez, M., Prieto, M., Navasa, M., Ortiz de Urbina, J., Orti, R., Gómez-Bravo, M.-Á., Otero, A., Varo, E., Tomé, S., et al. (2014) · Journal of Hepatology · doi:10.1016/j.jhep.2014.05.039 · Cited by 152 · see citing works
Medication misuse, nonadherence, and clinical outcomes among liver transplant recipients
Serper, M., Patzer, R. E., Reese, P. P., Przytula, K., Koval, R. R., Ladner, D. P., Levitsky, J., Abecassis, M. M., & Wolf, M. S. (2014) · Liver Transplantation · doi:10.1002/lt.24023 · Cited by 105 · see citing works
Self-citationTacrolimus and the Risk of Solid Cancers After Liver Transplant: A Dose Effect Relationship
Carenco, C., Assénat, E., Faure, S., Duny, Y., Danan, G., Bismuth, M., Herrero, A., Jung, B., Bedoya, J. U., Jaber, S., et al. (2015) · American Journal of Transplantation · doi:10.1111/ajt.13018 · Cited by 94 · see citing works
Quillin, R. C., Wilson, G. C., Wima, K., et al. (2014) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2014.05.020 · Cited by 88 · see citing works
A machine learning-based approach to prognostic analysis of thoracic transplantations
Delen, D., Oztekin, A., & Kong, Z. (2010) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2010.01.002 · Cited by 74 · see citing works
How to improve long‐term outcome after liver transplantation?
Durand, F. (2018) · Liver International · doi:10.1111/liv.13651 · Cited by 73 · see citing works
Improving prediction of heart transplantation outcome using deep learning techniques
Medved, D., Ohlsson, M., Höglund, P., Andersson, B., Nugues, P., & Nilsson, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-21417-7 · Cited by 71 · see citing works
Outcome of liver transplantation using donors older than 60 year of age
Serrano, T., Garcia‐Gil, A., Arenas, J. D., et al. (2009) · Clinical Transplantation · doi:10.1111/j.1399-0012.2009.01135.x · Cited by 62 · see citing works
Infection in Solid-Organ Transplant Recipients
Cross, T., Berry, P. A., & Burroughs, A. K. (2008) · New England Journal of Medicine · doi:10.1056/nejmc080060 · Cited by 62 · see citing works
Bhat, V., Tazari, M., Watt, K. D. S., & Bhat, M. (2018) · Mayo Clinic Proceedings · doi:10.1016/j.mayocp.2018.06.020 · Cited by 59 · see citing works
Self-citationBhat, M., Mara, K. C., Dierkhising, R., et al. (2018) · Transplantation · doi:10.1097/tp.0000000000002113 · Cited by 47 · see citing works
Self-citationSerum fibrosis biomarkers predict death and graft loss in liver transplantation recipients
Bhat, M., Ghali, P., Rollet‐Kurhajec, K. C., Bhat, A., Wong, P. Y. N., Deschênes, M., & Sebastiani, G. (2015) · Liver Transplantation · doi:10.1002/lt.24217 · Cited by 41 · see citing works
Self-citationSastre, L., Garcı́a, R., Gándara, J. G., Ruíz, P., Lombardo, J., Colmenero, J., Navasa, M., & Crespo, G. (2019) · Transplantation · doi:10.1097/tp.0000000000002852 · Cited by 30 · see citing works
Wallace, D., Walker, K., Charman, S. C., et al. (2019) · Transplantation · doi:10.1097/tp.0000000000002559 · Cited by 22 · see citing works
Wan, Q., Ye, Q. F., Ming, Y., et al. (2013) · Transplantation Proceedings · doi:10.1016/j.transproceed.2012.06.080 · Cited by 20 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Artificial Intelligence: Present and Future Potential for Solid Organ Transplantation
Peloso, A., Moeckli, B., Delaune, V., Oldani, G., Andres, A., & Compagnon, P. (2022) · Transplant International · doi:10.3389/ti.2022.10640 · Cited by 67 · see citing works
Who does the model learn from?
Charpignon, M.-L., Celi, L. A., & Samuel, M. C. (2021) · The Lancet Digital Health · doi:10.1016/s2589-7500(21)00057-1 · Cited by 4 · see citing works
- Authors
- Nitski, O., Azhie, A., Qazi-Arisar, F. A., Wang, X., Ma, S., Lilly, L., Watt, K. D., Levitsky, J., Asrani, S. K., Lee, D. S., Rubin, B. B., Bhat, M., & Wang, B.
- Affiliations
- University Health Network (CA)
- University of Toronto (CA)
- Mayo Clinic (US)
- Northwestern University (US)
- Baylor University Medical Center (US)
- Corresponding author
- Wang, Bo
- ORCID
- Venue
- The Lancet Digital Health
- Publisher
- Elsevier BV
- Published
- 2021-05
- Volume
- 3
- Issue
- 5
- Pages
- e295-e305
- ISSN
- 2589-7500 (ISSN-L)
- OpenAlex
- W3154550454
- Cited by
- 110
- References
- 32
- Citations by year
- 2026: 9 · 2025: 35 · 2024: 27 · 2023: 24 · 2022: 12 · 2021: 3
- FWCI
- 15.48
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- The Lancet Digital Health (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- Canadian Liver Foundation
- Toronto General and Western Hospital Foundation
- American Society of Transplantation
- Canadian Institute for Advanced Research
- Topics
- Health Sciences › Medicine › Surgery › Organ Transplantation Techniques and Outcomes
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Keywords
- liver transplantation, mortality prediction, risk stratification, deep learning, transformer models, longitudinal data, cause-specific mortality, graft failure, post-transplant infection, post-transplant malignancy, cardiovascular mortality, model transferability, Scientific Registry of Transplant Recipients, University Health Network, logistic regression comparison, long-term survival, model interpretability, clinical decision support
- MeSH terms
- Deep Learning, Adult, Aged, Algorithms, Canada, Female, Humans, Male, Middle Aged, Predictive Value of Tests, Retrospective Studies, ROC Curve, United States, Logistic Models, Liver Transplantation, Databases, Factual, Risk Assessment, Area Under Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
