Learning Tasks for Multitask Learning: Heterogenous Patient Populations in the ICU
Abstract
Machine learning approaches have been effective in predicting adverse outcomes in different clinical settings. These models are often developed and evaluated on datasets with heterogeneous patient populations. However, good predictive performance on the aggregate population does not imply good performance for specific groups. In this work, we present a two-step framework to 1) learn relevant patient subgroups, and 2) predict an outcome for separate patient populations in a multi-task framework, where each population is a separate task. We demonstrate how to discover relevant groups in an unsupervised way with a sequence-to-sequence autoencoder. We show that using these groups in a multi-task framework leads to better predictive performance of in-hospital mortality both across groups and overall. We also highlight the need for more granular evaluation of performance when dealing with heterogeneous populations.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
For outcomes that happen in hospital, such as inpatient mortality, length-of-stay, and intervention prediction, observation windows at hour level are developed.
“On the contrary, for those outcomes that happen in hospital, such as inpatient mortality [14,57,61,65], length-of-stay [14,83], and intervention prediction [32,55,65], observation windows at hour level are developed.”
From Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review (2021) · cites this paper as [61]
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · see citing works
Individual Comparisons by Ranking Methods
Wilcoxon, F. (1945) · Biometrics Bulletin · doi:10.2307/3001968 · Cited by 15,354 · see citing works
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288 · see citing works
Learning long-term dependencies with gradient descent is difficult
Bengio, Y., Simard, P., & Frasconi, P. (1994) · IEEE Transactions on Neural Networks · doi:10.1109/72.279181 · Cited by 8,542 · see citing works
Scikit-learn: Machine Learning in Python
(2011) · Journal of Machine Learning Research · doi:10.5555/1953048.2078195 · Cited by 8,539 · see citing works
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A., Salakhutdinov, R., Zemel, R., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1502.03044 · Cited by 8,248 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Le Gall, J.-R. (1993) · JAMA: The Journal of the American Medical Association · doi:10.1001/jama.270.24.2957 · Cited by 7,773 · see citing works
Predicting the future - big data, machine learning, and clinical medicine
Obermeyer, Z., & Emanuel, E. J. (2016) · New England Journal of Medicine · doi:10.1056/nejmp1606181 · Cited by 3,652 · see citing works
(2018) · Cited by 3,510 · see citing works
A Survey on Multi-Task Learning
Zhang, Y., & Yang, Q. (2021) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2021.3070203 · Cited by 3,083 · see citing works
Teaching Machines to Read and ComprehendPreprint
Hermann, K. M., Kočiský, T., Grefenstette, E., Espeholt, L., Kay, W., Suleyman, M., & Blunsom, P. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1506.03340 · Cited by 2,779 · see citing works
An Overview of Multi-Task Learning in Deep Neural NetworksPreprint
Ruder, S. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1706.05098 · Cited by 2,409 · see citing works
Recurrent Neural Networks for Multivariate Time Series with Missing Values
Che, Z., Purushotham, S., Cho, K., Sontag, D., & Liu, Y. (2018) · Scientific Reports · doi:10.1038/s41598-018-24271-9 · Cited by 2,370 · see citing works
Attention-Based Models for Speech Recognition
Chorowski, J., Bahdanau, D., Serdyuk, D., Cho, K., & Bengio, Y. (2015) · arXiv · doi:10.48550/arxiv.1506.07503 · Cited by 2,021 · see citing works
Miotto, R., Li, L., Kidd, B. A., & Dudley, J. T. (2016) · Scientific Reports · doi:10.1038/srep26094 · Cited by 1,710 · see citing works
Automated Experiments on Ad Privacy Settings: A Tale of Opacity, Choice, and Discrimination
Datta, A., Tschantz, M. C., & Datta, A. (2015) · Proceedings on Privacy Enhancing Technologies · doi:10.1515/popets-2015-0007 · Cited by 676 · see citing works
Prediction Modeling Using EHR Data
Wu, J., Roy, J. A., & Stewart, W. F. (2010) · Medical Care · doi:10.1097/mlr.0b013e3181de9e17 · Cited by 374 · see citing works
Mortality predictions in the intensive care unit: Comparing physicians with scoring systems*
Sinuff, T., Adhikari, N. K. J., Cook, D. J., Schünemann, H. J., Griffith, L. E., Rocker, G. M., & Walter, S. D. (2006) · Critical Care Medicine · doi:10.1097/01.ccm.0000201881.58644.41 · Cited by 262 · see citing works
Deep Computational Phenotyping
Che, Z., Kale, D. C., Li, W., Bahadori, M. T., & Liu, Y. (2015) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2783258.2783365 · Cited by 247 · see citing works
Ho, J. C., Ghosh, J., & Sun, J. (2014) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623658 · Cited by 214 · see citing works
Ghassemi, M., Naumann, T., Doshi‐Velez, F., Brimmer, N., Joshi, R., Rumshisky, A., & Szolovits, P. (2014) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623742 · Cited by 198 · see citing works
Shankar, S., Halpern, Y., Breck, E. J., Atwood, J. R., Wilson, J., & Sculley, D. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1711.08536 · Cited by 175 · see citing works
Limestone: High-throughput candidate phenotype generation via tensor factorization
Ho, J. C., Ghosh, J., Steinhubl, S. R., Stewart, W. F., Denny, J. C., Malin, B., & Sun, J. (2014) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2014.07.001 · Cited by 145 · see citing works
Learning Probabilistic Phenotypes from Heterogeneous EHR Data
Pivovarov, R., Perotte, A. J., Grave, E., Angiolillo, J., Wiggins, C., & Elhadad, N. (2015) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2015.10.001 · Cited by 140 · see citing works
State of the art review: the data revolution in critical care
Ghassemi, M., Celi, L. A., & Stone, D. J. (2015) · Critical Care · doi:10.1186/s13054-015-0801-4 · Cited by 128 · see citing works
Multi-task Prediction of Disease Onsets from Longitudinal Laboratory Tests
Razavian, N., Marcus, J., & Sontag, D. A. (2016) · Cited by 107 · see citing works
Clinical Intervention Prediction and Understanding using Deep NetworksPreprint
Suresh, H., Hunt, N. J., Johnson, A. E. W., Celi, L. A., Peter Szolovits, B., & Ghassemi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.08498 · Cited by 81 · see citing works
Self-citationUnfolding physiological state: mortality modelling in intensive care units
Ghassemi, M., Doshi‐Velez, F., Joshi, R., Rumshisky, A., Peter Szolovits, B., Naumann, T., & Brimmer, N. (2014) · Cited by 57 · see citing works
Patient risk stratification with time-varying parameters: a multitask learning approach
Wiens, J., Guttag, J. V., & Horvitz, E. (2016) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 54 · see citing works
Self-citationWu, M., Ghassemi, M., Feng, M., Celi, L. A., Peter Szolovits, B., & Doshi‐Velez, F. (2016) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw138 · Cited by 50 · see citing works
Predicting intervention onset in the ICU with switching state space models.
Ghassemi, M., Wu, M., Hughes, M. C., Peter Szolovits, B., & Doshi‐Velez, F. (2017) · PubMed Central · Cited by 48 · see citing works
Predicting Clinical Outcomes Across Changing Electronic Health Record Systems
Gong, J. J., Naumann, T., Szolovits, P., & Guttag, J. V. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098064 · Cited by 34 · see citing works
Self-citationD’Errigo, P., Tosti, M. E., Fusco, D., Perucci, C. A., & Seccareccia, F. (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-29 · Cited by 33 · see citing works
F ORMULA : FactORized MUlti-task LeArning for task discovery in personalized medical models
Xu, J., Zhou, J., & Tan, P. (2015) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611974010.56 · Cited by 23 · see citing works
Exploring joint disease risk prediction.
Wang, X., Wang, F., Hu, J., & Sorrentino, R. (2014) · PubMed Central · Cited by 22 · see citing works
The Use of Autoencoders for Discovering Patient PhenotypesPreprint
Suresh, H., Szolovits, P., & Ghassemi, M. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.07004 · Cited by 20 · see citing works
Self-citationCorbin, M., Richiardi, L., Vermeulen, R. C. H., et al. (2012) · PLoS ONE · doi:10.1371/journal.pone.0038944 · Cited by 13 · see citing works
Personalized Risk Scoring for Critical Care Patients using Mixtures of Gaussian Process ExpertsPreprint
Alaa, A. M., Yoon, J., Hu, S. B., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.00959 · Cited by 12 · see citing works
Hierarchical models for multiple, rare outcomes using massive observational healthcare databases
Shaddox, T. R., Ryan, P. B., Schuemie, M. J., Madigan, D., & Suchard, M. A. (2016) · Statistical Analysis and Data Mining The ASA Data Science Journal · doi:10.1002/sam.11324 · Cited by 12 · see citing works
Ngufor, C. G., Upadhyaya, S. G., Murphree, D. H., Kor, D. J., & Pathak, J. (2015) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2015.7344836 · Cited by 10 · see citing works
Learning Implicit Tasks for Patient-Specific Risk Modeling in ICU
Nori, N., Kashima, H., Yamashita, K., et al. (2017) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v31i1.10766 · Cited by 10 · see citing works
A Heterogeneous Multi-Task Learning for Predicting RBC Transfusion and Perioperative Outcomes
Ngufor, C. G., Upadhyaya, S. G., Murphree, D. H., Madde, N. R., Kor, D. J., & Pathak, J. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19551-3_37 · Cited by 6 · see citing works
Patient risk stratification with time-varying parameters
WiensJenna, GuttagJohn, & HorvitzEric (2016) · Journal of Machine Learning Research · doi:10.5555/2946645.3007032 · Cited by 3 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Si, Y., Du, J., Li, Z., Jiang, X., Miller, T., Wang, F., Jim Zheng, W., & Roberts, K. (2021) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2020.103671 · Cited by 235 · see citing works
Vale, D., El-Sharif, A., & Ali, M. (2022) · AI and Ethics · doi:10.1007/s43681-022-00142-y · Cited by 124 · see citing works
A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle
Suresh, H., & Guttag, J. (2021) · Equity and Access in Algorithms, Mechanisms, and Optimization · doi:10.1145/3465416.3483305 · Cited by 82 · see citing works
Self-citationBardak, B., & Tan, M. (2021) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2021.102112 · Cited by 48 · see citing works
- Authors
- Suresh, H., Gong, J. J., & Guttag, J. V.
- Affiliations
- Massachusetts Institute of Technology (US)
- Venue
- Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
- Publisher
- ACM
- Published
- 2018-07-19
- Pages
- 802-810
- OpenAlex
- W4289257494
- Cited by
- 54
- References
- 46
- Citations by year
- 2026: 3 · 2025: 4 · 2024: 8 · 2023: 7 · 2022: 9 · 2021: 15 · 2020: 5 · 2019: 3
- FWCI
- 2.35
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) (publishedVersion)
- Funders
- Wistron Corporation
- National Institutes of Health (P50DC015446)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Physical Sciences › Computer Science › Artificial Intelligence › Domain Adaptation and Few-Shot Learning
- Keywords
- multi-task learning, patient subgroups, subgroup discovery, sequence-to-sequence autoencoder, unsupervised learning, in-hospital mortality prediction, heterogeneous patient populations, clinical outcome prediction
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
