Modeling asynchronous event sequences with RNNs
Open accessAbstract
Sequences of events have often been modeled with computational techniques, but typical preprocessing steps and problem settings do not explicitly address the ramifications of timestamped events. Clinical data, such as is found in electronic health records (EHRs), typically comes with timestamp information. In this work, we define event sequences and their properties: synchronicity, evenness, and co-cardinality; we then show how asynchronous, uneven, and multi-cardinal problem settings can support explicit accountings of relative time. Our evaluation uses the temporally sensitive clinical use case of pediatric asthma, which is a chronic disease with symptoms (and lack thereof) evolving over time. We show several approaches to explicitly incorporating relative time into a recurrent neural network (RNN) model that improve the overall classification of patients into those with no asthma, those with persistent asthma, those in long-term remission, and those who have experienced relapse. We also compare and contrast these results with those in an inpatient intensive care setting.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Techniques and methods exist by which patients can be classified effectively as those who have asthma and those who do not have those in long-term remission.
“There are techniques and methods by which patients can be classified effectively as those who have asthma and those who do not have those in long-term remission [66].”
From Influential Usage of Big Data and Artificial Intelligence in Healthcare (2021) · cites this paper as [66]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
A tutorial on hidden Markov models and selected applications in speech recognition
Rabiner, L. R. (1989) · Proceedings of the IEEE · doi:10.1109/5.18626 · Cited by 22,743 · see citing works
Speech recognition with deep recurrent neural networks
Graves, A., Mohamed, A.-R., & Hinton, G. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638947 · Cited by 8,893 · see citing works
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141 · see citing works
Proceedings of the 19th International Joint Conference on Artificial Intelligence
Pujol, J. M., Delgado, J., Ramon Sangüesa, A., & Flache, A. (2005) · International Joint Conference on Artificial Intelligence · Cited by 5,771 · see citing works
Bayesian Networks and Decision Graphs
Jensen, F. V., & Nielsen, T. D. (2007) · Information science and statistics · doi:10.1007/978-0-387-68282-2 · Cited by 4,157 · see citing works
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data
Heckerman, D. E., Geiger, D., & Chickering, D. M. (1995) · Machine Learning · doi:10.1007/bf00994016 · Cited by 4,082 · see citing works
Learning Bayesian Networks: The Combination of Knowledge and Statistical Data
Heckerman, D. E., Geiger, D., & Chickering, D. M. (1995) · Machine Learning · doi:10.1023/a:1022623210503 · Cited by 3,251 · see citing works
Dynamic bayesian networks: representation, inference and learning
Murphy, K. P., & Russell, S. (2002) · Cited by 2,563 · see citing works
Recurrent Neural Networks for Multivariate Time Series with Missing Values
Che, Z., Purushotham, S., Cho, K., Sontag, D., & Liu, Y. (2018) · Scientific Reports · doi:10.1038/s41598-018-24271-9 · Cited by 2,370 · see citing works
A simplified acute physiology score for ICU patients
Gall, J. L., Loirat, P., Alpérovitch, A., GLASER, P., Granthil, C., Mathieu, D., MERCIER, P., THOMAS, R., & Villers, D. (1984) · Critical Care Medicine · doi:10.1097/00003246-198411000-00012 · Cited by 1,247 · see citing works
IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2012
Müller, A., Sejdinović, D., & Piechocki, R. J. (2012) · Cited by 1,001 · see citing works
A model for reasoning about persistence and causation
Dean, T., & Kanazawa, K. (1989) · Computational Intelligence · doi:10.1111/j.1467-8640.1989.tb00324.x · Cited by 999 · see citing works
Ephraim, Y., & Merhav, N. (2002) · IEEE Transactions on Information Theory · doi:10.1109/tit.2002.1003838 · Cited by 782 · see citing works
Yu, S. (2009) · Artificial Intelligence · doi:10.1016/j.artint.2009.11.011 · Cited by 737 · see citing works
Bayesian Networks and Decision Graphs
Ahmed, S. (2008) · Technometrics · doi:10.1198/tech.2008.s543 · Cited by 725 · see citing works
Outcome of Asthma and Wheezing in the First 6 Years of Life
Morgan, W. J., Stern, D. A., Sherrill, D. L., Guerra, S., Holberg, C. J., Guilbert, T. W., Taussig, L. M., Wright, A. L., & Martinez, F. D. (2005) · American Journal of Respiratory and Critical Care Medicine · doi:10.1164/rccm.200504-525oc · Cited by 650 · see citing works
Patient Subtyping via Time-Aware LSTM Networks
Baytaş, İ. M., Xiao, C., Zhang, X., Wang, F., Jain, A. K., & Zhou, J. (2017) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3097983.3097997 · Cited by 646 · see citing works
The detection and estimation of long memory in stochastic volatility
Breidt, F. J., Crato, N., & de Lima, P. (1998) · Journal of Econometrics · doi:10.1016/s0304-4076(97)00072-9 · Cited by 618 · see citing works
Ma, F., Chitta, R., Zhou, J., You, Q., Sun, T., & Gao, J. (2017) · Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining · doi:10.1145/3097983.3098088 · Cited by 576 · see citing works
A Framework for Knowledge-Based Temporal Abstraction
Shahar, Y. (1997) · Artificial Intelligence · doi:10.1016/s0004-3702(96)00025-2 · Cited by 461 · see citing works
Predicting healthcare trajectories from medical records: a deep learning approach
Pham, T., Tran, T., Phung, D., & Venkatesh, S. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2017.04.001 · Cited by 447 · see citing works
Knowledge-based temporal abstraction in clinical domains
D, Y. S. M., & Musen, M. A. (1996) · Artificial Intelligence in Medicine · doi:10.1016/0933-3657(95)00036-4 · Cited by 313 · see citing works
An Application of Recurrent Neural Networks to Discriminative Keyword Spotting
Fernández, S., Graves, A., & Schmidhuber, J. (2007) · Lecture notes in computer science · doi:10.1007/978-3-540-74695-9_23 · Cited by 251 · see citing works
2021 Computing in Cardiology (CinC)
(2021) · doi:10.23919/cinc53138.2021 · Cited by 241 · see citing works
Recurrent Neural Networks for Multivariate Time Series with Missing ValuesPreprint
Che, Z., Purushotham, S., Cho, K., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.01865 · Cited by 230 · see citing works
Rea, S., Pathak, J., Savova, G., Oniki, T. A., Westberg, L., Beebe, C. E., Tao, C., Parker, C. G., Haug, P. J., Huff, S. M., et al. (2012) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2012.01.009 · Cited by 206 · see citing works
Dynamic Network Models for Forecasting
Dagum, P., Galper, A. R., & Horvitz, E. (1992) · Uncertainty in Artificial Intelligence · doi:10.1016/b978-1-4832-8287-9.50010-4 · Cited by 200 · see citing works
Integration of Early Physiological Responses Predicts Later Illness Severity in Preterm Infants
Saria, S., Rajani, A. K., Gould, J. B., Koller, D., & Penn, A. A. (2010) · Science Translational Medicine · doi:10.1126/scitranslmed.3001304 · Cited by 177 · see citing works
Silva, I., Moody, G. B., Scott, D., Celi, L. A., & Mark, R. G. (2012) · PubMed · Cited by 167 · see citing works
Silva, I., Moody, G. B., Scott, D. J., et al. (2012) · DSpace@MIT (Massachusetts Institute of Technology) · Cited by 133 · see citing works
Distinguishing Asthma Phenotypes Using Machine Learning Approaches
Howard, R., Rattray, M., Prosperi, M., & Čustović, A. (2015) · Current Allergy and Asthma Reports · doi:10.1007/s11882-015-0542-0 · Cited by 120 · see citing works
Classification-driven temporal discretization of multivariate time series
Moskovitch, R., & Shahar, Y. (2015) · Data mining and knowledge discovery · doi:10.1007/s10618-014-0380-z · Cited by 116 · see citing works
Chen, Y., Carroll, R. J., Hinz, E. M., Shah, A., Eyler, A. E., Denny, J. C., & Xu, H. (2013) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-001945 · Cited by 111 · see citing works
Medical temporal-knowledge discovery via temporal abstraction.
Moskovitch, R., & Shaḥar, Y. (2009) · PubMed · Cited by 107 · see citing works
Predictors of remitting, periodic, and persistent childhood asthma
Covar, R. A., Strunk, R. C., Zeiger, R. S., Wilson, L., Liu, A. H., Weiss, S. T., Tonascia, J. A., Spahn, J. D., & Szefler, S. J. (2010) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2009.10.037 · Cited by 106 · see citing works
Xu, M., Tantisira, K. G., Wu, A. C., Litonjua, A. A., Chu, J., Himes, B. E., Damask, A., & Weiss, S. T. (2011) · BMC Medical Genetics · doi:10.1186/1471-2350-12-90 · Cited by 98 · see citing works
An autonomous mobile system for the management of COPD
van der Heijden, M., Lucas, P., Lijnse, B., Heijdra, Y. F., & Schermer, T. R. (2013) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2013.03.003 · Cited by 98 · see citing works
Learning from heterogeneous temporal data in electronic health records
Zhao, J., Papapetrou, P., Asker, L., & Boström, H. (2016) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.11.006 · Cited by 98 · see citing works
Wu, S., Sohn, S., Ravikumar, K. E., Wagholikar, K. B., Jonnalagadda, S., Liu, H., & Juhn, Y. J. (2013) · Annals of Allergy Asthma & Immunology · doi:10.1016/j.anai.2013.07.022 · Cited by 79 · see citing works
Self-citationProceedings of the Twenty-Sixth Conference on Uncertainty in Artificial Intelligence (2010)Preprint
Grünwald, P., & Spirtes, P. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1205.2597 · Cited by 77 · see citing works
Parameterizing time in electronic health record studies
Hripcsak, G. M., Albers, D. J., & Perotte, A. (2015) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocu051 · Cited by 75 · see citing works
Evolino: hybrid neuroevolution / optimal linear search for sequence learning
Schmidhuber, J., Wierstra, D., & Gomez, F. J. (2005) · International Joint Conference on Artificial Intelligence · Cited by 74 · see citing works
Amaral, J. L. M., Lopes, A. J., Veiga, J., et al. (2017) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2017.03.023 · Cited by 63 · see citing works
Temporal electronic phenotyping by mining careflows of breast cancer patients
Dagliati, A., Sacchi, L., Zambelli, A., Tibollo, V., Pavesi, L., Holmes, J. H., & Bellazzi, R. (2017) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2016.12.012 · Cited by 60 · see citing works
Predicting phenotypes of asthma and eczema with machine learning
Prosperi, M., Marinho, S., Simpson, A., Čustović, A., & Buchan, I. E. (2014) · BMC Medical Genomics · doi:10.1186/1755-8794-7-s1-s7 · Cited by 59 · see citing works
Patient specific predictions in the intensive care unit using a Bayesian ensemblePreprint
Johnson, A. E. W., Dunkley, N., Mayaud, L., Tsanas, A., Kramer, A. A., & Clifford, G. D. (2012) · Cited by 54 · see citing works
Lipton, Z. C., Kale, D. C., & Wetzel, R. C. (2016) · arXiv (Cornell University) · Cited by 39 · see citing works
Orphanou, K., Dagliati, A., Sacchi, L., Stassopoulou, A., Keravnou, E., & Bellazzi, R. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.03.002 · Cited by 35 · see citing works
A Comparative Study of Machine Learning Algorithms as Expert Systems in Medical Diagnosis (Asthma)
Prasad, B., Prasad, P. E. S. N. K., & Yeruva, S. (2010) · Communications in computer and information science · doi:10.1007/978-3-642-17857-3_56 · Cited by 30 · see citing works
Deep Learning Solutions to Computational Phenotyping in Health Care
Che, Z., & Liu, Y. (2017) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2017.156 · Cited by 29 · see citing works
Patient-level temporal aggregation for text-based asthma status ascertainment
Wu, S. T., Juhn, Y. J., Sohn, S., et al. (2014) · Journal of the American Medical Informatics Association · doi:10.1136/amiajnl-2013-002463 · Cited by 28 · see citing works
Self-citationAscertainment of asthma prognosis using natural language processing from electronic medical records
Sohn, S., Wi, C., Wu, S., et al. (2018) · Journal of Allergy and Clinical Immunology · doi:10.1016/j.jaci.2017.12.1003 · Cited by 25 · see citing works
Self-citationProceedings of the Eighth international conference on Uncertainty in artificial intelligence
Dubois, D., Wellman, M. P., D’Ambrosio, B., et al. (1992) · Cited by 22 · see citing works
Characteristics of Children with Asthma Who Achieved Remissionof Asthma
Javed, A., Yoo, K. H., Agarwal, K., Jacobson, R. M., Li, X., & Juhn, Y. J. (2013) · Journal of Asthma · doi:10.3109/02770903.2013.787625 · Cited by 19 · see citing works
Self-citationUsing machine learning to predict asthma exacerbations.
Finkelstein, J., & Gangopadhyay, A. (2007) · PubMed · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [2]Amodei D, Anubhai R, Battenberg E, Case C, Casper J, Catanzaro B, Chen J, Chrzanowski M, Coates A, Diamos G, 2015 Deep speech 2: End-to-end speech recognition in english and mandarin. arXiv preprint arXiv:1512.02595. · arXiv
- [9]Cui Z, Chen W, Chen Y, 2016 Multi-scale convolutional neural networks for time series classification. arXiv preprint arXiv: 1603 06995.
- [25]Johnson MJ, Willsky AS, 2013 Bayesian nonparametric hidden semi-markov models. Journal of Machine Learning Research 14 (Feb), 673– 701.
- [28]Liu J, Zhao K, Kusy B, Wen J. r., Jurdak R, 2015 Temporal embedding in convolutional neural networks for robust learning of abstract snippets. arXiv preprint arXiv:1502.05113. · arXiv
- [53]Yi S, Ju J, Yoon M-K, Choi J, 2017 Grouped convolutional neural networks for multivariate time series. arXiv preprint arXiv:1703.09938. · arXiv
Cited by
Sorted by citations
Influential Usage of Big Data and Artificial Intelligence in Healthcare
Yang, Y. C., Islam, S. U., Noor, A., Khan, S., Afsar, W., & Nazir, S. (2021) · Computational and Mathematical Methods in Medicine · doi:10.1155/2021/5812499 · Cited by 68 · see citing works
“There are techniques and methods by which patients can be classified effectively as those who have asthma and those who do not have those in long-term remission [66].”
MethodRetracted
- Authors
- Wu, S., Liu, S., Sohn, S., Moon, S., Wi, C.-I., Juhn, Y., & Liu, H.
- Affiliations
- Mayo Clinic (US)
- Corresponding author
- Wu, Stephen
- Venue
- Journal of Biomedical Informatics
- Publisher
- Elsevier BV
- Published
- 2018-07
- Volume
- 83
- Pages
- 167-177
- ISSN
- 1532-0464 (ISSN-L)
- PMC
- PMC6103779
- OpenAlex
- W2806153517
- Cited by
- 45
- References
- 58
- Citations by year
- 2026: 4 · 2025: 6 · 2024: 3 · 2023: 6 · 2022: 4 · 2021: 8 · 2020: 7 · 2019: 7
- FWCI
- 3.10
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Licence
- Locations
- Journal of Biomedical Informatics (publishedVersion)
- PubMed (publishedVersion)
- PubMed Central (acceptedVersion)
- Funders
- NIH (R21AI116839, R01LM011934)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Physical Sciences › Computer Science › Signal Processing › Time Series Analysis and Forecasting
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- recurrent neural networks, electronic health records, childhood asthma, asthma classification, asthma relapse, synchronicity, intensive care, temporal sequence modeling
- MeSH terms
- Asthma, Child, Child, Preschool, Computer Simulation, Humans, Infant, Intensive Care Units, Recurrence, Neural Networks, Computer, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
