Cited by 247Machine Learning in HealthcareElectronic Health Records SystemsArtificial Intelligence in Healthcare and Education

Deep learning for electronic health records: A comparative review of multiple deep neural architectures

Open access
Ayala Solares, Jose Roberto, Diletta Raimondi, Francesca Elisa, Zhu, Yajie, Rahimian, Fatemeh, Canoy, Dexter, Tran, Jenny, Pinho Gomes, Ana Catarina, Payberah, Amir H. +5 more·2020·Journal of Biomedical Informatics

Abstract

Despite the recent developments in deep learning models, their applications in clinical decision-support systems have been very limited. Recent digitalisation of health records, however, has provided a great platform for the assessment of the usability of such techniques in healthcare. As a result, the field is starting to see a growing number of research papers that employ deep learning on electronic health records (EHR) for personalised prediction of risks and health trajectories. While this can be a promising trend, vast paper-to-paper variability (from data sources and models they use to the clinical questions they attempt to answer) have hampered the field's ability to simply compare and contrast such models for a given application of interest. Thus, in this paper, we aim to provide a comparative review of the key deep learning architectures that have been applied to EHR data. Furthermore, we also aim to: (1) introduce and use one of the world's largest and most complex linked primary care EHR datasets (i.e., Clinical Practice Research Datalink, or CPRD) as a new asset for training such data-hungry models; (2) provide a guideline for working with EHR data for deep learning; (3) share some of the best practices for assessing the "goodness" of deep-learning models in clinical risk prediction; (4) and propose future research ideas for making deep learning models more suitable for the EHR data. Our results highlight the difficulties of working with highly imbalanced datasets, and show that sequential deep learning architectures such as RNN may be more suitable to deal with the temporal nature of EHR.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1016/j.jbi.2019.103337

Cites

84 of 84 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • It is usually a difficult task for traditional sequence labeling algorithms to extract relevant information from EHR data.

    “It is usually a difficult task for the traditional sequence labeling algorithms to extract relevant information from EHR data [43]”
  • Some recent comparative studies provided surprising observations in terms of how to choose the best representation learning methods.

    “Some recent comparative studies such as Ayala Solares et al. [137], Sadati et al. [138], and Min et al. [139] provided some surprising observations in terms of how to choose the best representation learning methods.”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 13

  1. S. Ebadollahi, J. Sun, D. Gotz, J. Hu, D. Sow, C. Neti, Predicting patient's trajectory of physiological data using temporal trends in similar patients: a system for near- term prognostics, in: AMIA Annual Symposium Proceedings, Vol. 2010, American Medical Informatics Association, 2010, p. 192.
  2. R.J. Carroll, A.E. Eyler, J.C. Denny, Naïve electronic health record phenotype identification for rheumatoid arthritis, in: AMIA Annual Symposium Proceedings, Vol. 2011, American Medical Informatics Association, 2011, p. 189.
  3. T. Yue, H. Wang, Deep Learning for Genomics: A Concise Overview, arXiv.
  4. E. Choi, M.T. Bahadori, J. Sun, J. Kulas, A. Schuetz, W. Stewart, RETAIN: An Interpretable Predictive Model for Healthcare using Reverse Time Attention Mechanism, in: Advances in Neural Information Processing Systems, 2016, pp. 3504-3512.
  5. E. Choi, M.T. Bahadori, A. Schuetz, W.F. Stewart, J. Sun, Doctor AI: Predicting Clinical Events via Recurrent Neural Networks, in: Proceedings of the 1st Machine Learning for Healthcare Conference, Vol. 56 of Proceedings of Machine Learning Research, PMLR, 2016, pp. 301-318. <http://proceedings.mlr.press/v56/Choi16. html>.
  6. B.C. Kwon, M.-J. Choi, J.T. Kim, E. Choi, Y.B. Kim, S. Kwon, J. Sun, J. Choo, RetainVis: visual analytics with interpretable and interactive recurrent neural networks on electronic medical records, IEEE Trans. Visual. Comput. Graph.
  7. L. Buitinck, G. Louppe, M. Blondel, F. Pedregosa, A. Mueller, O. Grisel, V. Niculae, P. Prettenhofer, A. Gramfort, J. Grobler, R. Layton, J. VanderPlas, A. Joly, B. Holt, G. Varoquaux, API design for machine learning software: experiences from the scikit-learn project, in: ECML PKDD Workshop: Languages for Data Mining and Machine Learning, 2013, pp. 108-122.
  8. M. Pelikan, D.E. Goldberg, E. Cantú-Paz, BOA: The Bayesian Optimization Algorithm, in: Proceedings of the 1st Annual Conference on Genetic and Evolutionary Computation -Volume 1, GECCO'99, Morgan Kaufmann Publishers Inc., 1999, pp. 525-532.
  9. J. Snoek, O. Rippel, K. Swersky, R. Kiros, N. Satish, N. Sundaram, M.M.A. Patwary, Prabhat, R.P. Adams, Scalable Bayesian Optimization Using Deep Neural Networks, in: ICML, 2015.
  10. J.T. Springenberg, A. Klein, S. Falkner, F. Hutter, Bayesian Optimization with Robust Bayesian Neural Networks, in: NIPS, 2016.
  11. Y. Gal, Uncertainty in Deep Learning, University of Cambridge.
  12. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, I. Polosukhin, Attention Is All You Need, in: NIPS, 2017.
  13. S. Lee, Natural language generation for electronic health records, Comput. Res. Reposit., abs/1806.01353. · arXiv

Cited by

Sorted by citations

Authors
Ayala Solares, J. R., Diletta Raimondi, F. E., Zhu, Y., Rahimian, F., Canoy, D., Tran, J., Pinho Gomes, A. C., Payberah, A. H., Zottoli, M., Nazarzadeh, M., Conrad, N., Rahimi, K., & Salimi-Khorshidi, G.
Affiliations
The George Institute for Global Health (GB)
Corresponding author
Zhu, Yajie
Venue
Journal of Biomedical Informatics
Publisher
Elsevier BV
Published
2020-01
Volume
101
Pages
103337
ISSN
1532-0464 (ISSN-L)
OpenAlex
W3000238064
Cited by
247
References
80
Citations by year
2026: 23 · 2025: 36 · 2024: 37 · 2023: 72 · 2022: 37 · 2021: 29 · 2020: 13
FWCI
14.83
Type
journal article
Language
en
Open access
Locations
  • Journal of Biomedical Informatics (publishedVersion)
  • PubMed (publishedVersion)
  • Newcastle University ePrints (Newcastle Univesity)
Funders
  • NIHR Oxford Biomedical Research Centre
  • UK Research and Innovation
  • Oxford Martin School, University of Oxford
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
electronic health records, deep learning architectures, clinical risk prediction, Clinical Practice Research Datalink, class imbalance, recurrent neural networks, temporal data, clinical decision support
MeSH terms
Deep Learning, Forecasting, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review