Cited by 792Machine Learning in HealthcareElectronic Health Records SystemsArtificial Intelligence in Healthcare and Education

Opportunities and challenges in developing deep learning models using electronic health records data: a systematic review

Open access
Xiao, Cao, Choi, Edward, Sun, Jimeng·2018·Journal of the American Medical Informatics Association

Abstract

Objective: To conduct a systematic review of deep learning models for electronic health record (EHR) data, and illustrate various deep learning architectures for analyzing different data sources and their target applications. We also highlight ongoing research and identify open challenges in building deep learning models of EHRs. Design/method: We searched PubMed and Google Scholar for papers on deep learning studies using EHR data published between January 1, 2010, and January 31, 2018. We summarize them according to these axes: types of analytics tasks, types of deep learning model architectures, special challenges arising from health data and tasks and their potential solutions, as well as evaluation strategies. Results: We surveyed and analyzed multiple aspects of the 98 articles we found and identified the following analytics tasks: disease detection/classification, sequential prediction of clinical events, concept embedding, data augmentation, and EHR data privacy. We then studied how deep architectures were applied to these tasks. We also discussed some special challenges arising from modeling EHR data and reviewed a few popular approaches. Finally, we summarized how performance evaluations were conducted for each task. Discussion: Despite the early success in using deep learning for health analytics applications, there still exist a number of issues to be addressed. We discuss them in detail including data and label availability, the interpretability and transparency of the model, and ease of deployment.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/jamia/ocy068

Cites

92 of 92 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 34

  1. [11]Baytas IM, Xiao C, Zhang X, Wang F, Jain AK, Zhou J. Patient subtyping via time-aware LSTM networks. In: proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. Halifax, NS, Canada: ACM; 2017: 65–74.
  2. [12]Cheng Y, Wang F, Zhang P, Hu J. Risk prediction with electronic health records: a deep learning approach. In: proceedings of the 2016 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, Miami, Florida, USA; 2016: 432– 40.
  3. [14]Che C, Xiao C, Liang J, Jin B, Zho J, Wang F. An RNN architecture with dynamic temporal matching for personalized predictions of Parkinson’s disease. In: proceedings of the 2017 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics, Houston, Texas, USA; 2017: 198–206.
  4. [27]Bajor JM, Lasko TA. Predicting medications from diagnostic codes with recurrent neural networks. 2016 https://openreview.net/pdf? id=rJEgeXFex
  5. [28]Zhang Y, Chen R, Tang J, Stewart WF, Sun J. LEAP: learning to prescribe effective and safe treatment combinations for multimorbidity. In: proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY: ACM; 2017: 1315– 24.
  6. [29]Choi E, Bahadori MT, Searles E, et al Multi-layer representation learning for medical concepts. In: proceedings of the 22Nd ACM SIGKDD international conference on knowledge discovery and data Mining. New York, NY: ACM; 2016: 1495–1504.
  7. [33]Che Z, Cheng Y, Zhai S, Sun Z, Liu Y. Boosting deep learning risk prediction with generative adversarial networks for electronic health records. In: 2017 IEEE International Conference on Data Mining (ICDM). New Orleans, LA, USA: IEEE; 2017: 787–92.
  8. [38]Tompson JJ, Jain A, LeCun Y, Bregler C. Joint training of a convolutional network and a graphical model for human pose estimation In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ, eds. Advances in Neural Information Processing Systems 27. Curran Associates, Inc; 2014: 1799– 1807.
  9. [39]Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ, eds. Advances in Neural Information Processing Systems 27. Curran Associates, Inc; 2014: 3104– 112.
  10. [45]Ma F, Chitta R, Zhou J, You Q, Sun T, Gao J. Dipole: diagnosis prediction in healthcare via attention-based bidirectional recurrent neural networks. In: proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York, NY: ACM; 2017: 1903–11.
  11. [50]Veličković P, Karazija L, Lane ND, et al Cross-modal Recurrent Models for Human Weight Objective Prediction from Multimodal Time-series Data. arXiv preprint arXiv: 1709 08073. 2017 https://arxiv.org/abs/1709.08073 · arXiv
  12. [51]Thodoroff P, Pineau J, Lim A. Learning robust features using deep learning for automatic seizure detection. In: Machine Learning for Healthcare Conference. Children's Hospital LA, Los Angeles, CA, USA: PMLR; 2016: 178–190.
  13. [56]Esteban C, Staeck O, Baier S, Yang Y, Tresp V. Predicting clinical events by combining static and dynamic information using recurrent neural networks. In: 2016 IEEE International Conference on Healthcare Informatics (ICHI). Chicago, IL, USA: IEEE; 2016: 93– 101.
  14. [60]Yang Y, Fasching PA, Tresp V. Modeling progression free survival in breast cancer with tensorized recurrent neural networks and accelerated failure time models. In: Machine Learning for Healthcare Conference. Boston, Massachusetts: PMLR; 2017: 164– 176.
  15. [62]Razavian N, Marcus J, Sontag D. Multi-task prediction of disease onsets from longitudinal laboratory tests. In: Machine Learning for Healthcare Conference. Los Angeles, CA, USA: PMLR; 2016: 73–100.
  16. [64]Che C, Xiao C, Liang J, Jin B, Zho J, Wang F. An RNN architecture with dynamic temporal matching for personalized predictions of Parkinson’s disease. In: proceedings of the 2017 SIAM International Conference on Data Mining. Society for Industrial and Applied Mathematics; 2017. pp. 198–206.
  17. [66]Jia Y, Zhou C, Motani M. Spatio-temporal autoencoder for feature learning in patient data with missing observations. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). IEEE; 2017: 886– 90.
  18. [67]Lipton ZC, Kale DC, Wetzel R. Modeling missing data in clinical time series with rnns. Machine Learning for Healthcare. 2016 http://www.jmlr.org/proceedings/papers/v56/Lipton16.pdf
  19. [74]Zhu Z, Yin C, Qian B, Cheng Y, Wei J, Wang F. Measuring patient similarities via a deep architecture with medical concept embedding. In: 2016 IEEE 16th International Conference on Data Mining (ICDM). Barcelona, Spain: IEEE; 2016: 749–758.
  20. [78]Suo Q, Xue H, Gao J, Zhang A. Risk factor analysis based on deep learning models. In: proceedings of the 7th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics. Seattle, WA, USA: ACM; 2016: 394–403.
  21. [79]Yuan Y, Xun G, Jia K, Zhang A. A multi-view deep learning method for epileptic seizure detection using short-time Fourier transform. In: proceedings of the 8th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics. ACM; 2017: 213–22.
  22. [82]Che Z, Kale D, Li W, Bahadori MT, Liu Y. Deep computational phenotyping. In: proceedings of the 21th ACM. dl.acm.org; 2015 http://dl.acm.org/citation.cfm? id=2783365
  23. [86]Ulloa Cerna AE, Wehner G, Hartzel DN, Haggerty C, Fornwalt B. Abstract 16708: data driven phenotyping of patients with heart failure using a deep-learning cluster representation of echocardiographic and electronic health record data Circulation. 2017; 136: A16708.
  24. [88]Yuan Y, Xun G, Suo Q, Jia K, Zhang A. Wave2vec: learning deep representations for biosignals. In: 2017 IEEE International Conference on Data Mining (ICDM). IEEE; 2017: 1159–64.
  25. [92]Liang Z, Zhang G, Huang JX, Hu QV. Deep learning for healthcare decision making with EMRs. In: 2014 IEEE International Conference on Bioinformatics and Biomedicine (BIBM): Belfast, UK: IEEE; 2014: 556–9.
  26. [97]Prakash A, Zhao S, Hasan SA, et al Condensed Memory Networks for Clinical Diagnostic Inferencing. AAAI; 2017: 3274–80.
  27. [102]Yan B, Wang Y, Li Y, Gong Y, Guan L, Yu S. An EEG signal classification method based on sparse auto-encoders and support vector machine. In: 2016 IEEE/CIC International Conference on Communications in China (ICCC). Chengdu, China: IEEE; 2016: 1–6.
  28. [105]Goodfellow I, Pouget-Abadie J, Mirza M, et al Generative adversarial nets In: Ghahramani Z, Welling M, Cortes C, Lawrence ND, Weinberger KQ, eds. Advances in Neural Information Processing Systems 27. Curran Associates, Inc, Montreal, Canada; 2014: 2672– 2680.
  29. [106]Choi E, Bahadori MT, Schuetz A, Stewart WF, Sun J. Doctor AI: predicting clinical events via recurrent neural networks. In: Machine Learning for Healthcare Conference. 2016: 301–18. PMC5341604 28286600
  30. [107]Jagannatha A, Yu H. Structured prediction models for RNN based sequence labeling in clinical text. In: proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Stroudsburg, PA: Association for Computational Linguistics; 2016: 856–65. PMC5167535 28004040
  31. [110]Henao R, Lu JT, Lucas JE, Ferranti J, Carin L. Electronic health record analysis via deep poisson factor models. J Mach Learn Res. 2016; 17: 1– 32.
  32. [111]Dubois S, Romano N, Jung K, Shah N, Kale DC. The Effectiveness of Transfer Learning in Electronic Health Records Data. 2017 https://openreview.net/forum? id=B1_E8xrKe
  33. [118]Johansson F, Shalit U, Sontag D. Learning Representations for Counterfactual Inference. International Conference on Machine Learning. 2016: 3020–9.
  34. [121]Avati A, Jung K, Harman S, Downing L, Ng A, Shah NH. Improving palliative care with deep learning. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Kansas City, MO, USA: IEEE; 2017: 311–6.

Cited by

Sorted by citations

Authors
Xiao, C., Choi, E., & Sun, J.
Affiliations
  • IBM Research - Cambridge (United States) (US)
  • Georgia Institute of Technology (US)
Corresponding author
Sun, Jimeng
Venue
Journal of the American Medical Informatics Association
Published
2018-06-08
Volume
25
Issue
10
Pages
1419-1428
ISSN
  • 1067-5027 (ISSN-L)
  • 1527-974X
OpenAlex
W2805089815
Cited by
792
References
92
Citations by year
2026: 70 · 2025: 112 · 2024: 112 · 2023: 112 · 2022: 100 · 2021: 137 · 2020: 81 · 2019: 60 · 2018: 8
FWCI
8.36
Type
journal article
Language
en
Open access
Locations
  • Journal of the American Medical Informatics Association (publishedVersion, cc-by-nc)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • National Science Foundation (IIS-#1418511, CCF-#1533768)
  • National Institutes of Health (1R01MD011682-01, R56HL138415)
  • Children’s Healthcare of Atlanta
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
  • Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
  • Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
Keywords
deep learning, electronic health records, deep learning architectures, clinical event prediction, concept embeddings, data augmentation, model interpretability, model deployment
MeSH terms
Data Anonymization, Deep Learning, Disease, Humans, Prognosis, Neural Networks, Computer, Electronic Health Records

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review