The DOI matches its Crossref record; 34 of the 35 references it cites were found in a registry or an index.
Combining structured and unstructured data for predictive models: a deep learning approach
Open accessPublished version of a preprint
Abstract
BACKGROUND: The broad adoption of electronic health records (EHRs) provides great opportunities to conduct health care research and solve various clinical problems in medicine. With recent advances and success, methods based on machine learning and deep learning have become increasingly popular in medical informatics. However, while many research studies utilize temporal structured data on predictive modeling, they typically neglect potentially valuable information in unstructured clinical notes. Integrating heterogeneous data types across EHRs through deep learning techniques may help improve the performance of prediction models. METHODS: In this research, we proposed 2 general-purpose multi-modal neural network architectures to enhance patient representation learning by combining sequential unstructured notes with structured data. The proposed fusion models leverage document embeddings for the representation of long clinical note documents and either convolutional neural network or long short-term memory networks to model the sequential clinical notes and temporal signals, and one-hot encoding for static information representation. The concatenated representation is the final patient representation which is used to make predictions. RESULTS: We evaluate the performance of proposed models on 3 risk prediction tasks (i.e. in-hospital mortality, 30-day hospital readmission, and long length of stay prediction) using derived data from the publicly available Medical Information Mart for Intensive Care III dataset. Our results show that by combining unstructured clinical notes with structured data, the proposed models outperform other models that utilize either unstructured notes or structured data only. CONCLUSIONS: The proposed fusion models learn better patient representation by combining structured and unstructured data. Integrating heterogeneous data types across EHRs helps improve the performance of prediction models and reduce errors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“For evaluation, we use a 0.70-0.15-0.15 train-validation-test split following the previous work [13]. However, we equalized the size of the validation set and the test set to get a better estimation of test performance on the validation set. We split the dataset according to patient ID and admission ID so that there's no overlap between training, validation, and test sets.”
From Multimodal risk prediction with physiological signals, medical images and clinical notes (2024) · cites this paper as [13]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“They hold great promise in deriving meaningful insights from EHR, which can aid in accurately predicting clinical outcomes, such as mortality [13] and readmission [13], [14].”
From Multimodal risk prediction with physiological signals, medical images and clinical notes (2024) · cites this paper as [13]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“, logistic regression and random forest) with scikit-learn [32].”
BackgroundPyTorch: An Imperative Style, High-Performance Deep Learning Library
Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L., et al. (2019) · arXiv · doi:10.48550/arxiv.1912.01703 · Cited by 31,473 · cited 1×
“Deep learning models are implemented using PyTorch [33].”
MethodSantos, A. G., da Rocha, G. O., & de Andrade, J. B. (2019) · Scientific Reports · doi:10.1038/s41598-018-37186-2 · Cited by 8,495
MIMIC-III, a freely accessible critical care database
Johnson, A. E. W., Pollard, T. J., Shen, L., Lehman, L.-W. H., Feng, M., Ghassemi, M., Moody, B., Szolovits, P., Anthony Celi, L., & Mark, R. G. (2016) · Scientific Data · doi:10.1038/sdata.2016.35 · Cited by 8,141
Distributed Representations of Sentences and DocumentsPreprint
Le, Q. V., & Mikolov, T. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1405.4053 · Cited by 7,905 · cited 1×
“Doc2Vec) [30] to learn the embedding of each clinical note.”
BackgroundA unified architecture for natural language processing: deep neural networks with multitask learning
Collobert, R., & Weston, J. (2008) · Proceedings of the 25th international conference on Machine learning - ICML '08 · doi:10.1145/1390156.1390177 · Cited by 5,058
Software Framework for Topic Modelling with Large Corpora
Rek, R. R. U., & Sojka, P. (2010) · doi:10.13140/2.1.2393.1847 · Cited by 3,789
Scalable and accurate deep learning with electronic health records
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · npj Digital Medicine · doi:10.1038/s41746-018-0029-1 · Cited by 2,648
Scalable and accurate deep learning with electronic health records.
Rajkomar, A., Oren, E., Chen, K., Dai, A. M., Hajaj, N., Hardt, M., Liu, P. J., Liu, X., Marcus, J., Sun, M., et al. (2018) · eScholarship (California Digital Library) · Cited by 2,242
Research works printed but not linked: 3
- Henry J, Pylypchuk Y, Searcy T, Patel V. Adoption of electronic health record systems among US non-federal acute care hospitals: 2008–2015. ONC Data Brief. 2016;35:1–9.
- Ghassemi M, Naumann T, Joshi R, Rumshisky A. Topic models for mortality modeling in intensive care units. In: ICML machine learning for clinical data analysis workshop; 2012. p. 1–4.
- Hackbarth G, Reischauer R, Miller M. Report to the congress: promoting greater efficiency in medicare. Washington, DC: MedPAC; 2007.
Cited by
Sorted by citations
Multimodal deep learning for biomedical data fusion: a review
Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab569 · Cited by 766
“[64] proposed a CNN- and a RNN-based fusion model that take temporal signals, sequential clinical notes as well as static demographic and admission data as input to the different branches.”
MethodMultimodal machine learning in precision health: A scoping review
Kline, A., Wang, H., Li, Y., Dennis, S., Hutch, M., Xu, Z., Wang, F., Cheng, F., & Luo, Y. (2022) · npj Digital Medicine · doi:10.1038/s41746-022-00712-8 · Cited by 528
Synthetic data generation for tabular health records: A systematic review
Hernandez, M., Epelde, G., Alberdi, A., Cilla, R., & Rankin, D. (2022) · Neurocomputing · doi:10.1016/j.neucom.2022.04.053 · Cited by 278
“Despite recent trends that combine non-structured data with structured health data [22], the most common approach for developing ML models for healthcare decision making has been through structured data and the structuring of unstructured data through coding or feature extraction algorithms.”
BackgroundTime Series Prediction Using Deep Learning Methods in Healthcare
Morid, M. A., Sheng, O. R. L., & Dunbar, J. (2022) · ACM Transactions on Management Information Systems · doi:10.1145/3531326 · Cited by 280
“Zhang et al. (2020)[77] later enhanced this initial ontological strategy by considering more than one parent for each node and also by providing an ordered set of ancestors for each medical concept.”
BackgroundMethodInfluentialA systematic review of the prediction of hospital length of stay: Towards a unified framework
Stone, K., Reyer Zwiggelaar, A., Jones, P., & Parthaláin, N. M. (2022) · PLOS Digital Health · doi:10.1371/journal.pdig.0000017 · Cited by 274
Metadata
- Authors
- Zhang, D., Yin, C., Zeng, J., Yuan, X., & Zhang, P.
- Affiliations
- Wuhan University of Technology (CN)
- The Ohio State University (US)
- Corresponding author
- Zhang, Ping
- Venue
- BMC Medical Informatics and Decision Making
- Published
- 2020-10-29
- Volume
- 20
- Issue
- 1
- Pages
- 280-280
- ISSN
- 1472-6947 (ISSN-L)
- PMC
- PMC7596962
- OpenAlex
- W3095791468
- Cited by
- 269
- References
- 35
- Citations by year
- 2026: 41 · 2025: 72 · 2024: 52 · 2023: 52 · 2022: 34 · 2021: 18
- FWCI
- 10.13
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- BMC Medical Informatics and Decision Making (publishedVersion, cc-by)
- medRxiv (acceptedVersion, cc-by-nc-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Machine Learning in Healthcare
- Health Sciences › Nursing › Research and Theory › Nursing Diagnosis and Documentation
- Health Sciences › Health Professions › Health Information Management › Electronic Health Records Systems
- Keywords
- electronic health records, unstructured clinical notes, multimodal neural networks, patient representation learning, in-hospital mortality prediction, hospital readmission prediction, length of stay prediction, MIMIC-III, clinical risk prediction
- MeSH terms
- Machine Learning, Deep Learning, Humans, Patient Readmission, Neural Networks, Computer, Electronic Health Records
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
