MildInt: Deep Learning-Based Multimodal Longitudinal Data Integration Framework
Open accessThe DOI matches its Crossref record; 5 of the 5 references it cites were found in a registry or an index.
Abstract
As large amounts of heterogeneous biomedical data become available, numerous methods for integrating such datasets have been developed to extract complementary knowledge from multiple domains of sources. Recently, a deep learning approach has been proposed with promising results in a variety of research areas. However, applying such deep learning approaches to heterogeneous dataset involves the strenuous design of deep architectures. Thus, in this paper, we present a deep learning-based python package that can efficiently perform data integration. The python package we propose is called MildInt (Deep learning-based Multimodal longitudinal data integration framework), which provides a pre-trained deep learning architecture for classification tasks. MildInt contains two learning phases: 1) learning feature representation from each modality of the data and 2) training a classifier for the final decision. Adopting a deep architecture in the first phase leads to learning more task-relevant feature representation than a linear model. In the second phase, a linear classifier is used for detecting and investigating biomarkers from multimodal data. By combining the linear model and the deep learning model, higher accuracy and better interpretability can be achieved at the same time. We validate the performance of our package using simulation data and real data. For the real data, as a pilot study, we used clinical and multimodal neuroimaging datasets in Alzheimer’s Disease to predict the disease progression. In such experiments, we show that MildInt is capable of integrating multiple forms of numerical data, including time series data, for extracting complementary features from the multimodal dataset.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
LeCun, Y., Bengio, Y., & Hinton, G. (2015) · Nature · doi:10.1038/nature14539 · Cited by 84,126
Using recurrent neural network models for early detection of heart failure onset
Choi, E., Schuetz, A., Stewart, W. F., & Sun, J. (2017) · Journal of the American Medical Informatics Association · doi:10.1093/jamia/ocw112 · Cited by 928 · cited 1×
“In addition, in the field of translational research, deep learning-based predictive models have shown comparable results (Chaudhary et al., 2017; Choi et al., 2017; Lu et al., 2018; Lee et al., 2019).”
ResultPredicting alzheimer’s disease progression using multi-modal deep learning approach
Lee, G., Nho, K., Kang, B., Sohn, K.-A., Kim, D., Initiative, F. A. D. N., Weiner, M. W., Aisen, P., Petersen, R., Jack, C. R., et al. (2019) · Scientific Reports · doi:10.1038/s41598-018-37769-z · Cited by 406
Young, J., Modat, M., Cardoso, M. J., Mendelson, A., Cash, D., & Ourselin, S. (2013) · NeuroImage Clinical · doi:10.1016/j.nicl.2013.05.004 · Cited by 249
Huang, M., Yang, W., Feng, Q., Chen, W., Initiative, T. A. D. N., Weiner, M. W., Aisen, P., Petersen, R., Jack, C. R., Jagust, W., et al. (2017) · Scientific Reports · doi:10.1038/srep39880 · Cited by 55 · cited 1×
“52 Hierarchical ensemble (Huang et al., 2017) 70/61 MRI 0.”
Cited by
Harnessing multimodal data integration to advance precision oncology
Boehm, K. M., Khosravi, P., Vanguri, R., Gao, J., & Shah, S. P. (2022) · Nature Reviews Cancer · doi:10.1038/s41568-021-00408-3 · Cited by 662
Deep learning-based approaches for multi-omics data integration and analysis
Ballard, J. L., Wang, Z., Li, W., Shen, L., & Long, Q. (2024) · BioData Mining · doi:10.1186/s13040-024-00391-z · Cited by 258
Generative artificial intelligence empowers digital twins in drug discovery and clinical trials
Bordukova, M., Makarov, N., Rodriguez-Esteban, R., Schmich, F., & Menden, M. P. (2024) · Expert Opinion on Drug Discovery · doi:10.1080/17460441.2023.2273839 · Cited by 235
“Besides that, NNs have been shown to be well suited for complex, multimodal, multidimensional and longitudinal data [39] and have thus spearheaded developments in the field of DTs.”
BackgroundDeep learning for Alzheimer's disease diagnosis: A survey
Khojaste-Sarakhsi, M., Haghighi, S. S., Ghomi, S. M. T. F., & Marchiori, E. (2022) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102332 · Cited by 194
Metadata
- Authors
- Lee, G., Kang, B., Nho, K., Sohn, K.-A., & Kim, D.
- Venue
- Frontiers in Genetics
- Publisher
- Frontiers Media SA
- Published
- 2019-06-28
- Volume
- 10
- Pages
- 617-617
- ISSN
- 1664-8021 (ISSN-L)
- OpenAlex
- W2949231527
- Cited by
- 62
- References
- 5
- FWCI
- 2.99
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.