Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
The outbreak of coronavirus disease (COVID-19) has been a nightmare to citizens, hospitals, healthcare practitioners, and the economy in 2020. The overwhelming number of confirmed cases and suspected cases put forward an unprecedented challenge to the hospital's capacity of management and medical resource distribution. To reduce the possibility of cross-infection and attend a patient according to his severity level, expertly diagnosis and sophisticated medical examinations are often required but hard to fulfil during a pandemic. To facilitate the assessment of a patient's severity, this paper proposes a multi-modality feature learning and fusion model for end-to-end covid patient severity prediction using the blood test supported electronic medical record (EMR) and chest computerized tomography (CT) scan images. To evaluate a patient's severity by the co-occurrence of salient clinical features, the High-order Factorization Network (HoFN) is proposed to learn the impact of a set of clinical features without tedious feature engineering. On the other hand, an attention-based deep convolutional neural network (CNN) using pre-trained parameters are used to process the lung CT images. Finally, to achieve cohesion of cross-modality representation, we design a loss function to shift deep features of both-modality into the same feature space which improves the model's performance and robustness when one modality is absent. Experimental results demonstrate that the proposed multi-modality feature learning and fusion model achieves high performance in an authentic scenario.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation
Kamnitsas, K., Ledig, C., Newcombe, V. F. J., Simpson, J. P., Kane, A. D., Menon, D. K., Rueckert, D., & Glocker, B. (2017) · Medical Image Analysis · doi:10.1016/j.media.2016.10.004 · Cited by 3,498
Learning Important Features Through Propagating Activation Differences
Shrikumar, A., Greenside, P., & Kundaje, A. (2017) · arXiv · doi:10.48550/arxiv.1704.02685 · Cited by 2,761
Babyak, M. A. (2004) · Psychosomatic Medicine · doi:10.1097/01.psy.0000127692.23278.a9 · Cited by 2,061
Liang, W., Liang, H., Ou, L., Chen, B., Chen, A., Li, C., Li, Y., Guan, W., Sang, L., Lu, J., et al. (2020) · JAMA Internal Medicine · doi:10.1001/jamainternmed.2020.2033 · Cited by 1,387 · cited 1×
“In [23], by employing the Least Absolute Shrinkage and Selection Operation (LASSO) regression, significant medical features are selected to build a logistic regression model.”
MethodMultimodal Data Fusion: An Overview of Methods, Challenges, and Prospects
Lahat, D., Adali, T., & Jutten, C. (2015) · Proceedings of the IEEE · doi:10.1109/jproc.2015.2460697 · Cited by 1,296 · cited 1×
“The appropriate fusion of available information from multi-modality data is still an important way to increase the prediction accuracy of the COVID-19 patient severity, even if some data missing issues (e.g. modality missing) may be encountered [9], [10].”
BackgroundSanity Checks for Saliency Maps
Adebayo, J., Gilmer, J., Muelly, M., Goodfellow, I., Hardt, M., & Kim, B. (2018) · arXiv · doi:10.48550/arxiv.1810.03292 · Cited by 1,294
A survey on deep learning for multimodal data fusion
Gao, J., Li, P., Chen, Z., & Zhang, J. (2020) · Neural Computation · doi:10.1162/neco_a_01273 · Cited by 821 · cited 2×
“Multi-modality data fusion and classification aim to integrate the data of different distributions, sources, and types into a global space in which both inter-modality and cross-modality can be represented in a unified manner [11].”
BackgroundFully convolutional networks for multi-modality isointense infant brain image segmentation
Nie, D., Wang, L., Gao, Y., & Shen, D. (2016) · 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493515 · Cited by 304 · cited 1×
“Nie et al. [38] and Chen et al. [39] follow a similar approach and propose to increase the model’s performance using different types of visual data from the same region.”
BackgroundGeorgiadou, S. P., Sipsas, N. V., Marom, E. M., & Kontoyiannis, D. P. (2011) · Clinical Infectious Diseases · doi:10.1093/cid/cir122 · Cited by 292
Cited by
Li, Y., El Habib Daho, M., Conze, P.-H., Zeghlache, R., Le Boité, H., Tadayoni, R., Cochener, B., Lamard, M., & Quellec, G. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108635 · Cited by 274
“…out of 114 reviewed papers, 27 incorporate non-image contextual information in various forms, often structured demographic and clinical metadata extracted from electronic health records (Huang et al., 2020; Zhou et al., 2021a; Yan et al., 2021; Venugopalan et al., 2021; Prabhu et al., 2022).”
BackgroundReview of multimodal machine learning approaches in healthcare
Krones, F., Marikkar, U., Parsons, G., Szmul, A., & Mahdi, A. (2025) · Information Fusion · doi:10.1016/j.inffus.2024.102690 · Cited by 235
Pei, X., Zuo, K., Li, Y., & Pang, Z. (2023) · International Journal of Computational Intelligence Systems · doi:10.1007/s44196-023-00225-6 · Cited by 77
“[44] CT, clinical data, lab tests 733 patients Classification of COVID19 severity”
Method
Metadata
- Authors
- Zhou, J., Zhang, X., Zhu, Z., Lan, X., Fu, L., Wang, H., & Wen, H.
- Venue
- IEEE Transactions on Circuits and Systems for Video Technology
- Published
- 2022-05
- Volume
- 32
- Issue
- 5
- Pages
- 2535-2549
- ISSN
- 1051-8215 (ISSN-L)
- 1558-2205
- OpenAlex
- W3134151358
- Cited by
- 38
- References
- 10
- FWCI
- 2.92
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction. https://app.cochat.ai/papers/10.1109/tcsvt.2021.3063952
- The paper
- Zhou, J., Zhang, X., Zhu, Z., Lan, X., Fu, L., Wang, H., & Wen, H. (2022). Cohesive Multi-Modality Feature Learning and Fusion for COVID-19 Patient Severity Prediction. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/tcsvt.2021.3063952
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.