GMRLNet: A Graph-Based Manifold Regularization Learning Framework for Placental Insufficiency Diagnosis on Incomplete Multimodal Ultrasound Data
The DOI matches its Crossref record; 32 of the 35 references it cites were found in a registry or an index.
Abstract
Multimodal analysis of placental ultrasound (US) and microflow imaging (MFI) could greatly aid in the early diagnosis and interventional treatment of placental insufficiency (PI), ensuring a normal pregnancy. Existing multimodal analysis methods have weaknesses in multimodal feature representation and modal knowledge definitions and fail on incomplete datasets with unpaired multimodal samples. To address these challenges and efficiently leverage the incomplete multimodal dataset for accurate PI diagnosis, we propose a novel graph-based manifold regularization learning (MRL) framework named GMRLNet. It takes US and MFI images as input and exploits their modality-shared and modality-specific information for optimal multimodal feature representation. Specifically, a graph convolutional-based shared and specific transfer network (GSSTN) is designed to explore intra-modal feature associations, thus decoupling each modal input into interpretable shared and specific spaces. For unimodal knowledge definitions, graph-based manifold knowledge is introduced to describe the sample-level feature representation, local inter-sample relations, and global data distribution of each modality. Then, an MRL paradigm is designed for inter-modal manifold knowledge transfer to obtain effective cross-modal feature representations. Furthermore, MRL transfers the knowledge between both paired and unpaired data for robust learning on incomplete datasets. Experiments were conducted on two clinical datasets to validate the PI classification performance and generalization of GMRLNet. State-of-the-art comparisons show the higher accuracy of GMRLNet on incomplete datasets. Our method achieves 0.913 AUC and 0.904 balanced accuracy (bACC) for paired US and MFI images, as well as 0.906 AUC and 0.888 bACC for unimodal US images, illustrating its application potential in PI CAD systems.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Semi-supervised classification with graph convolutional networks
Kipf, T. N., & Welling, M. (2016) · arXiv · doi:10.48550/arxiv.1609.02907 · Cited by 24,853
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602
Image-to-Image Translation with Conditional Adversarial Networks
Isola, P., Zhu, J.-Y., Zhou, T., & Efros, A. A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.632 · Cited by 22,930 · cited 1×
“Recently, generative adversarial network (GAN)-based models have been proposed to complement incomplete datasets for multimodal learning [24], [25].”
BackgroundFaceNet: A unified embedding for face recognition and clustering
Schroff, F., Kalenichenko, D., & Philbin, J. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298682 · Cited by 15,267 · cited 1×
“Equation (6) is modified from the triplet loss [27].”
BackgroundDistilling the Knowledge in a Neural Network
Hinton, G., Vinyals, O., & Dean, J. (2015) · arXiv · doi:10.48550/arxiv.1503.02531 · Cited by 14,204
UMAP: Uniform Manifold Approximation and Projection
McInnes, L., Healy, J., Saul, N., & Großberger, L. (2018) · Journal of Open Source Software · doi:10.21105/joss.00861 · Cited by 9,826 · cited 1×
“Furthermore, we employed UMAP [32] to visualize the sample distribution in the feature spaces.”
MethodSterne, J. A. C., White, I. R., Carlin, J. B., Spratt, M., Royston, P., Kenward, M. G., Wood, A. M., & Carpenter, J. R. (2009) · BMJ · doi:10.1136/bmj.b2393 · Cited by 7,299
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · cited 2×
“Intermediate fusion fuses the highly abstract features encoded from each modal input by the individual encoders, which is more powerful in exploring the association among modalities on the representative features [15].”
BackgroundMethodMultimodal Machine Learning: A Survey and Taxonomy
Baltrusaitis, T., Ahuja, C., & Morency, L.-P. (2019) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2798607 · Cited by 5,018
Convolutional Two-Stream Network Fusion for Video Action Recognition
Feichtenhofer, C., Pinz, A., & Zisserman, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.213 · Cited by 2,684
Deep Canonical Correlation Analysis
Andrew, G., Arora, R., Bilmes, J., & Livescu, K. (2013) · International Conference on Machine Learning · Cited by 2,401
Zagoruyko, S., & Komodakis, N. (2016) · arXiv · doi:10.48550/arxiv.1612.03928 · Cited by 2,027
Relational Knowledge Distillation
Park, W., Kim, D., Lu, Y., & Cho, M. (2019) · 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2019.00409 · Cited by 1,681
Multimodal classification of Alzheimer's disease and mild cognitive impairment
Zhang, D., Wang, Y., Zhou, L., Yuan, H., & Shen, D. (2011) · NeuroImage · doi:10.1016/j.neuroimage.2011.01.008 · Cited by 1,314
Self-citationWeisfeiler and Leman Go Neural: Higher-Order Graph Neural Networks
Morris, C., Ritzert, M., Fey, M., Hamilton, W. L., Lenssen, J. E., Rattan, G., & Grohe, M. (2019) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v33i01.33014602 · Cited by 1,210
Multimodal deep learning for biomedical data fusion: a review
Stahlschmidt, S. R., Ulfenborg, B., & Synnergren, J. (2022) · Briefings in Bioinformatics · doi:10.1093/bib/bbab569 · Cited by 766 · cited 4×
“…last layer feature of each modality, joint-space-based DCCA [18], DCCAE [19], and contrastive-learning-based MICLe [21]; (III) Late fusion: the LF [8] and TMC [17] are introduced in our comparison, where the former fuses the modal decisions based on learnable weight and the latter is based on DST.”
BackgroundMethodInfluentialBig Self-Supervised Models Advance Medical Image Classification
Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., Loh, A., Karthikesalingam, A., Kornblith, S., Chen, T., et al. (2021) · 2021 IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv48922.2021.00346 · Cited by 685
Convolutional Networks with Dense Connectivity
Huang, G., Liu, Z., Pleiss, G., Maaten, L. V. D., & Weinberger, K. Q. (2022) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2918284 · Cited by 591
Trusted Multi-View Classification With Dynamic Evidential Fusion
Han, Z., Zhang, C., Fu, H., & Zhou, J. T. (2023) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2022.3171983 · Cited by 536
Multiple Classifier Systems for the Classification of Audio-Visual Emotional States
Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S. J., Kächele, M., Schmidt, M., Neumann, H., Palm, G., et al. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_47 · Cited by 155
Probabilistic Knowledge Transfer for Lightweight Deep Representation Learning
Passalis, N., Tzelepi, M., & Tefas, A. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2020.2995884 · Cited by 134
Herraìz, I., Simon, E. D. S., Gómez‐Arriaga, P. I., Martínez-Moratalla, J. M., García-Burguillo, A., Jiménez, E., & Galindo, A. (2015) · International Journal of Molecular Sciences · doi:10.3390/ijms160819009 · Cited by 130
Placental structural abnormalities in gestational diabetes and when they develop: A scoping review
Ehlers, E., Talton, O. O., Schust, D. J., & Schulz, L. C. (2021) · Placenta · doi:10.1016/j.placenta.2021.04.005 · Cited by 113
3D cGAN based cross-modality MR image synthesis for brain tumor segmentation
Yu, B., Zhou, L., Wang, L., Fripp, J., & Bourgeat, P. (2018) · 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI 2018) · doi:10.1109/isbi.2018.8363653 · Cited by 109 · cited 2×
“Their outperforming capability has been demonstrated by studies concerning medical image segmentation [4] and classification [5].”
BackgroundMethodPan, Y., Liu, M., Lian, C., Xia, Y., & Shen, D. (2020) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2020.2983085 · Cited by 99
Feature Fusion for Multimodal Emotion Recognition Based on Deep Canonical Correlation Analysis
Zhang, K., Li, Y., Wang, J., Wang, Z., & Li, X. (2021) · IEEE Signal Processing Letters · doi:10.1109/lsp.2021.3112314 · Cited by 60
Liu, Y., Yue, L., Xiao, S., Yang, W., Shen, D., & Liu, M. (2022) · Medical Image Analysis · doi:10.1016/j.media.2021.102266 · Cited by 58
Hunt, K. F., Kennedy, S., & Vatish, M. (2016) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/j.ejogrb.2016.08.029 · Cited by 47
Acoustic Feature Learning via Deep Variational Canonical Correlation Analysis
Tang, Q., Wang, W., & Livescu, K. (2017) · Interspeech (USB) · doi:10.21437/interspeech.2017-1581 · Cited by 32
Imaging and assessment of placental function
Moran, M., & McAuliffe, F. M. (2011) · Journal of Clinical Ultrasound · doi:10.1002/jcu.20846 · Cited by 31
Shan, Q., Wei, X., & Cai, Z. (2022) · Journal of Physics Conference Series · doi:10.1088/1742-6596/2224/1/012024 · Cited by 5
Chen, T., Wang, Z., Li, L., Li, D., Li, Z., Chang, X., Bian, P., Liang, G., Wang, Q., & Xie, T. (2025) · Schloss Dagstuhl – Leibniz-Zentrum für Informatik · doi:10.4230/lipics.ecoop.2025.6
Research works printed but not linked: 1
- Zhou, High-order regularization on graphs, 1
Cited by
A systematic review of intermediate fusion in multimodal deep learning for biomedical applications
Guarrasi, V., Aksu, F., Caruso, C. M., Di Feola, F., Rofena, A., Ruffini, F., & Soda, P. (2025) · Image and Vision Computing · doi:10.1016/j.imavis.2025.105509 · Cited by 95
Li, X., Li, L., Li, M., Yan, P., Feng, T., Luo, H., Zhao, Y. L., & Shen, Y. (2025) · Medical Image Analysis · doi:10.1016/j.media.2025.103819 · Cited by 69
Yu, L. J., Liu, X., Zhou, J., Li, F., Wang, Y., & Liu, Q. (2025) · Frontiers in Pharmacology · doi:10.3389/fphar.2025.1541509 · Cited by 67
Self-citationFine-Grained Self-Supervised Learning with Jigsaw puzzles for medical image classification
Park, W., & Ryu, J. (2024) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2024.108460 · Cited by 28
A Systematic Review of Intermediate Fusion in Multimodal Deep Learning for Biomedical ApplicationsPreprint
Guarrasi, V., Aksu, F., Caruso, C. M., Feola, F. D., Rofena, A., Ruffini, F., & Soda, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4952813 · Cited by 21
Metadata
- Authors
- Jiao, J., Sun, H., Huang, Y., Xia, M., Qiao, M., Ren, Y., Wang, Y., & Guo, Y.
- Affiliations
- Fudan University (CN)
- Shanghai Medical College of Fudan University (CN)
- Venue
- IEEE Transactions on Medical Imaging
- Published
- 2023-11
- Volume
- 42
- Issue
- 11
- Pages
- 3205-3218
- ISSN
- 0278-0062 (ISSN-L)
- 1558-254X
- OpenAlex
- W4377235393
- Semantic Scholar
- 662e76b3d72dd747cece725aeb16ed1b5ab16eae
- Cited by
- 15
- References
- 35
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 3 · 2023: 1
- FWCI
- 4.68
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Transactions on Medical Imaging (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Natural Science Foundation of China (61871135, 81830058)
- Science and Technology Commission of Shanghai Municipality (22ZR1404800)
- Topics
- Health Sciences › Medicine › Obstetrics and Gynecology › Pregnancy and preeclampsia studies
- Health Sciences › Health Professions › Health Information Management › Artificial Intelligence in Healthcare
- Health Sciences › Medicine › Radiology, Nuclear Medicine and Imaging › Radiomics and Machine Learning in Medical Imaging
- Keywords
- placental insufficiency, multimodal ultrasound, graph convolutional networks, unpaired data, computer-aided diagnosis
- MeSH terms
- Female, Humans, Placenta, Placental Insufficiency, Pregnancy, Ultrasonography
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.