Multimodal fusion model for classifying placenta ultrasound imaging in pregnancies with hypertension disorders

Abstract

Background A multimodal fusion model was proposed to assist the traditional visual diagnosis in evaluating the placental features of hypertension disorders of pregnancy (HDP).Objective The aim of this study was to analyse and compare the placental features between normal and HDP pregnancies and propose a multimodal fusion deep learning model for differentiating and characterizing the placental features from HDP to normal pregnancy.Methods This observational prospective study included 654 pregnant women, including 75 with HDPs. Grayscale ultrasound images (GSIs) and Microflow images (MFIs) of the placentas were collected from all patients during routine obstetric examinations. On the basis of intelligent extraction and features fusion, after quantities of training and optimization, the classification model named GMNet (the intelligent network based on GSIs and MFIs) was introduced for differentiating the placental features of normal and HDP pregnancies. The distributions of placental features extracted by the deep convolutional neural networks (DCNNs) were visualized by Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP). Metrics including sensitivity, specificity, accuracy, and the area under the curve (AUC) were used to score the model. Finally, placental tissue samples were randomly selected for microscopic analyses to prove the interpretability and effectiveness of the GMNet model.Results Compared with the Normal group in ultrasonic images, the light spots were rougher and the parts with focal cystic or hypoechogenic lesions were increased in the HDP groups. The overall diagnostic performance of the GMNet model depending on the region of interest (ROI) was excellent (AUC: 97%), with a sensitivity of 90.0%, a specificity of 93.5%, and an accuracy of 93.1%. The fusion features of GSIs and MFIs in the placenta showed a higher discriminative power than single-mode features (fusion features vs GSI features vs MFI features, 97.0% vs 91.2% vs 94.8%). Furthermore, according to the microscopic analysis, unevenly distributed villi, increased syncyte nodules and aggregated intervillous cellulose deposition were particularly frequent in the HDP cases.Conclusions The GMNet model could sensitively identify abnormal changes in the placental microstructure in pregnancies with HDP.