Improving the Segmentation Accuracy of Ovarian-Tumor Ultrasound Images Using Image Inpainting
Open accessAbstract
Diagnostic results can be radically influenced by the quality of 2D ovarian-tumor ultrasound images. However, clinically processed 2D ovarian-tumor ultrasound images contain many artificially recognized symbols, such as fingers, crosses, dashed lines, and letters which assist artificial intelligence (AI) in image recognition. These symbols are widely distributed within the lesion’s boundary, which can also affect the useful feature-extraction-utilizing networks and thus decrease the accuracy of lesion classification and segmentation. Image inpainting techniques are used for noise and object elimination from images. To solve this problem, we observed the MMOTU dataset and built a 2D ovarian-tumor ultrasound image inpainting dataset by finely annotating the various symbols in the images. A novel framework called mask-guided generative adversarial network (MGGAN) is presented in this paper for 2D ovarian-tumor ultrasound images to remove various symbols from the images. The MGGAN performs to a high standard in corrupted regions by using an attention mechanism in the generator to pay more attention to valid information and ignore symbol information, making lesion boundaries more realistic. Moreover, fast Fourier convolutions (FFCs) and residual networks are used to increase the global field of perception; thus, our model can be applied to high-resolution ultrasound images. The greatest benefit of this algorithm is that it achieves pixel-level inpainting of distorted regions without clean images. Compared with other models, our model achieveed better results with only one stage in terms of objective and subjective evaluations. Our model obtained the best results for 256 × 256 and 512 × 512 resolutions. At a resolution of 256 × 256, our model achieved 0.9246 for SSIM, 22.66 for FID, and 0.07806 for LPIPS. At a resolution of 512 × 512, our model achieved 0.9208 for SSIM, 25.52 for FID, and 0.08300 for LPIPS. Our method can considerably improve the accuracy of computerized ovarian tumor diagnosis. The segmentation accuracy was improved from 71.51% to 76.06% for the Unet model and from 61.13% to 66.65% for the PSPnet model in clean images.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
One study focused on the automated elimination of artefacts and objects in US images to increase the accuracy of the AI model.
“one study focused on the automated elimination of artefacts and objects in US images to increase the accuracy of the AI model [194]”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Image quality assessment: from error visibility to structural similarity
Wang, Z., Bovik, A. C., Sheikh, H. R., & Simoncelli, E. P. (2004) · IEEE Transactions on Image Processing · doi:10.1109/tip.2003.819861 · Cited by 57,892 · see citing works
Zhao, H., Shi, J., Qi, X., Wang, X., & Jia, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.660 · Cited by 16,438 · see citing works
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
Zhang, R., Isola, P., Efros, A. A., Shechtman, E., & Wang, O. (2018) · 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2018.00068 · Cited by 14,623 · see citing works
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
Johnson, J., Alahi, A., & Fei-Fei, L. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46475-6_43 · Cited by 11,283 · see citing works
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S. (2017) · arXiv · doi:10.48550/arxiv.1706.08500 · Cited by 8,365 · see citing works
Context Encoders: Feature Learning by Inpainting
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., & Efros, A. A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.278 · Cited by 5,691 · see citing works
High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
Wang, T.-C., Liu, M.-Y., Zhu, J.-Y., Tao, A., Kautz, J., & Catanzaro, B. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00917 · Cited by 4,514 · see citing works
Generative Image Inpainting with Contextual Attention
Yu, J., Lin, Z., Yang, J., Shen, X., Lu, X., & Huang, T. S. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00577 · Cited by 2,633 · see citing works
Image Inpainting for Irregular Holes Using Partial Convolutions
Liu, G., Reda, F. A., Shih, K. J., Wang, T.-C., Tao, A., & Catanzaro, B. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-01252-6_6 · Cited by 2,419 · cited 1× · see citing works
“Following [36,39,42], we set η1 = 10, η2 = 10, η3 = 30, and η4 = 100 in training.”
Dataset of breast ultrasound images
Al-Dhabyani, W., Gomaa, M., Khaled, H., & Fahmy, A. (2020) · Data in Brief · doi:10.1016/j.dib.2019.104863 · Cited by 2,405 · see citing works
Globally and locally consistent image completion
Iizuka, S., Simo-Serra, E., & Ishikawa, H. (2017) · ACM Transactions on Graphics · doi:10.1145/3072959.3073659 · Cited by 2,203 · see citing works
Frid-Adar, M., Diamant, I., Klang, E., Amitai, M. M., Goldberger, J., & Greenspan, H. K. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.09.013 · Cited by 1,952 · see citing works
Resolution-robust Large Mask Inpainting with Fourier Convolutions
Suvorov, R. E., Logacheva, E., Mashikhin, A., Remizova, A., Ashukha, A., Silvestrov, A., Kong, N., Goka, H., Park, K., & Lempitsky, V. (2022) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv51458.2022.00323 · Cited by 1,192 · see citing works
A Novel Transfer Learning Based Approach for Pneumonia Detection in Chest X-ray Images
Chouhan, V., Singh, S. K., Khamparia, A., Gupta, D., Tiwari, P., Moreira, C., Damaševičius, R., & de Albuquerque, V. H. C. (2020) · Applied Sciences · doi:10.3390/app10020559 · Cited by 753 · see citing works
Explainable Deep Learning Models in Medical Image Analysis
Singh, A., Sengupta, S., & Lakshminarayanan, V. (2020) · Journal of Imaging · doi:10.3390/jimaging6060052 · Cited by 718 · see citing works
EdgeConnect: Structure Guided Image Inpainting using Edge Prediction
Nazeri, K., Ng, E., Joseph, T., Qureshi, F. Z., & Ebrahimi, M. (2019) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2019.00408 · Cited by 593 · see citing works
Loey, M., Smarandache, F., & M. Khalifa, N. E. (2020) · Symmetry · doi:10.3390/sym12040651 · Cited by 538 · see citing works
Learning Pyramid-Context Encoder Network for High-Quality Image Inpainting
Zeng, Y., Fu, J., Chao, H., & Guo, B. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00158 · Cited by 536 · see citing works
Shift-Net: Image Inpainting via Deep Feature Rearrangement
Yan, Z., Li, X., Li, M., Zuo, W., & Shan, S. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_1 · Cited by 529 · see citing works
Data Augmentation Using Learned Transformations for One-Shot Medical Image Segmentation
Zhao, A. T., Balakrishnan, G., Durand, F., Guttag, J. V., & Dalca, A. V. (2019) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2019.00874 · Cited by 476 · see citing works
Contextual Residual Aggregation for Ultra High-Resolution Image Inpainting
Yi, Z., Tang, Q., Azizi, S., Jang, D., & Xu, Z. W. (2020) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr42600.2020.00753 · Cited by 392 · see citing works
Chi, L., Jiang, B., & Mu, Y. (2020) · neural information processing systems · Cited by 374 · see citing works
Zhao, Y., Li, Y., & Wu, N. (2021) · IEEE Transactions on Geoscience and Remote Sensing · doi:10.1109/tgrs.2020.3042202 · Cited by 277 · see citing works
An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images
Li, H., Weng, J., Shi, Y., Gu, W., Mao, Y., Wang, Y., Liu, W., & Zhang, J. (2018) · Scientific Reports · doi:10.1038/s41598-018-25005-7 · Cited by 172 · see citing works
Christiansen, F., Epstein, E. L., Smedberg, E., Åkerlund, M., Smith, K., & Epstein, E. (2021) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.23530 · Cited by 142 · see citing works
Karimi, D., Zeng, Q., Mathur, P., Avinash, A., Mahdavi, S. S., Spadinger, I., Abolmaesumi, P., & Salcudean, S. E. (2019) · Medical Image Analysis · doi:10.1016/j.media.2019.07.005 · Cited by 127 · see citing works
Automatic Segmentation and Inpainting of Specular Highlights for Endoscopic Imaging
Arnold, M., Ghosh, A., Ameling, S., et al. (2010) · EURASIP Journal on Image and Video Processing · doi:10.1155/2010/814319 · Cited by 121 · see citing works
Zhang, Z., & Han, Y. (2020) · IEEE Access · doi:10.1109/access.2020.2977962 · Cited by 83 · see citing works
Adversarial Inpainting of Medical Image Modalities
Armanious, K., Mecky, Y., Gatidis, S., et al. (2019) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2019.8682677 · Cited by 72 · see citing works
Jin, J., Zhu, H., Zhang, J., Ai, Y., Zhang, J., Teng, Y., Xie, C., & Jin, X. (2021) · Frontiers in Oncology · doi:10.3389/fonc.2020.614201 · Cited by 63 · see citing works
Classification of Carotid Artery Intima Media Thickness Ultrasound Images with Deep Learning
Savaş, S., Topaloğlu, N., Kazcı, Ö., et al. (2019) · Journal of Medical Systems · doi:10.1007/s10916-019-1406-2 · Cited by 57 · see citing works
MGML: Multigranularity Multilevel Feature Ensemble Network for Remote Sensing Scene Classification
Zhao, Q., Lyu, S., Li, Y., Ma, Y., & Chen, L. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3106391 · Cited by 56 · see citing works
Self-citationMedical image inpainting with edge and structure priors
Wang, Q., Chen, Y., Zhang, N., et al. (2021) · Measurement · doi:10.1016/j.measurement.2021.110027 · Cited by 41 · see citing works
Using Guided Self-Attention with Local Information for Polyp Segmentation
Cai, L., Wu, M., Chen, L., et al. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-16440-8_60 · Cited by 29 · see citing works
Self-citationDeep Learning for Ovarian Tumor Classification with Ultrasound Images
Wu, C., Wang, Y., & Wang, F. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00764-5_36 · Cited by 25 · see citing works
Yao, S., Yan, J., Wu, M., Yang, X., Zhang, W., Lu, H. J., & Qian, B. (2020) · Frontiers in Bioengineering and Biotechnology · doi:10.3389/fbioe.2020.00599 · Cited by 25 · see citing works
Self-citationNon-Local Means Inpainting of MS Lesions in Longitudinal Image Processing
Guizard, N., Nakamura, K., Coupé, P., et al. (2015) · Frontiers in Neuroscience · doi:10.3389/fnins.2015.00456 · Cited by 23 · see citing works
Medical Image Inpainting with RBF Interpolation Technique
Alsalamah, M., & Amin, S. A. (2016) · International Journal of Advanced Computer Science and Applications · doi:10.14569/ijacsa.2016.070814 · Cited by 18 · see citing works
Xie, E., Ni, P., Zhang, R., & Li, X. (2022) · Applied Sciences · doi:10.3390/app12126268 · Cited by 11 · see citing works
Analysis of Kidney Ultrasound Images Using Deep Learning and Machine Learning Techniques: A Review
George, M., & Anita, H. B. (2022) · Lecture notes in networks and systems · doi:10.1007/978-981-16-5640-8_15 · Cited by 9 · see citing works
Fuzzy image inpainting aimed to medical images
Vlašánek, P. (2018) · Computer Science Research Notes · doi:10.24132/csrn.2018.2801.7 · Cited by 9 · see citing works
MagGAN: High-Resolution Face Attribute Editing with Mask-Guided Generative Adversarial NetworkPreprint
Wei, Y., Gan, Z., Li, W., et al. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2010.01424 · Cited by 6 · see citing works
Kwon, H., & Lee, S.-H. (2022) · Applied Sciences · doi:10.3390/app12178668 · Cited by 4 · see citing works
Flagged first, then by citations
Research works printed but not linked: 2
- [14]Zhao Q. Lyu S. Bai W. Cai L. Liu B. Wu M. Sang X. Yang M. Chen L. A Multi-Modality Ovarian Tumor Ultrasound Image Dataset for Unsupervised Cross-Domain Semantic Segmentation arXiv 2022 2207.06799
- [40]Yu J. Lin Z. Yang J. Shen X. Lu X. Huang T.S. Free-form image inpainting with gated convolution Proceedings of the IEEE/CVF International Conference on Computer Vision Seoul, Republic of Korea 27 October–2 November 2019 4471 4480
Cited by
Sorted by citations
Jost, E., Kosian, P., Jimenez Cruz, J., Albarqouni, S., Gembruch, U., Strizek, B., & Recker, F. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12216833 · Cited by 44 · see citing works
“…studies assessed the performance of automated tumor classification [184,191,192], one study developed a population-based screening method for BRCA mutations [193], and one study focused on the automated elimination of artefacts and objects in US images to increase the accuracy of the AI model [194].”
Method
- Authors
- Chen, L., Qiao, C., Wu, M., Cai, L., Yin, C., Yang, M., Sang, X., & Bai, W.
- Affiliations
- Beihang University (CN)
- Capital Medical University (CN)
- Department of Obstetrics and Gynecology, Beijing Shijitan Hospital, Capital Medical University, Beijing 100038, China
- Corresponding author
- Bai, Wenpei
- Venue
- Bioengineering
- Publisher
- MDPI AG
- Published
- 2023-02-01
- Volume
- 10
- Issue
- 2
- Pages
- 184
- ISSN
- 2306-5354 (ISSN-L)
- PMC
- PMC9952248
- OpenAlex
- W4318832382
- Cited by
- 27
- References
- 45
- Citations by year
- 2026: 4 · 2025: 7 · 2024: 10 · 2023: 6
- FWCI
- 2.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- Bioengineering (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Natural Science Foundation of China (62072021, YWF-22-L-532, DFL20190701)
- Fundamental Research Funds for the Central Universities (62072021, YWF-22-L-532, DFL20190701)
- Beijing Hospitals Authority’Ascent Plan (62072021, YWF-22-L-532, DFL20190701)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Physical Sciences › Computer Science › Artificial Intelligence › AI in cancer detection
- Keywords
- ovarian neoplasms, ultrasound image segmentation, image inpainting, attention mechanism, fast Fourier convolution, ResNet, lesion boundary, computer-aided diagnosis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
