Multi-task learning for quality assessment of fetal head ultrasound images
Abstract
It is essential to measure anatomical parameters in prenatal ultrasound images for the growth and development of the fetus, which is highly relied on obtaining a standard plane. However, the acquisition of a standard plane is, in turn, highly subjective and depends on the clinical experience of sonographers. In order to deal with this challenge, we propose a new multi-task learning framework using a faster regional convolutional neural network (MF R-CNN) architecture for standard plane detection and quality assessment. MF R-CNN can identify the critical anatomical structure of the fetal head and analyze whether the magnification of the ultrasound image is appropriate, and then performs quality assessment of ultrasound images based on clinical protocols. Specifically, the first five convolution blocks of the MF R-CNN learn the features shared within the input data, which can be associated with the detection and classification tasks, and then extend to the task-specific output streams. In training, in order to speed up the different convergence of different tasks, we devise a section train method based on transfer learning. In addition, our proposed method also uses prior clinical and statistical knowledge to reduce the false detection rate. By identifying the key anatomical structure and magnification of the ultrasound image, we score the ultrasonic plane of fetal head to judge whether it is a standard image or not. Experimental results on our own-collected dataset show that our method can accurately make a quality assessment of an ultrasound plane within half a second. Our method achieves promising performance compared with state-of-the-art methods, which can improve the examination effectiveness and alleviate the measurement error caused by improper ultrasound scanning.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Models by Lin et al., Yaqub et al., and Skelton et al. effectively performed image quality assessment of whether a standard plane was correctly acquired.
“The image quality assessment of whether a standard plane was correctly acquired, either by human operators or by automated extraction from 3D images, was effectively performed by models developed by the research groups of Lin et al. [90], Yaqub et al. [91], and Skelton et al. [92].”
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474 · see citing works
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 74,248 · see citing works
You Only Look Once: Unified, Real-Time Object Detection
Redmon, J., Divvala, S., Girshick, R., & Farhadi, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.91 · Cited by 53,803 · see citing works
(2008) · Journal of Machine Learning Research · Cited by 33,051 · see citing works
(2016) · Cited by 31,590 · see citing works
Fast R-CNNPreprint
Girshick, R. B. (2015) · arXiv (Cornell University) · doi:10.48550/arxiv.1504.08083 · Cited by 29,416 · see citing works
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
Selvaraju, R. R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., & Batra, D. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.74 · Cited by 24,513 · see citing works
YOLO9000: Better, Faster, Stronger
Redmon, J., & Farhadi, A. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.690 · Cited by 19,892 · see citing works
A survey on deep learning in medical image analysis
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., van der Laak, J. A. W. M., van Ginneken, B., & Sánchez, C. I. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.07.005 · Cited by 15,332 · see citing works
Proceedings of IEEE Conference on Computer Vision and Pattern Recognition
(1993) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.1993.341010 · Cited by 14,474 · see citing works
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2389824 · Cited by 11,971 · see citing works
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428 · see citing works
How transferable are features in deep neural networks?
Yosinski, J., Clune, J., Bengio, Y., & Lipson, H. (2014) · arXiv · doi:10.48550/arxiv.1411.1792 · Cited by 6,061 · see citing works
Proceedings European Conference on Computer Vision 2002
Heyden, A., Sparr, G., Nielsen, M., & Johansen, P. (2002) · Lund University Publications (Lund University) · Cited by 5,153 · see citing works
Domain Adaptation via Transfer Component Analysis
Pan, S. J., Tsang, I. W.-H., Kwok, J. T.-Y., & Yang, Q. (2010) · IEEE Transactions on Neural Networks · doi:10.1109/tnn.2010.2091281 · Cited by 4,809 · see citing works
2013 IEEE International Conference on Computer Vision
(2013) · doi:10.1109/iccv19984.2013 · Cited by 4,232 · see citing works
DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition
Donahue, J., Jia, Y., Vinyals, O., Hoffman, J., Zhang, N., Tzeng, E., & Darrell, T. (2013) · arXiv · doi:10.48550/arxiv.1310.1531 · Cited by 4,019 · see citing works
Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks
Chattopadhay, A., Sarkar, A., Howlader, P., & Balasubramanian, V. N. (2018) · 2018 IEEE Winter Conference on Applications of Computer Vision (WACV) · doi:10.1109/wacv.2018.00097 · Cited by 3,639 · see citing works
Convolutional Neural Networks for Medical Image Analysis: Full Training or Fine Tuning?
Tajbakhsh, N., Shin, J. Y., Gurudu, S. R., Hurst, R. T., Kendall, C. B., Gotway, M. B., & Liang, J. (2016) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2016.2535302 · Cited by 3,277 · see citing works
Soft-NMS — Improving Object Detection with One Line of Code
Bodla, N., Singh, B., Chellappa, R., & Davis, L. S. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.593 · Cited by 2,274 · see citing works
2007 IEEE Conference on Computer Vision and Pattern Recognition
(2007) · doi:10.1109/cvpr12085.2007 · Cited by 1,726 · see citing works
IEEE Winter Conference on Applications of Computer Vision
(2014) · doi:10.1109/wacv31811.2014 · Cited by 1,001 · see citing works
Paladini, D., Malinger, G., Monteagudo, A., Timor Tritsch, I., Toi, A., & Pilu, G. (2007) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.3909 · Cited by 577 · see citing works
SonoNet: Real-Time Detection and Localisation of Fetal Standard Scan Planes in Freehand Ultrasound
Baumgartner, C. F., Kamnitsas, K., Matthew, J., Fletcher, T. P., Smith, S., Koch, L. M., Kainz, B., & Rueckert, D. (2017) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2017.2712367 · Cited by 416 · see citing works
Standard Plane Localization in Fetal Ultrasound via Domain Transferred Deep Neural Networks
Chen, H., Ni, D., Qin, J., Li, S., Yang, X., Wang, T., & Heng, P. A. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2425041 · Cited by 379 · see citing works
Self-citationSINet: A Scale-Insensitive Convolutional Neural Network for Fast Vehicle Detection
Hu, X., Xu, X., Xiao, Y., Chen, H., He, S., Qin, J., & Heng, P.-A. (2019) · IEEE Transactions on Intelligent Transportation Systems · doi:10.1109/tits.2018.2838132 · Cited by 341 · see citing works
FUIQA: Fetal Ultrasound Image Quality Assessment with Deep Convolutional Networks
Wu, L., Cheng, J.-Z., Li, S., Lei, B., Wang, T., & Ni, D. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2671898 · Cited by 242 · see citing works
Carneiro, G., Georgescu, B., Good, S., & Comaniciu, D. (2008) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2008.928917 · Cited by 205 · see citing works
Ultrasound image-based thyroid nodule automatic segmentation using convolutional neural networks
Ma, J., Wu, F., Jiang, T., Zhao, Q., & Kong, D. (2017) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-017-1649-7 · Cited by 187 · see citing works
Shi, J., Zhou, S., Liu, X., Zhang, Q., Lu, M., & Wang, T. (2016) · Neurocomputing · doi:10.1016/j.neucom.2016.01.074 · Cited by 166 · see citing works
Self-citationUltrasound Standard Plane Detection Using a Composite Neural Network Framework
Chen, H., Wu, L., Dou, Q., Qin, J., Li, S., Cheng, J.-Z., Ni, D., & Heng, P.-A. (2017) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2685080 · Cited by 164 · see citing works
Self-citationShin, S. Y., Lee, S., Yun, I. D., Kim, S. M., & Lee, K. M. (2018) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2872031 · Cited by 160 · see citing works
Full left ventricle quantification via deep multitask relationships learning
Xue, W., Brahm, G. L., Pandey, S. K., Leung, S., & Li, S. (2017) · Medical Image Analysis · doi:10.1016/j.media.2017.09.005 · Cited by 157 · see citing works
Self-citationSalomon, L. J., Bernard, J. P., Duyme, M., Doris, B., Mas, N., & Ville, Y. G. (2005) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.2665 · Cited by 146 · see citing works
Diagnosis of midline anomalies of the fetal brain with the three-dimensional median view
Pilu, G., Segata, M., Ghi, T., et al. (2006) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.2751 · Cited by 125 · see citing works
Li, J., Wang, Y., Lei, B., Cheng, J.-Z., Qin, J., Wang, T., Li, S., & Ni, D. (2018) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2703890 · Cited by 120 · see citing works
Self-citationBucher, H. C., & Schmidt, J. G. (1993) · BMJ · doi:10.1136/bmj.307.6895.13 · Cited by 117 · see citing works
Towards automated semantic segmentation in prenatal volumetric ultrasound
Yang, X., Yu, L., Li, S., Wen, H., Luo, D., Bian, C., Qin, J., Ni, D., & Heng, P.-A. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2858779 · Cited by 115 · see citing works
Self-citation(1999) · Radiology · doi:10.1148/radiology.210.2.r99fe68404 · Cited by 104 · see citing works
Namburete, A. I. L., Xie, W., Yaqub, M., Zisserman, A., & Noble, J. A. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.02.006 · Cited by 95 · see citing works
The importance of quality management in fetal measurement
Dudley, N. J., & Chapman, E. (2002) · Ultrasound in Obstetrics & Gynecology · doi:10.1046/j.0960-7692.2001.00549.x · Cited by 94 · see citing works
Wagner, M. E. H., Gellrich, N., Friese, K., Becker, M., Wolter, F.-E., Lichtenstein, J., Stoetzer, M., Rana, M., & Essig, H. (2015) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-015-1228-8 · Cited by 80 · see citing works
Sinclair, M. D., Baumgartner, C. F., Matthew, J., Bai, W., Martinez, J. C., Li, Y., Smith, S., Knight, C. L., Kainz, B., Hajnal, J. V., et al. (2018) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2018.8512278 · Cited by 78 · see citing works
Shi, J., Xue, Z., Dai, Y., Peng, B., Dong, Y., Zhang, Q., & Zhang, Y. (2018) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2018.2889398 · Cited by 76 · see citing works
Standard Plane Localization in Ultrasound by Radial Component Model and Selective Search
Ni, D., Yang, X., Chen, X., Chin, C.-T., Chen, S., Heng, P. A., Li, S., Qin, J., & Wang, T. (2014) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2014.06.006 · Cited by 65 · see citing works
Standard Plane Detection in 3D Fetal Ultrasound Using an Iterative Transformation Network
Li, Y., Khanal, B., Hou, B., Alansary, A., Cerrolaza, J. J., Sinclair, M., Matthew, J., Gupta, C., Knight, C., Kainz, B., et al. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-00928-1_45 · Cited by 61 · see citing works
Sundaresan, V., Bridge, C. P., Ioannou, C., & Noble, J. A. (2017) · 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) · doi:10.1109/isbi.2017.7950609 · Cited by 58 · see citing works
Acquisition of Localization Confidence for Accurate Object DetectionPreprint
Jiang, B., Luo, R., Mao, J., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1807.11590 · Cited by 57 · see citing works
Learning non-maximum suppressionPreprint
Hosang, J., Benenson, R., & Schiele, B. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1705.02950 · Cited by 48 · see citing works
VP-Nets : Efficient automatic localization of key brain structures in 3D fetal neurosonography
Huang, R., Xie, W., & Noble, J. A. (2018) · Medical Image Analysis · doi:10.1016/j.media.2018.04.004 · Cited by 47 · see citing works
Describing ultrasound video content using deep convolutional neural networks
Gao, Y., Maraci, M. A., & Noble, J. A. (2016) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2016.7493384 · Cited by 46 · see citing works
Automatic image quality assessment and measurement of fetal head in two-dimensional ultrasound image
Zhang, L., Dudley, N. J., Lambrou, T., Allinson, N. M., & Ye, X. (2017) · Journal of medical imaging · doi:10.1117/1.jmi.4.2.024001 · Cited by 33 · see citing works
Reflections on ultrasound image analysis
Noble, J. A. (2016) · Medical Image Analysis · doi:10.1016/j.media.2016.06.015 · Cited by 29 · see citing works
Plane Localization in 3-D Fetal Neurosonography for Longitudinal Analysis of the Developing Brain
Yaqub, M. K., Rueda, S., Kopuri, A., Melo, P., Papageorghiou, A. T., Sullivan, P. B., McCormick, K., & Noble, J. A. (2015) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2015.2435651 · Cited by 14 · see citing works
IEEE International Symposium on Biomedical Imaging
(2016) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2016.2610586 · Cited by 9 · see citing works
Xu, Z., Huo, Y., Park, J., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1805.10376 · Cited by 7 · see citing works
Weakly Supervised Estimation of Shadow Confidence Maps in Fetal Ultrasound ImagingPreprint
Meng, Q., Sinclair, M. D., Zimmer, V. A., et al. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1811.08164 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Explainable artificial intelligence (XAI) in deep learning-based medical image analysis
van der Velden, B. H. M., Kuijf, H. J., Gilhuijs, K. G. A., & Viergever, M. A. (2022) · Medical Image Analysis · doi:10.1016/j.media.2022.102470 · Cited by 1,328 · see citing works
Jost, E., Kosian, P., Jimenez Cruz, J., Albarqouni, S., Gembruch, U., Strizek, B., & Recker, F. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12216833 · Cited by 44 · see citing works
“The image quality assessment of whether a standard plane was correctly acquired, either by human operators or by automated extraction from 3D images, was effectively performed by models developed by the research groups of Lin et al. [90], Yaqub et al. [91], and Skelton et al. [92].”
MethodAgent with Warm Start and Adaptive Dynamic Termination for Plane Localization in 3D Ultrasound
Yang, X., Dou, H., Huang, R., Xue, W., Huang, Y., Qian, J., Zhang, Y., Luo, H., Guo, H., Wang, T., et al. (2021) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2021.3069663 · Cited by 32 · see citing works
DeepGA for automatically estimating fetal gestational age through ultrasound imaging
Dan, T., Chen, X., He, M., Guo, H., He, X., Chen, J., Xian, J., Hu, Y., Zhang, B., Wang, N., et al. (2023) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2022.102453 · Cited by 25 · see citing works
- Authors
- Lin, Z., Li, S., Ni, D., Liao, Y., Wen, H., Du, J., Chen, S., Wang, T., & Lei, B.
- Affiliations
- Shenzhen University (CN)
- Shenzhen Maternity and Child Healthcare Hospital (CN)
- Corresponding author
- Wang, Tianfu
- Lei, Baiying
- ORCID
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2019-12
- Volume
- 58
- Pages
- 101548
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W2971555134
- Cited by
- 129
- References
- 58
- Citations by year
- 2026: 14 · 2025: 18 · 2024: 18 · 2023: 29 · 2022: 23 · 2021: 19 · 2020: 6 · 2019: 2
- FWCI
- 5.53
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Medical Image Analysis (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Shenzhen Fundamental Research Program (JCYJ20170818142347251, JCYJ20180507184647636, JCYJ20170818094109846, JCYJ20170413152804728)
- Guangdong Provincial Pearl River Talents Program (2016ZT06S220)
- Shenzhen Peacock Plan (KQTD2015033016104926, KQTD2016053112051497)
- National Natural Science Foundation of China (61871274, 81571758, 61801305)
- Guangdong Medical Research Foundation (B2018031)
- Natural Science Foundation of Guangdong Province (2017A030313377)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Advanced Neural Network Applications
- Keywords
- image quality assessment, multi-task learning, Faster R-CNN, transfer learning, anatomical structure detection, prenatal ultrasound
- MeSH terms
- Female, Head, Humans, Image Interpretation, Computer-Assisted, Pregnancy, Ultrasonography, Prenatal, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
