Simulating realistic fetal neurosonography images with appearance and growth change using cycle-consistent adversarial networks and an evaluation
Open accessAbstract
Purpose: We present an original method for simulating realistic fetal neurosonography images specifically generating third-trimester pregnancy ultrasound images from second-trimester images. Our method was developed using unpaired data, as pairwise data were not available. We also report original insights on the general appearance differences between second- and third-trimester fetal head transventricular (TV) plane images. Approach: We design a cycle-consistent adversarial network (Cycle-GAN) to simulate visually realistic third-trimester images from unpaired second- and third-trimester ultrasound images. Simulation realism is evaluated qualitatively by experienced sonographers who blindly graded real and simulated images. A quantitative evaluation is also performed whereby a validated deep-learning-based image recognition algorithm (ScanNav®) acts as the expert reference to allow hundreds of real and simulated images to be automatically analyzed and compared efficiently. Results: Qualitative evaluation shows that the human expert cannot tell the difference between real and simulated third-trimester scan images. 84.2% of the simulated third-trimester images could not be distinguished from the real third-trimester images. As a quantitative baseline, on 3000 images, the visibility drop of the choroid, CSP, and mid-line falx between real second- and real third-trimester scans was computed by ScanNav® and found to be 72.5%, 61.5%, and 67%, respectively. The visibility drop of the same structures between real second-trimester and simulated third-trimester was found to be 77.5%, 57.7%, and 56.2%, respectively. Therefore, the real and simulated third-trimester images were consider to be visually similar to each other. Our evaluation also shows that the third-trimester simulation of a conventional GAN is much easier to distinguish, and the visibility drop of the structures is smaller than our proposed method. Conclusions: The results confirm that it is possible to simulate realistic third-trimester images from second-trimester images using a modified Cycle-GAN, which may be useful for deep learning researchers with a restricted availability of third-trimester scans but with access to ample second trimester images. We also show convincing simulation improvements, both qualitatively and quantitatively, using the Cycle-GAN method compared with a conventional GAN. Finally, the use of a machine learning-based reference (in the case ScanNav®) for large-scale quantitative image analysis evaluation is also a first to our knowledge.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The primary limitation of AI US imaging in fetal neurosonography was rapid anatomical development of fetal brain structures due to brain maturation, increasing head size and degree of ossification with rising GA.
“The primary limitation of AI US imaging in this topic was described to be the rapid anatomical development of fetal brain structures due to brain maturation, increasing head size and degree of ossification with rising GA [78,80,84].”
From Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology (2023) · cites this paper as [84]
The Introduction (page unavailable) says fetal neurosonography images change between the second and third trimesters because of brain maturation, fetal size, and skull ossification. Section 3.1 (page unavailable) adds that skull calcification makes internal brain structures harder to see. The paper does not identify these changes as the primary limitation of AI ultrasound imaging. (LLM, conf=0.84)
Experimental — accuracy not yet measuredXu et al. presented an AI method for authentically simulating third-trimester images from second-trimester images.
“Xu et al. presented an AI method for authentically simulating third-trimester images from second-trimester images for deep-learning researchers with restricted access to third-trimester images [84].”
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
U-Net: Convolutional Networks for Biomedical Image Segmentation
Ronneberger, O., Fischer, P., & Brox, T. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-24574-4_28 · Cited by 99,404 · see citing works
Fully convolutional networks for semantic segmentation
Long, J., Shelhamer, E., & Darrell, T. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298965 · Cited by 38,834 · see citing works
Unpaired image-to-image translation using cycle-consistent adversarial networks
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.244 · Cited by 23,602 · see citing works
Labaca-Castro, R. (2023) · Springer eBooks · doi:10.1007/978-3-658-40442-0_9 · Cited by 21,100 · see citing works
Unsupervised representation learning with deep convolutional generative adversarial networks
Radford, A., Metz, L., & Chintala, S. (2015) · arXiv · doi:10.48550/arxiv.1511.06434 · Cited by 12,235 · see citing works
Fully convolutional networks for semantic segmentation
Shelhamer, E., Long, J., & Darrell, T. (2017) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2572683 · Cited by 11,898 · see citing works
“Least squares generative adversarial networks,”
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.304 · Cited by 5,437 · see citing works
Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial NetworksPreprint
Zhu, J.-Y., Park, T., Isola, P., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10593 · Cited by 2,278 · see citing works
Coupled Generative Adversarial NetworksPreprint
Liu, M.-Y., & Tuzel, O. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1606.07536 · Cited by 1,433 · see citing works
Practice guidelines for performance of the routine mid-trimester fetal ultrasound scan
Salomon, L. J., Alfirevic, Z., Berghella, V., Bilardo, C., Hernandez‐Andrade, E., Johnsen, S. L., Kalache, K., Leung, K. ‐., Malinger, G., Munoz, H., et al. (2011) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.8831 · Cited by 1,006 · see citing works
Simulation‐based learning in nurse education: systematic review
Cant, R. P., & Cooper, S. J. (2010) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2009.05240.x · Cited by 984 · see citing works
Sovio, U., White, I. R., Dacey, A., Pasupathy, D., & Smith, G. C. S. (2015) · The Lancet · doi:10.1016/s0140-6736(15)00131-2 · Cited by 610 · see citing works
On the Effectiveness of Least Squares Generative Adversarial Networks
Mao, X., Li, Q., Xie, H., Lau, R. Y. K., Wang, Z., & Smolley, S. P. (2018) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2018.2872043 · Cited by 225 · see citing works
Freehand Ultrasound Image Simulation with Spatially-Conditioned Generative Adversarial Networks
Hu, Y., Gibson, E., Lee, L.-L., Xie, W., Barratt, D. C., Vercauteren, T. K., & Noble, J. A. (2017) · Lecture notes in computer science · doi:10.1007/978-3-319-67564-0_11 · Cited by 85 · see citing works
Self-citationYaqub, M. K., Kelly, B., Stobart, H., Napolitano, R., Noble, J. A., & Papageorghiou, A. T. (2018) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.20144 · Cited by 37 · see citing works
Self-citationDrukker, L., Cavallaro, A., Salim, I., Ioannou, C., Impey, L. W. M., & Papageorghiou, A. T. (2020) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2020.05.052 · Cited by 33 · see citing works
Self-citationDrukker, L., Droste, R., Chatelain, P., Noble, J. A., & Papageorghiou, A. T. (2020) · Ultraschall in der Medizin - European Journal of Ultrasound · doi:10.1055/a-1074-0722 · Cited by 27 · see citing works
Self-citationThe IRIS Study Group, Henrichs, J., Verfaille, V., Viester, L., Westerneng, M., Molewijk, B. C., Franx, A., van der Horst, H. E., Bosmans, J. E., de Jonge, A., et al. (2016) · BMC Pregnancy and Childbirth · doi:10.1186/s12884-016-1104-8 · Cited by 26 · see citing works
On the Effectiveness of Least Squares Generative Adversarial NetworksPreprint
Mao, X., Li, Q., Xie, H., et al. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1712.06391 · Cited by 16 · see citing works
Yaqub, M. K., Cook, K., Cocks, K., Chen, Z., Chikkanna, B., Sleep, N., Noble, J. A., & Papageorghiou, A. T. (2019) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.20656 · Cited by 4 · see citing works
Self-citationAnatomy-Aware Self-supervised Fetal MRI Synthesis from Unpaired Ultrasound Images
Jiao, J., Namburete, A. I. L., Papageorghiou, A. T., et al. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-32692-0_21 · Cited by 4 · see citing works
Self-citationAnatomy-Aware Self-supervised Fetal MRI Synthesis from Unpaired Ultrasound ImagesPreprint
Jiao, J., Namburete, A. I. L., Papageorghiou, A. T., et al. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03483 · Cited by 1 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 4
- J. M. Wolterink et al., "MR-to-CT synthesis using cycle-consistent generative adversarial networks," in 31st Conf. Neural Inf. Process. Syst. (2017).
- T. Salimans et al., "Improved techniques for training GANs," in Adv. Neural Inf. Process. Syst., pp. 2234-2242 (2016).
- Yangdi Xu graduated from the University of Bristol and specialized in machine learning and computer vision. His research focused on modeling human daily routine/activity patterns from images and videos. As a research associate at the University of Oxford, his research focuses on ultrasound image analysis using deep-learning models. His project collaborates with the John Redcliff Hospital and involves but is not limited to obstetric ultrasound image simulations and gaze pattern modeling of sonographers.
- Lok Hin Lee is a DPhil candidate at the Institute of Biomedical Imaging, University of Oxford.
Cited by
Sorted by citations
Jost, E., Kosian, P., Jimenez Cruz, J., Albarqouni, S., Gembruch, U., Strizek, B., & Recker, F. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12216833 · Cited by 44 · see citing works
“Xu et al. presented an AI method for authenti-cally simulating third-trimester images from second-trimester images for deep-learning researchers with restricted access to third-trimester images [84].”
BackgroundMethod
- Authors
- Xu, Y., Lee, L. H., Drukker, L., Yaqub, M., Papageorghiou, A. T., & Noble, A. J.
- Affiliations
- University of Oxford (GB)
- John Radcliffe Hospital (GB)
- Intelligent Ultrasound Limited (United Kingdom) (GB)
- Univ. of Oxford
- Venue
- Journal of medical imaging
- Publisher
- SPIE-Intl Soc Optical Eng
- Published
- 2020-09-16
- Volume
- 7
- Issue
- 05
- Pages
- 1-15
- ISSN
- 2329-4302 (ISSN-L)
- 2329-4310
- PMC
- PMC7492851
- OpenAlex
- W3087663537
- Cited by
- 8
- References
- 22
- Citations by year
- 2025: 1 · 2024: 2 · 2023: 3 · 2022: 2
- FWCI
- 1.20
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Journal of medical imaging (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- St George's Online Research Archive (St George's University of London) (submittedVersion)
- Funders
- National Institute for Health and Care Research
- Engineering and Physical Sciences Research Council (EP/M013774/1, EP/MO13774/1, EP/L505316/1)
- H2020 European Research Council (694581)
- Topics
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Fetal and Pediatric Neurological Disorders
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Generative Adversarial Networks and Image Synthesis
- Keywords
- fetal neurosonography, image simulation, third trimester ultrasound, second-trimester ultrasound, fetal head, CycleGAN, unpaired data, corpus callosum, falx
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
