Learning to map 2D ultrasound images into 3D space with minimal human annotation
Abstract
In fetal neurosonography, aligning two-dimensional (2D) ultrasound scans to their corresponding plane in the three-dimensional (3D) space remains a challenging task. In this paper, we propose a convolutional neural network that predicts the position of 2D ultrasound fetal brain scans in 3D atlas space. Instead of purely supervised learning that requires heavy annotations for each 2D scan, we train the model by sampling 2D slices from 3D fetal brain volumes, and target the model to predict the inverse of the sampling process, resembling the idea of self-supervised learning. We propose a model that takes a set of images as input, and learns to compare them in pairs. The pairwise comparison is weighted by the attention module based on its contribution to the prediction, which is learnt implicitly during training. The feature representation for each image is thus computed by incorporating the relative position information to all the other images in the set, and is later used for the final prediction. We benchmark our model on 2D slices sampled from 3D fetal brain volumes at 18-22 weeks' gestational age. Using three evaluation metrics, namely, Euclidean distance, plane angles and normalized cross correlation, which account for both the geometric and appearance discrepancy between the ground-truth and prediction, in all these metrics, our model outperforms a baseline model by as much as 23%, when the number of input images increases. We further demonstrate that our model generalizes to (i) real 2D standard transthalamic plane images, achieving comparable performance as human annotations, as well as (ii) video sequences of 2D freehand fetal brain scans.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Sonographers performing neurosonography require accurate understanding of fetal neuroanatomy, skill to interpret 2D planes in a complex 3D structure, and substantial clinical experience and training.
“Sonographers performing neurosonography require an accurate understanding of fetal neuroanatomy, the skill to interpret 2D planes in a complex 3D structure, and, consequently, substantial clinical experience and training [78].”
AI algorithms in fetal neurosonography enabled the development of guiding methods for skill training.
“the development of guiding methods for skill training [78,83]”
Yeung et al., Namburete et al., Yu et al., and Di Vece et al. proposed a plane localization system as a 3D reference space for locating 2D planes to improve acquisition of standard planes and facilitate anatomical orientation.
“The establishment of a plane localization system as a 3D reference space for locating 2D planes was proposed by Yeung et al., Namburete et al., Yu et al., and Di Vece et al. for improving the acquisition of standard planes and facilitating anatomical orientation for sonographers [78,80,82,83].”
The primary limitation of AI US imaging in fetal neurosonography was rapid anatomical development of fetal brain structures due to brain maturation, increasing head size and degree of ossification with rising GA.
“The primary limitation of AI US imaging in this topic was described to be the rapid anatomical development of fetal brain structures due to brain maturation, increasing head size and degree of ossification with rising GA [78,80,84].”
AI models designed for automatic biometric measurements are usually restricted to a specific range of GA and can fail in images of different GA.
“AI models designed for automatic biometric measurements are usually restraint to a specific range of GA and can fail in images of different GA [34,40,78].”
The citation lists haven’t been fetched yet
References 49 · Cited by 55
- Authors
- Yeung, P.-H., Aliasi, M., Papageorghiou, A. T., Haak, M., Xie, W., & Namburete, A. I. L.
- Affiliations
- University of Oxford (GB)
- Leiden University Medical Center (NL)
- Corresponding author
- Yeung, Pak-Hei
- Venue
- Medical Image Analysis
- Publisher
- Elsevier BV
- Published
- 2021-05
- Volume
- 70
- Pages
- 101998
- ISSN
- 1361-8415 (ISSN-L)
- OpenAlex
- W3132522774
- Cited by
- 55
- References
- 49
- Citations by year
- 2026: 7 · 2025: 14 · 2024: 15 · 2023: 11 · 2022: 7 · 2021: 1
- FWCI
- 3.06
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Locations
- Medical Image Analysis (publishedVersion)
- PubMed (publishedVersion)
- Funders
- Engineering and Physical Sciences Research Council (EP/M013774/1, EP/T028572/1)
- National Institute for Health Research
- Royal Academy of Engineering
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Fetal and Pediatric Neurological Disorders
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Keywords
- fetal neurosonography, self-supervised learning, attention mechanism, convolutional neural networks, slice sampling, freehand ultrasound
- MeSH terms
- Gestational Age, Humans, Image Processing, Computer-Assisted, Ultrasonography, Neural Networks, Computer, Neuroimaging
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review