3-D Ultrasound Segmentation of the Placenta Using the Random Walker Algorithm: Reliability and Agreement
Abstract
Volumetric segmentation of the placenta using 3-D ultrasound is currently performed clinically to investigate correlation between organ volume and fetal outcome or pathology. Previously, interpolative or semi-automatic contour-based methodologies were used to provide volumetric results. We describe the validation of an original random walker (RW)-based algorithm against manual segmentation and an existing semi-automated method, virtual organ computer-aided analysis (VOCAL), using initialization time, inter- and intra-observer variability of volumetric measurements and quantification accuracy (with respect to manual segmentation) as metrics of success. Both semi-automatic methods require initialization. Therefore, the first experiment compared initialization times. Initialization was timed by one observer using 20 subjects. This revealed significant differences (p < 0.001) in time taken to initialize the VOCAL method compared with the RW method. In the second experiment, 10 subjects were used to analyze intra-/inter-observer variability between two observers. Bland-Altman plots were used to analyze variability combined with intra- and inter-observer variability measured by intra-class correlation coefficients, which were reported for all three methods. Intra-class correlation coefficient values for intra-observer variability were higher for the RW method than for VOCAL, and both were similar to manual segmentation. Inter-observer variability was 0.94 (0.88, 0.97), 0.91 (0.81, 0.95) and 0.80 (0.61, 0.90) for manual, RW and VOCAL, respectively. Finally, a third observer with no prior ultrasound experience was introduced and volumetric differences from manual segmentation were reported. Dice similarity coefficients for observers 1, 2 and 3 were respectively 0.84 ± 0.12, 0.94 ± 0.08 and 0.84 ± 0.11, and the mean was 0.87 ± 0.13. The RW algorithm was found to provide results concordant with those for manual segmentation and to outperform VOCAL in aspects of observer reliability. The training of an additional untrained observer was investigated, and results revealed that with the appropriate initialization protocol, results for observers with varying levels of experience were concordant. We found that with appropriate training, the RW method can be used for fast, repeatable 3-D measurement of placental volume.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Commercial tools such as VOCAL and a semiautomated RW-derived method have been developed to facilitate PlVol estimation, but they remain too time consuming and operator dependent to be used as anything other than research tools.
“Commercial tools, such as Virtual Organ Computer-aided AnaLysis (VOCAL; General Electric Healthcare) and a semiautomated random walker-derived (RW-derived) method, have been developed (13) to facilitate this process, but they remain too time consuming and operator dependent to be used as anything other than research tools.”
From Fully automated, real-time 3D ultrasound segmentation to estimate first trimester placental volume using deep learning (2018) · cites this paper as [13]
The RW technique has been shown to be comparable to manual segmentation in all aspects of observer reliability and is less time consuming but still remains dependent on the operator's ability to identify the placenta and its boundaries.
“The RW technique has been shown to be comparable to manual segmentation in all aspects of observer reliability (13) and is less time consuming but still remains dependant on the operator's ability to identify the placenta and its boundaries.”
From Fully automated, real-time 3D ultrasound segmentation to estimate first trimester placental volume using deep learning (2018) · cites this paper as [13]
There have been semi-automated and fully automated methods proposed for placental segmentation from 3D ultrasound, with varying results.
“There have been some semi-automated and fully automated methods proposed for the segmentation task, with varying results [10,11,12,13,14,15,16,17].”
From Fully Automated Placental Volume Quantification From<scp>3D</scp>Ultrasound for Prediction of Small‐for‐Gestational‐Age Infants (2022) · cites this paper as [16]
No method has been utilized ultrasound imaging to assess placenta morphology and its change sequentially.
“to the best of our knowledge, no method has been utilized ultrasound imaging to assess placenta morphology and its change sequentially.”
From Automated placental abruption identification using semantic segmentation, quantitative features, SVM, ensemble and multi-path CNN (2023) · cites this paper as [5]
Previous studies were limited to using MRI to assess placenta morphology.
“However, previous studies were limited to using MRI to assess placenta morphology”
From Automated placental abruption identification using semantic segmentation, quantitative features, SVM, ensemble and multi-path CNN (2023) · cites this paper as [5]
Placental volume estimates generated with OxNNet perform well in the prediction of SGA when compared with those generated by the random walker algorithm, and outperform PlaV estimates generated with VOCALTM.
“The placental volume estimates generated with this method perform well in the prediction of SGA when compared with those generated by the random walker algorithm, and outperform PlaV estimates generated with VOCAL™ [35, 38, 41] (see Figure 2).”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
The random walker method is comparable to manual segmentation with respect to reproducibility and outperforms VOCALTM in terms of inter-observer reliability.
“Despite faster volume acquisition, the random walker method is comparable to manual segmentation with respect to reproducibility and outperforms VOCAL™ in terms of inter-observer reliability [35].”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
In early pregnancy, there can be very poor sonographic differentiation between the placenta and myometrium, leading to difficulty determining the true placental border.
“Even in the best hands though, in early pregnancy there can be very poor sonographic differentiation between the placenta and myometrium, which leads to difficulty determining the true placental border [35].”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
The random walker method can still be affected by human error.
“However, this method can still be affected by human error [35, 38].”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
The random walker algorithm overcomes limitations of VOCALTM with respect to an irregularly shaped placenta, is less likely to over-estimate placental volume compared with 2-D techniques, and its initialisation is significantly faster compared with VOCAL (p < 0.001).
“The random walker algorithm overcomes the limitations of VOCAL™ with respect to an irregularly shaped placenta, is less likely to over-estimated placental volume compared with 2-D techniques and its initialisation is significantly faster compared with VOCAL (p = <0.001) [34, 35].”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
The VOCALTM method is purely geometric and estimation of volume in an irregularly shaped organ may be more prone to error.
“This method is purely geometric and consequently estimation of volume in an irregularly shaped organ may be more prone to error [35], this is important as it has been demonstrated that the shape of the placenta in the first trimester is not always regular [34].”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
The placental volumes generated by OxNNet outperform the commercially available tool VOCALTM in the prediction of cSGA.
“The placental volumes generated outperform the commercially available tool VOCAL™ in the prediction of cSGA.”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [35]
Current techniques for quantifying placental volume and morphology from 3DUS images require substantial manual input, which can be cumbersome and poorly standardized.
“As a result of these challenges, current techniques for quantifying placental volume and morphology from 3DUS images require substantial amount of manual input from the user, which can be cumbersome and poorly standardized.”
From Minimally interactive placenta segmentation from three-dimensional ultrasound images (2020) · cites this paper as [9]
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
Yushkevich, P. A., Piven, J., Hazlett, H. C., Smith, R. G., Ho, S., Gee, J. C., & Gerig, G. (2006) · NeuroImage · doi:10.1016/j.neuroimage.2006.01.015 · Cited by 8,868 · see citing works
van der Vorst, H. A. (1992) · SIAM Journal on Scientific and Statistical Computing · doi:10.1137/0913035 · Cited by 5,365 · see citing works
Random Walks for Image Segmentation
Grady, L. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.233 · Cited by 2,624 · see citing works
Guidelines for Reporting Reliability and Agreement Studies (GRRAS) were proposed
Kottner, J., Audigé, L., Brorson, S., Donner, A. P., Gajewski, B. J., Hróbjartsson, A., Roberts, C., Shoukri, M. M., & Streiner, D. L. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.03.002 · Cited by 2,239 · see citing works
Kloosterman, G. J. (1970) · International Journal of Gynecology & Obstetrics · doi:10.1002/j.1879-3479.1970.tb00313.x · Cited by 648 · see citing works
Raine‐Fenning, N., Clewes, J. S., Kendall, N. R., Bunkheila, A., Campbell, B. K., & Johnson, I. R. (2003) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.61 · Cited by 228 · see citing works
Centrality of the Umbilical Cord Insertion in a Human Placenta Influences the Placental Efficiency
Yampolsky, M., Salafia, C. M., Shlakhter, O., Haas, D., Eucker, B., & Thorp, J. M. (2009) · Placenta · doi:10.1016/j.placenta.2009.10.001 · Cited by 125 · see citing works
Falcón, O., Węgrzyn, P., Faro, C. B. D. A., Peralta, C. F. A., & Nicolaides, K. H. (2005) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.1898 · Cited by 109 · see citing works
Wegrzyn, P., Faro, C., Falcon, O., Peralta, C. F. A., & Nicolaides, K. H. (2005) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.1923 · Cited by 103 · see citing works
Placental Volume at 11–13 Weeks’ Gestation in the Prediction of Birth Weight Percentile
Plasencia, W., Akolekar, R., Dagklis, T., Veduta, A., & Nicolaides, K. H. (2011) · Fetal Diagnosis and Therapy · doi:10.1159/000324318 · Cited by 86 · see citing works
Second-trimester placental volume measurement by ultrasound: Prediction of fetal outcome
Wolf, H., Oosting, H., & Treffers, P. E. (1989) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(89)90102-6 · Cited by 74 · see citing works
Rizzo, G., Capponi, A., Pietrolucci, M. E., Capece, A., & Arduini, D. (2009) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2009.28.12.1615 · Cited by 68 · see citing works
Odeh, M., Ophir, E., Maximovsky, O., Grinin, V., & Bornstein, J. (2011) · Prenatal Diagnosis · doi:10.1002/pd.2697 · Cited by 66 · see citing works
Which ultrasound or biochemical markers are independent predictors of small‐for‐gestational age?
Law, L. W., Leung, T. Y., Sahota, D. S., Chan, L. W., Fung, T. Y., & Lau, T. K. (2009) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.6455 · Cited by 59 · see citing works
Wataganara, T., Metzenbauer, M., Peter, I., Johnson, K. L., & Bianchi, D. W. (2005) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2004.12.015 · Cited by 58 · see citing works
Collins, S. L., Stevenson, G. N., Noble, J. A., Impey, L., & Welsh, A. W. (2012) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.10122 · Cited by 52 · see citing works
Self-citationThe volumetric growth of the human placenta: A longitudinal ultrasonic study
Bleker, O. P., Kloosterman, G. J., Breur, W., & Mieras, D. J. (1977) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(77)90369-6 · Cited by 50 · see citing works
Hoogland, H. J., de Haan, J., & Martin, C. B. (1980) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(80)90143-x · Cited by 50 · see citing works
Cheong, K. B., Leung, K. Y., Li, T. K. T., Chan, H. Y., Lee, Y. P., & Tang, M. H. Y. (2010) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.7609 · Cited by 45 · see citing works
Comparison of Placental Volume in Early Pregnancy Using Multiplanar and VOCAL Methods
NOWAK, P. M., Nardozza, L. M. M., Júnior, E. A., Rôlo, L. C., & Moron, A. F. (2008) · Placenta · doi:10.1016/j.placenta.2007.12.003 · Cited by 43 · see citing works
Collins, S. L., Stevenson, G. N., Noble, J. A., & Impey, L. (2013) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2012.09.003 · Cited by 39 · see citing works
Self-citationStevenson, G. N., Collins, S. L., Welsh, A. W., Impey, L. W., & Noble, J. A. (2015) · Radiology · doi:10.1148/radiol.14132363 · Cited by 38 · see citing works
Self-citationWolf, H., Oosting, H., & Treffers, P. E. (1989) · American Journal of Obstetrics and Gynecology · doi:10.1016/0002-9378(89)90651-0 · Cited by 38 · see citing works
Huster, K., Haas, K., Schoenborn, J. A., McVean, D., & Odibo, A. O. (2010) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2010.29.6.911 · Cited by 37 · see citing works
Effect of malaria on placental volume measured using three-dimensional ultrasound: a pilot study
Rijken, M. J., Moroski, W. E., Kiricharoen, S., Karunkonkowit, N., Stevenson, G. N., Ohuma, E. O., Noble, J. A., Kennedy, S., McGready, R. M., Papageorghiou, A. T., et al. (2012) · Malaria Journal · doi:10.1186/1475-2875-11-5 · Cited by 16 · see citing works
Self-citationComparison of 2-D and 3-D estimates of placental volume in early pregnancy
Aye, C. Y. L., Stevenson, G. N., Impey, L., & Collins, S. L. (2015) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2014.10.001 · Cited by 13 · see citing works
Self-citationIs the placental disk really an ellipse?
Yampolsky, M., Salafia, C. M., Misra, D. P., Shlakhter, O., & Gill, J. S. (2013) · Placenta · doi:10.1016/j.placenta.2012.12.002 · Cited by 13 · see citing works
Automatic performance optimization in ViennaCL for GPUs
Rupp, K., Weinbub, J., & Rudolf, F. (2010) · doi:10.1145/2039312.2039318 · Cited by 11 · see citing works
OP10.09: A novel semi‐automated (SA) technique for 3D ultrasound measurement of placental volume
Stevenson, G. N., Collins, S., Impey, L. W. M., & Noble, J. A. (2010) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.8020 · Cited by 8 · see citing works
Self-citationSurface parameterisation of the utero/placental interface using 3D power doppler ultrasound
Stevenson, G. N., Collins, S., Impey, L. W. M., et al. (2011) · IEEE International Symposium on Biomedical Imaging: From Nano to Macro (ISBI) · doi:10.1109/isbi.2011.5872547 · Cited by 4 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
User-Guided Segmentation of Multi-modality Medical Imaging Datasets with ITK-SNAP
Yushkevich, P. A., Pashchinskiy, A., Oguz, I., Mohan, S., Schmitt, J. E., Stein, J. M., Zukić, D., Vicory, J., McCormick, M., Yushkevich, N., et al. (2019) · Neuroinformatics · doi:10.1007/s12021-018-9385-x · Cited by 175 · see citing works
Towards automated semantic segmentation in prenatal volumetric ultrasound
Yang, X., Yu, L., Li, S., Wen, H., Luo, D., Bian, C., Qin, J., Ni, D., & Heng, P.-A. (2019) · IEEE Transactions on Medical Imaging · doi:10.1109/tmi.2018.2858779 · Cited by 115 · see citing works
Looney, P., Stevenson, G. N., Nicolaides, K. H., Plasencia, W., Molloholli, M., Natsis, S., & Collins, S. L. (2018) · JCI Insight · doi:10.1172/jci.insight.120178 · Cited by 111 · see citing works
“The RW technique has been shown to be comparable to manual segmentation in all aspects of observer reliability (13) and is less time consuming but still remains dependant on the operator’s ability to identify the placenta and its boundaries.”
MethodInfluentialSelf-citationSegmentation and classification in MRI and US fetal imaging: Recent trends and future prospects
Torrents-Barrena, J., Piella, G., Masoller, N., Gratacós, E., Eixarch, E., Ceresa, M., & Ballester, M. Á. G. (2019) · Medical Image Analysis · doi:10.1016/j.media.2018.10.003 · Cited by 97 · see citing works
Fast Fully Automatic Segmentation of the Human Placenta from Motion Corrupted MRI
Alansary, A., Kamnitsas, K., Davidson, A., Khlebnikov, R., Rajchl, M., Malamateniou, C., Rutherford, M., Hajnal, J. V., Glocker, B., Rueckert, D., et al. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46723-8_68 · Cited by 77 · see citing works
Towards Automatic Semantic Segmentation in Volumetric Ultrasound
Yang, X., Yu, L., Li, S., Wang, X., Wang, N., Qin, J., Ni, D., & Heng, P.-A. (2017) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-66182-7_81 · Cited by 61 · see citing works
Automatic 3D ultrasound segmentation of the first trimester placenta using deep learning
Looney, P., Stevenson, G. N., Nicolaides, K. H., Plasencia, W., Molloholli, M., Natsis, S., & Collins, S. L. (2017) · 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) · doi:10.1109/isbi.2017.7950519 · Cited by 47 · see citing works
Self-citationTorrents-Barrena, J., Monill, N., Piella, G., Gratacós, E., Eixarch, E., Ceresa, M., & González Ballester, M. A. (2021) · Academic Radiology · doi:10.1016/j.acra.2019.11.006 · Cited by 42 · see citing works
Looney, P., Yin, Y., Collins, S. L., Nicolaides, K. H., Plasencia, W., Molloholli, M., Natsis, S., & Stevenson, G. N. (2021) · IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control · doi:10.1109/tuffc.2021.3052143 · Cited by 40 · see citing works
Self-citationVolume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta
Mathewlynn, S., & Collins, S. L. (2019) · Placenta · doi:10.1016/j.placenta.2019.06.379 · Cited by 30 · see citing works
Self-citationHu, R., Singla, R., Yan, R., Mayer, C., & Rohling, R. N. (2019) · 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2019.8857448 · Cited by 27 · see citing works
Collins, S. L., Welsh, A. W., Impey, L., Noble, J. A., & Stevenson, G. N. (2017) · PLOS ONE · doi:10.1371/journal.pone.0178675 · Cited by 25 · see citing works
Self-citationTowards Whole Placenta Segmentation at Late Gestation Using Multi-view Ultrasound Images
Zimmer, V. A., Gomez, A., Skelton, E., Toussaint, N., Zhang, T., Khanal, B., Wright, R., Noh, Y., Ho, A., Matthew, J., et al. (2019) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-32254-0_70 · Cited by 20 · see citing works
Combining Deep Learning and Multi-atlas Label Fusion for Automated Placenta Segmentation from 3DUS
Oguz, B. U., Wang, J., Yushkevich, N., Pouch, A., Gee, J., Yushkevich, P. A., Schwartz, N., & Oguz, I. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-00807-9_14 · Cited by 13 · see citing works
Minimally interactive placenta segmentation from three-dimensional ultrasound images
Oguz, I., Yushkevich, N., Pouch, A. M., Oguz, B. U., Wang, J., Parameshwaran, S., Gee, J. C., Yushkevich, P. A., & Schwartz, N. (2020) · Journal of Medical Imaging · doi:10.1117/1.jmi.7.1.014004 · Cited by 13 · see citing works
Schwartz, N., Oguz, I., Wang, J., Pouch, A., Yushkevich, N., Parameshwaran, S., Gee, J., Yushkevich, P., & Oguz, B. (2022) · Journal of Ultrasound in Medicine · doi:10.1002/jum.15835 · Cited by 11 · see citing works
Asadpour, V., Puttock, E. J., Getahun, D., Fassett, M. J., & Xie, F. (2023) · Heliyon · doi:10.1016/j.heliyon.2023.e13577 · Cited by 11 · see citing works
Torrents-Barrena, J., Piella, G., Masoller, N., Gratacos, E., Eixarch, E., Ceresa, M., & Ballester, M. A. G. (2019) · 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019) · doi:10.1109/isbi.2019.8759296 · Cited by 9 · see citing works
- Authors
- Stevenson, G. N., Collins, S. L., Ding, J., Impey, L., & Noble, J. A.
- Affiliations
- Rosie Hospital (GB)
- John Radcliffe Hospital (GB)
- University of Oxford (GB)
- Corresponding author
- Stevenson, Gordon N.
- Venue
- Ultrasound in Medicine & Biology
- Publisher
- Elsevier BV
- Published
- 2015-12
- Volume
- 41
- Issue
- 12
- Pages
- 3182-3193
- ISSN
- 0301-5629 (ISSN-L)
- OpenAlex
- W1205104456
- Cited by
- 46
- References
- 32
- Citations by year
- 2026: 2 · 2025: 2 · 2024: 3 · 2023: 2 · 2022: 2 · 2021: 5 · 2020: 7 · 2019: 6 · 2018: 9 · 2017: 5 · 2016: 3
- FWCI
- 4.92
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Ultrasound in Medicine & Biology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- National Institute for Health and Care Research
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Medical Image Segmentation Techniques
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Ultrasound in Clinical Applications
- Keywords
- placenta segmentation, 3D ultrasound, random walker algorithm, VOCAL, observer variability, interobserver agreement, start-up time, semi-automatic segmentation, observer training
- MeSH terms
- Algorithms, Female, Humans, Organ Size, Placenta, Pregnancy, Ultrasonography, Reproducibility of Results, Observer Variation, Imaging, Three-Dimensional
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
