Comparison of inter‐ and intraobserver agreement and reliability between three different types of placental volume measurement technique (XI VOCAL™, VOCAL™ and multiplanar) and validity in the in‐vitro setting
Abstract
OBJECTIVES: To compare XI VOCAL (eXtended Imaging Virtual Organ Computer-aided AnaLysis) for three-dimensional (3D) ultrasound volumetry of the placenta and of phantom objects with a rotational method using VOCAL and with the multiplanar method. METHODS: We acquired 3D volume datasets from 32 fetuses at 11-14 weeks' gestation. Placental volume was calculated twice by each of two observers using XI VOCAL (with 5, 10, 15 and 20 slices), multiplanar (1-mm interval) and VOCAL (with 12 degrees, 18 degrees and 30 degrees rotation) methods. In addition, validity was assessed using the in-vitro setting with three phantom objects of known volume. RESULTS: Both inter- and intraobserver reliabilities were very high for all three methods. There was no systematic bias between any two methods except between XI VOCAL (10 slices) and the multiplanar (1-mm interval) method, with a smaller volume using the former method. The limits of agreement were wide between any two of the three methods. In the in-vitro setting, there was a trend towards less valid measurements with the XI VOCAL technique and fewer slices. With the same number of steps, measurements made with VOCAL (12 degrees and 18 degrees) were more valid than were those made with XI VOCAL (15 and 10 slices, respectively). CONCLUSION: XI VOCAL cannot be used interchangeably with VOCAL or multiplanar techniques in measuring placental volume at 11-14 weeks' gestation.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
There is uncertainty regarding the repeatability and reproducibility of volume results produced using VOCALTM, with certain iterations producing less reliable results.
“There is also some uncertainty regarding the repeatability and reproducibility of volume results produced using this method, with certain iterations of the VOCAL™ software producing less reliable results [36].”
From Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta (2019) · cites this paper as [36]
Cites
STATISTICAL METHODS FOR ASSESSING AGREEMENT BETWEEN TWO METHODS OF CLINICAL MEASUREMENT
Martin Bland, J., & Altman, D. (1986) · The Lancet · doi:10.1016/s0140-6736(86)90837-8 · Cited by 47,965 · see citing works
Forming inferences about some intraclass correlation coefficients.
McGraw, K. O., & Wong, S. P. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.1.30 · Cited by 6,903 · see citing works
Clinical biostatistics: LIV. The biostatistics of concordance
Kramer, M. S., & Feinstein, A. R. (1981) · Clinical Pharmacology & Therapeutics · doi:10.1038/clpt.1981.18 · Cited by 1,087 · see citing works
Three‐dimensional ultrasound: accuracy of distance and volume measurements
Riccabona, M., Nelson, T. R., & Pretorius, D. H. (1996) · Ultrasound in Obstetrics and Gynecology · doi:10.1046/j.1469-0705.1996.07060429.x · Cited by 275 · see citing works
Raine‐Fenning, N., Clewes, J. S., Kendall, N. R., Bunkheila, A., Campbell, B. K., & Johnson, I. R. (2003) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.61 · Cited by 228 · see citing works
Measuring agreement in medical informatics reliability studies
Hripcsak, G. M., & Heitjan, D. F. (2002) · Journal of Biomedical Informatics · doi:10.1016/s1532-0464(02)00500-2 · Cited by 227 · see citing works
Distance and volume measurement using three-dimensional ultrasonography.
Riccabona, M., Nelson, T. R., Pretorius, D. H., & Davidson, T. E. (1995) · Journal of Ultrasound in Medicine · doi:10.7863/jum.1995.14.12.881 · Cited by 164 · see citing works
Hafner, E., Metzenbauer, M., Höfinger, D., Stonek, F., Schuchter, K., Waldhör, T., & Philipp, K. (2006) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.2641 · Cited by 137 · see citing works
Evaluation of a clinical test. I: Assessment of reliability
Kha, K. S., & Chien, P. F. W. (2001) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2001.00150.x · Cited by 135 · see citing works
Kalache, K. D., Espinoza, J., Chaiworapongsa, T., Londono, J., Schoen, M. L., Treadwell, M. C., Lee, W., & Romero, R. J. (2003) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.39 · Cited by 127 · see citing works
Hafner, E., Metzenbauer, M., Höfinger, D., Münkel, M., Gassner, R., Schuchter, K., Dillinger‐Paller, B., & Philipp, K. (2003) · Placenta · doi:10.1053/plac.2002.0918 · Cited by 127 · see citing works
Fetal lung volumetry using two‐ and three‐dimensional ultrasound
Moeglin, D., Talmant, C., Duyme, M., Lopez, A., & C.F.E.F. (2005) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.1799 · Cited by 113 · see citing works
Raine‐Fenning, N., Campbell, B. K., Clewes, J. S., & Johnson, I. R. (2003) · Ultrasound in Medicine & Biology · doi:10.1016/s0301-5629(03)01068-8 · Cited by 110 · see citing works
Falcón, O., Węgrzyn, P., Faro, C. B. D. A., Peralta, C. F. A., & Nicolaides, K. H. (2005) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.1898 · Cited by 109 · see citing works
Schuchter, K., Metzenbauer, M., Hafner, E., & Philipp, K. (2001) · Ultrasound in Obstetrics and Gynecology · doi:10.1046/j.0960-7692.2001.00596.x · Cited by 95 · see citing works
Hafner, E., Philipp, T., Schuchter, K., Dillinger‐Paller, B., Philipp, K., & Bauer, P. (1998) · Ultrasound in Obstetrics and Gynecology · doi:10.1046/j.1469-0705.1998.12020097.x · Cited by 95 · see citing works
Metzenbauer, M., Hafner, E., Schuchter, K., & Philipp, K. (2002) · Ultrasound in Obstetrics & Gynecology · doi:10.1046/j.1469-0705.2002.00655.x · Cited by 79 · see citing works
Evaluation of a clinical test. II: Assessment of validity
Chien, P. F. W., & Khan, K. S. (2001) · BJOG: An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2001.00128.x · Cited by 78 · see citing works
Three-dimensional first trimester fetal volumetry: comparison with crown rump length
Aviram, R., Shpan, D. K., Markovitch, O., Fishman, A., & Tepper, R. (2004) · Early Human Development · doi:10.1016/j.earlhumdev.2004.02.005 · Cited by 75 · see citing works
de Paula, C. F. S., Ruano, R., Campos, J. Á. D. B., & Zugaib, M. (2008) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2008.27.11.1583 · Cited by 70 · see citing works
Rizzo, G., Capponi, A., Pietrolucci, M. E., & Arduini, D. (2008) · American Journal of Obstetrics and Gynecology · doi:10.1016/j.ajog.2008.10.041 · Cited by 64 · see citing works
Farrell, T. J., Leslie, J., CHIEN, P. F. W., & Agustsson, P. (2001) · BJOG An International Journal of Obstetrics & Gynaecology · doi:10.1111/j.1471-0528.2001.00148.x · Cited by 61 · see citing works
Oedema formation with the vasodilators nifedipine and diazoxide
van Hamersvelt, H. W., Kloke, H. J., de Jong, D. J., Koene, R. A. R., & Huysmans, F. T. M. (1996) · Journal of Hypertension · doi:10.1097/00004872-199608000-00016 · Cited by 59 · see citing works
Hafner, E., Schuchter, K., van Leeuwen, M., Metzenbauer, M., Dillinger‐Paller, B., & Philipp, K. (2001) · Ultrasound in Obstetrics and Gynecology · doi:10.1046/j.1469-0705.2001.00489.x · Cited by 56 · see citing works
Rovas, L., Sladkevicius, P., Strobel, E., & Valentin, L. (2005) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.1884 · Cited by 54 · see citing works
Comparison of Placental Volume in Early Pregnancy Using Multiplanar and VOCAL Methods
NOWAK, P. M., Nardozza, L. M. M., Júnior, E. A., Rôlo, L. C., & Moron, A. F. (2008) · Placenta · doi:10.1016/j.placenta.2007.12.003 · Cited by 43 · see citing works
Cheong, K. B., Leung, K. Y., Chan, H. Y., Lee, Y. P., Yang, F., & Tang, M. H. Y. (2009) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.6255 · Cited by 32 · see citing works
Chen, M., Leung, K. Y., Lee, C. P., Tang, M. H. Y., & Ho, P. C. (2006) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.2721 · Cited by 31 · see citing works
Węgrzyn, P., Fabio, C., Peralta, A., Faro, C. B. D. A., Borenstein, M., & Nicolaides, K. H. (2006) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.2783 · Cited by 31 · see citing works
Filho, H. A. G., da Costa, L. L. D., Júnior, E. A., Pires, C. R., Nardozza, L. M. M., & Mattar, R. (2007) · Archives of Gynecology and Obstetrics · doi:10.1007/s00404-006-0312-2 · Cited by 25 · see citing works
P28.02: Three‐dimensional sonographic measurements of placental volume at early pregnancy
Lee, P. R., Shim, J., Kim, A., Ma, J. Y., & Won, H. (2007) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.4730 · Cited by 1 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Standardized approach for imaging and measuring Cesarean section scars using ultrasonography
Naji, O., Abdallah, Y., de Vaate, A. J. M. B., Smith, A. A., Pexsters, A., Stalder, C. M., McIndoe, A., Ghaem‐Maghami, S., Lees, C. C., Brölmann, H. A. M., et al. (2011) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.10077 · Cited by 189 · see citing works
Three-dimensional ultrasound evaluation of the placenta
Hata, T., Tanaka, H., Noguchi, J., & Hata, K. (2010) · Placenta · doi:10.1016/j.placenta.2010.11.001 · Cited by 115 · see citing works
Neto, M. A. C., Roncato, P., Nastri, C. O., & Martins, W. P. (2014) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.14654 · Cited by 58 · see citing works
Stevenson, G. N., Collins, S. L., Ding, J., Impey, L., & Noble, J. A. (2015) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2015.07.021 · Cited by 46 · see citing works
Reus, A. D., El-Harbachi, H., Rousian, M., Willemsen, S. P., Steegers-Theunissen, R. P. M., Steegers, E. A. P., & Exalto, N. (2013) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.13197 · Cited by 46 · see citing works
Collins, S. L., Stevenson, G. N., Noble, J. A., & Impey, L. (2013) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2012.09.003 · Cited by 39 · see citing works
Choux, C., Ginod, P., Barberet, J., Rousseau, T., Bruno, C., Sagot, P., Astruc, K., & Fauque, P. (2018) · Reproductive BioMedicine Online · doi:10.1016/j.rbmo.2018.12.023 · Cited by 37 · see citing works
Role of 3-D Ultrasound in Clinical Obstetric Practice: Evolution Over 20 Years
Tonni, G., Martins, W. P., Filho, H. G., & Júnior, E. A. (2015) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2014.12.009 · Cited by 36 · see citing works
León, R. L., Li, K. T., & Brown, B. P. (2018) · Pediatric Radiology · doi:10.1007/s00247-018-4213-x · Cited by 36 · see citing works
Volume and vascularity: Using ultrasound to unlock the secrets of the first trimester placenta
Mathewlynn, S., & Collins, S. L. (2019) · Placenta · doi:10.1016/j.placenta.2019.06.379 · Cited by 30 · see citing works
Martins, W. P., Raine‐Fenning, N. J., Leite, S. P., Ferriani, R. A., & Nastri, C. O. (2011) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.9016 · Cited by 29 · see citing works
Third trimester placental volume and biometry measurement: A method-development study
Higgins, L. E., Simcox, L. E., Sibley, C. P., Heazell, A. E. P., & Johnstone, E. (2016) · Placenta · doi:10.1016/j.placenta.2016.04.010 · Cited by 28 · see citing works
Milligan, N., Rowden, M., Wright, E., Melamed, N., Lee, Y. M., Windrim, R. C., & Kingdom, J. C. (2014) · The Journal of Maternal-Fetal & Neonatal Medicine · doi:10.3109/14767058.2014.963047 · Cited by 17 · see citing works
You, J., Lv, G., Liu, X., & He, S. (2014) · Prenatal Diagnosis · doi:10.1002/pd.4451 · Cited by 16 · see citing works
Effect of malaria on placental volume measured using three-dimensional ultrasound: a pilot study
Rijken, M. J., Moroski, W. E., Kiricharoen, S., Karunkonkowit, N., Stevenson, G. N., Ohuma, E. O., Noble, J. A., Kennedy, S., McGready, R. M., Papageorghiou, A. T., et al. (2012) · Malaria Journal · doi:10.1186/1475-2875-11-5 · Cited by 16 · see citing works
Júnior, E. A., Cavalcante, R. O., Nardozza, L. M. M., Rôlo, L. C., Ruano, R., Martins, W. P., & Moron, A. F. (2011) · Prenatal Diagnosis · doi:10.1002/pd.2870 · Cited by 15 · see citing works
Barreto, E. Q., Milani, H. J. F., Haratz, K. K., Júnior, E. A., Nardozza, L. M. M., & Moron, A. F. (2012) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2012.31.5.673 · Cited by 13 · see citing works
Multiscale and multimodal imaging of utero-placental anatomy and function in pregnancy
Srinivasan, V., Melbourne, A., Oyston, C., James, J. L., & Clark, A. R. (2021) · Placenta · doi:10.1016/j.placenta.2021.07.290 · Cited by 13 · see citing works
Barra, D. D. A., Lima, J. C., Filho, F. M., Júnior, E. A., & Martins, W. P. (2013) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2013.02.003 · Cited by 12 · see citing works
Simcox, L. E., Higgins, L. E., Myers, J., & Johnstone, E. (2017) · Journal of Ultrasound in Medicine · doi:10.7863/ultra.16.03045 · Cited by 11 · see citing works
Becsek, A., Tzanidakis, N., Blanco, M., & Bollwein, H. (2018) · Theriogenology · doi:10.1016/j.theriogenology.2018.11.012 · Cited by 10 · see citing works
Hanchard, T. J., de Vries, B. S., Quinton, A. E., Sinosich, M. J., & Hyett, J. A. (2020) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.21962 · Cited by 11 · see citing works
Bózsa, S., Pótó, L., Bódis, J., Halvax, L., Koppán, M., Arany, A., Csermely, T., & Vizer, M. G. (2011) · Ultrasound in Medicine & Biology · doi:10.1016/j.ultrasmedbio.2011.01.002 · Cited by 11 · see citing works
Intrastomal 3D ultrasound; an inter- and intra-observer evaluation
Strigård, K., Gurmu, A. G., Näsvall, P., Påhlman, P., & Gunnarsson, U. (2012) · International Journal of Colorectal Disease · doi:10.1007/s00384-012-1526-3 · Cited by 10 · see citing works
Comparing reliability between different ultrasound techniques
Martins, W. P. (2012) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.11138 · Cited by 9 · see citing works
Cavalcante, R. O., Júnior, E. A., Nardozza, L. M. M., Rôlo, L. C., & Moron, A. F. (2011) · Journal of Perinatal Medicine · doi:10.1515/jpm.2011.083 · Cited by 6 · see citing works
Biophysical markers for abnormal placentation: first and/or second trimester
Farina, A. (2014) · Prenatal Diagnosis · doi:10.1002/pd.4377 · Cited by 6 · see citing works
Trilla, C., Platero, J., Móra, J., Nan, M. N., Medina, C., Alejos, O., Parra, J., & Llurba, E. (2025) · Ultrasound in Obstetrics and Gynecology · doi:10.1002/uog.70095 · Cited by 5 · see citing works
Three-dimensional Evaluation of the Placenta: Review of the Literature
Antsaklis, A. J., & Natsis, S. (2013) · Donald School Journal of Ultrasound in Obstetrics & Gynecology · doi:10.5005/jp-journals-10009-1272 · Cited by 5 · see citing works
Small estimated placental volume (EPV) in the setting of decreased fetal movement
Murdaugh, K. M., & Florescue, H. (2023) · Clinical Imaging · doi:10.1016/j.clinimag.2023.110027 · Cited by 3 · see citing works
Placental Volume Measurement in Clinical Practice
Hasegawa, J., Arakaki, T., Nakamura, M., Takita, H., & Sekizawa, A. (2015) · Donald School Journal of Ultrasound in Obstetrics & Gynecology · doi:10.5005/jp-journals-10009-1427 · Cited by 3 · see citing works
(2015) · PubMed Central · Cited by 3 · see citing works
Trilla, C., Parra, J., & Llurba, E. (2024) · Diagnostics · doi:10.3390/diagnostics14141556 · Cited by 2 · see citing works
Segmentation of the placenta and its vascular tree in Doppler ultrasound for fetal surgery planning
Perera-Bel, E., Ceresa, M., Torrents‐Barrena, J., Masoller, N., Valenzuela‐Alcaráz, B., Gratacós, E., Eixarch, E., & Ballester, M. Á. G. (2020) · International Journal of Computer Assisted Radiology and Surgery · doi:10.1007/s11548-020-02256-2 · Cited by 2 · see citing works
Hernandez‐Andrade, E. A., Gerulewicz, D., Refuerzo, J., Tsao, K., Lopez, S. M., Swinford, R. D., Portella, P. P., Bergh, E. P., Khan, A., Papanna, R. M., et al. (2025) · Prenatal Diagnosis · doi:10.1002/pd.6839 · Cited by 1 · see citing works
Zambella, E., Inversetti, A., Pivato, C. A., Cristodoro, M., Stefanini, G. G., & Simone, N. D. (2026) · European Heart Journal - Quality of Care and Clinical Outcomes · doi:10.1093/ehjqcco/qcag020 · Cited by 1 · see citing works
Leung, K. Y., Ma, T., Lau, B., & Chen, M. (2012) · Expert Review of Obstetrics & Gynecology · doi:10.1586/eog.12.37 · Cited by 1 · see citing works
Milon, H., & Baleydier, A. (1997) · PubMed · Cited by 1 · see citing works
Dückelmann, A.-M., Mentzel, H., Kalache, K. D., & Schlembach, D. (2018) · Springer eBooks · doi:10.1007/978-3-662-55622-1_9 · Cited by 1 · see citing works
Repeatability of volume calculations of the fetal urinary bladder
Sætherhaug, T., Fasting, M., Røset, M. A., Naylor, M., & Blaas, H.-G. K. (2016) · Bladder · doi:10.14440/bladder.2016.75 · Cited by 1 · see citing works
Hanchard, T. J., de Vries, B. S., Quinton, A. E., Sinosich, M. J., & Hyett, J. A. (2020) · Australasian Journal of Ultrasound in Medicine · doi:10.1002/ajum.12224 · Cited by 1 · see citing works
Bergwerff, J., Schreurs, A. M. F., Lier, M. C. I., van Waesberghe, J. H. T. M., van der Houwen, L. E. E., & Mijatovic, V. (2022) · Journal of Endometriosis and Pelvic Pain Disorders · doi:10.1177/22840265221084928 · Cited by 1 · see citing works
Alzyoud, K. S. (2019) · University of Salford Institutional Repository (University of Salford) · doi:10.17866/z93q-7f12 · Cited by 1 · see citing works
The application of first-trimester volumetry in predicting pregnancy complications
Okeke, T. C., Agwuna, K. K., & Agu, P. U. (2014) · Journal of Basic and Clinical Reproductive Sciences · doi:10.4103/2278-960x.129272
Dückelmann, A.-M., Mentzel, H., Kalache, K. D., & Schlembach, D. (2023) · Springer eBooks · doi:10.1007/978-3-662-66256-4_9
- Authors
- Cheong, K. B., Leung, K. Y., Li, T. K. T., Chan, H. Y., Lee, Y. P., & Tang, M. H. Y.
- Affiliations
- Queen Mary Hospital (CN)
- Tsan Yuk Hospital (CN)
- Corresponding author
- Leung, K. Y.
- Venue
- Ultrasound in Obstetrics & Gynecology
- Publisher
- Wiley
- Published
- 2010-07-28
- Volume
- 36
- Issue
- 2
- Pages
- 210-217
- ISSN
- 0960-7692 (ISSN-L)
- 1469-0705
- OpenAlex
- W2007900078
- Cited by
- 45
- References
- 31
- Citations by year
- 2026: 1 · 2025: 2 · 2024: 1 · 2023: 2 · 2022: 1 · 2021: 1 · 2020: 3 · 2019: 2 · 2018: 4 · 2017: 1 · 2016: 2 · 2015: 4 · 2014: 5 · 2013: 3 · 2012: 6
- FWCI
- 3.69
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Ultrasound in Obstetrics and Gynecology (publishedVersion)
- PubMed (publishedVersion)
- The HKU Scholars Hub (University of Hong Kong) (submittedVersion)
- Topics
- Health Sciences › Medicine › Critical Care and Intensive Care Medicine › Ultrasound in Clinical Applications
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Prenatal Screening and Diagnostics
- Health Sciences › Medicine › Obstetrics and Gynecology › Pregnancy and preeclampsia studies
- Keywords
- placental volume, VOCAL, multiplanar reconstruction, interobserver agreement, intra-rater reliability, phantom validation, early pregnancy
- MeSH terms
- Female, Gestational Age, Humans, Image Processing, Computer-Assisted, Placenta, Pregnancy, Reproducibility of Results, Observer Variation, Ultrasonography, Prenatal, Phantoms, Imaging, Imaging, Three-Dimensional, Fetal Development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review