Recognition of facial expression of fetuses by artificial intelligence (AI)
Open accessAbstract
OBJECTIVES: The development of the artificial intelligence (AI) classifier to recognize fetal facial expressions that are considered as being related to the brain development of fetuses as a retrospective, non-interventional pilot study. METHODS: Images of fetal faces with sonography obtained from outpatient pregnant women with a singleton fetus were enrolled in routine conventional practice from 19 to 38 weeks of gestation from January 1, 2020, to September 30, 2020, with completely de-identified data. The images were classified into seven categories, such as eye blinking, mouthing, face without any expression, scowling, smiling, tongue expulsion, and yawning. The category in which the number of fetuses was less than 10 was eliminated before preparation. Next, we created a deep learning AI classifier with the data. Statistical values such as accuracy for the test dataset and the AI confidence score profiles for each category per image for all data were obtained. RESULTS: The number of fetuses/images in the rated categories were 14/147, 23/302, 33/320, 8/55, and 10/72 for eye blinking, mouthing, face without any expression, scowling, and yawning, respectively. The accuracy of the AI fetal facial expression for the entire test data set was 0.985. The accuracy/sensitivity/specificity values were 0.996/0.993/1.000, 0.992/0.986/1.000, 0.985/1.000/0.979, 0.996/0.888/1.000, and 1.000/1.000/1.000 for the eye blinking, mouthing, face without any expression, scowling categories, and yawning, respectively. CONCLUSIONS: The AI classifier has the potential to objectively classify fetal facial expressions. AI can advance fetal brain development research using ultrasound.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Three studies assessed the ability of deep learning models to undergo self-supervised learning by correcting the order of a reshuffled fetal video clip, by context restoration of unlabeled 2D fetal images and the use of random forests classifiers for classification of unlabeled fetal ultrasound images.
“Three studies assessed the ability of deep learning models to undergo self-supervised learning by correcting the order of a reshuffled fetal video clip, 115 by context restoration of unlabeled 2D fetal images 116 and the use of random forests classifiers for classification of unlabeled fetal ultrasound images.117”
From Artificial intelligence in obstetric ultrasound: A scoping review. (2023) · cites this paper as [116]
Citation 117 is attached to the random-forest classification of unlabeled fetal ultrasound images. The AI classifier section (page unavailable) instead describes a convolutional neural network trained on fetal-face images assigned to expression categories. The Discussion also explicitly says the model depends on supervised data, contrary to the claimed self-supervised use of unlabeled images. (LLM, conf=0.88)
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredMiyagi et al. proposed an AI classifier analyzing 4D US volumes to assess fetal facial expressions and classify them into different categories.
“Miyagi et al. proposed an AI classifier analyzing 4D US volumes to assess fetal facial expressions and classify them into different categories [102,103]”
From Evolving the Era of 5D Ultrasound? A Systematic Literature Review on the Applications for Artificial Intelligence Ultrasound Imaging in Obstetrics and Gynecology (2023) · cites this paper as [103]
The Patients and data preparation section says the fetal-face images came from four-dimensional ultrasound, and the AI classifier section describes a model that assigns each image to a facial-expression category. But the paper says it captured and analyzed still images, not 4D ultrasound volumes. (LLM, conf=0.88)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155 · see citing works
Gradient-based learning applied to document recognition
Lecun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998) · Proceedings of the IEEE · doi:10.1109/5.726791 · Cited by 58,805 · see citing works
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562 · see citing works
(2014) · Cited by 33,419 · see citing works
Human-level control through deep reinforcement learning
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. (2015) · Nature · doi:10.1038/nature14236 · Cited by 30,357 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Ioffe, S., & Szegedy, C. (2015) · arXiv · doi:10.48550/arxiv.1502.03167 · Cited by 27,818 · see citing works
Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate ShiftPreprint
Ioffe, S. (2024) · arXiv (Cornell University) · doi:10.57702/o9raffed · Cited by 15,273 · see citing works
(2010) · International Conference on Machine Learning · Cited by 13,995 · see citing works
Representation Learning: A Review and New Perspectives
Bengio, Y., Courville, A., & Vincent, P. (2013) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.50 · Cited by 13,001 · see citing works
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891 · see citing works
Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation
Powers, D. M. W. (2020) · arXiv · doi:10.48550/arxiv.2010.16061 · Cited by 5,440 · see citing works
Neural Networks: Tricks of the Trade: Second Edition
(2012) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-35289-8 · Cited by 2,116 · see citing works
Robust Object Recognition with Cortex-Like Mechanisms
Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., & Poggio, T. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.56 · Cited by 1,667 · see citing works
Evaluation of Pooling Operations in Convolutional Architectures for Object Recognition
Scherer, D., Müller, A., & Behnke, S. (2010) · Lecture notes in computer science · doi:10.1007/978-3-642-15825-4_10 · Cited by 1,556 · see citing works
Bridle, J. S. (1990) · Springer eBooks · doi:10.1007/978-3-642-76153-9_28 · Cited by 1,278 · see citing works
LeCun, Y., Bottou, L., Orr, G., & Müller, K. (2012) · Lecture notes in computer science · doi:10.1007/978-3-642-35289-8_3 · Cited by 1,268 · see citing works
(2011) · doi:10.5591/978-1-57735-516-8/ijcai11-210 · Cited by 1,229 · see citing works
The emergence of fetal behaviour. I. Qualitative aspects
de Vries, J. I. P., Visser, G. H. A., & Prechtl, H. F. R. (1982) · Early Human Development · doi:10.1016/0378-3782(82)90033-0 · Cited by 896 · see citing works
Time Series Classification Using Multi-Channels Deep Convolutional Neural Networks
Zheng, Y., Liu, Q., Chen, E., Ge, Y., & Zhao, J. L. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-08010-9_33 · Cited by 689 · see citing works
Prechtl, H. F. R. (1990) · Early Human Development · doi:10.1016/0378-3782(90)90011-7 · Cited by 620 · see citing works
Simplifying Neural Networks by Soft Weight-Sharing
Nowlan, S. J., & Hinton, G. E. (1992) · Neural Computation · doi:10.1162/neco.1992.4.4.473 · Cited by 595 · see citing works
A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction
Wiatowski, T., & Bölcskei, H. (2017) · IEEE Transactions on Information Theory · doi:10.1109/tit.2017.2776228 · Cited by 411 · see citing works
The emergence of fetal behaviour. II. Quantitative aspects
de Vries, J. I. P., Visser, G. H. A., & Prechtl, H. F. R. (1985) · Early Human Development · doi:10.1016/0378-3782(85)90174-4 · Cited by 329 · see citing works
Large-scale Learning with SVM and Convolutional for Generic Object Categorization
Huang, F. J., & LeCun, Y. (2006) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2006.164 · Cited by 320 · see citing works
Kim, J., Kim, B., Roy, P. P., & Jeong, D.-M. (2019) · IEEE Access · doi:10.1109/access.2019.2907327 · Cited by 309 · see citing works
Prechtl, H. F. R. (1997) · Early Human Development · doi:10.1016/s0378-3782(97)00088-1 · Cited by 246 · see citing works
New scoring system for fetal neurobehavior assessed by three- and four-dimensional sonography
Kurjak, A., Mišković, B., Stanojević, M., Amiel‐Tison, C., Ahmed, B. I., Azumendi, G., Vasilj, O., Andonotopo, W., Turudic, T., & Salihagič-Kadić, A. (2008) · Journal of Perinatal Medicine · doi:10.1515/jpm.2008.007 · Cited by 186 · see citing works
Is neurological assessment of the fetus possible?
Prechtl, H. F. R., & Einspieler, C. (1997) · European Journal of Obstetrics & Gynecology and Reproductive Biology · doi:10.1016/s0301-2115(97)00197-8 · Cited by 95 · see citing works
Nijhuis, J. G. (2003) · Neurobiology of Aging · doi:10.1016/s0197-4580(03)00054-x · Cited by 92 · see citing works
Kuno, A., Akiyama, M., Yamashiro, C., Tanaka, H., Yanagihara, T., & Hata, T. (2001) · Journal of Ultrasound in Medicine · doi:10.7863/jum.2001.20.12.1271 · Cited by 85 · see citing works
Self-citationAn Attempt to Standardize Kurjak's Antenatal Neurodevelopmental Test: Osaka Consensus Statement
Predojević, M., Mišković, B., Ahmed, B. I., Vlădăreanu, R., Kadić, A. S., Shaddad, A. N., Al-Noobi, M., Talić, A., Stanojević, M., Lebit, D., et al. (2011) · Donald School Journal of Ultrasound in Obstetrics & Gynecology · doi:10.5005/jp-journals-10009-1209 · Cited by 78 · see citing works
Ultrasound for evaluation of fetal neurobehavioural development: from 2‐D to 4‐D ultrasound
Hata, T., Dai, S., & Marumo, G. (2009) · Infant and Child Development · doi:10.1002/icd.659 · Cited by 68 · see citing works
Self-citationConvolution neural network for automatic facial expression recognition
Chen, X., Yang, X., Wang, M., & Zou, J. (2017) · International Conference on Applied System Innovation · doi:10.1109/icasi.2017.7988558 · Cited by 61 · see citing works
Miyagi, Y., Habara, T., Hirata, R., & Hayashi, N. (2019) · Reproductive Medicine and Biology · doi:10.1002/rmb2.12267 · Cited by 60 · see citing works
Self-citationMiyagi, Y., Takehara, K., & Miyake, T. M. (2019) · Molecular and Clinical Oncology · doi:10.3892/mco.2019.1932 · Cited by 60 · see citing works
Self-citationMiyagi, Y., Habara, T., Hirata, R., & Hayashi, N. (2019) · Reproductive Medicine and Biology · doi:10.1002/rmb2.12266 · Cited by 59 · see citing works
Self-citationMiyagi, Y., Takehara, K., Nagayasu, Y., & Miyake, T. M. (2019) · Oncology Letters · doi:10.3892/ol.2019.11214 · Cited by 59 · see citing works
Self-citationMiyagi, Y., Habara, T., Hirata, R., & Hayashi, N. (2019) · Reproductive Medicine and Biology · doi:10.1002/rmb2.12284 · Cited by 36 · see citing works
Self-citationFetal face as important indicator of fetal brain function
AboEllail, M. A. M., & Hata, T. (2017) · Journal of Perinatal Medicine · doi:10.1515/jpm-2016-0377 · Cited by 33 · see citing works
Self-citationHu, M. Q., Yang, C., Zheng, Y., Wang, X., He, L., & Ren, F. (2019) · IEEE Access · doi:10.1109/access.2019.2936976 · Cited by 29 · see citing works
Reissland, N., Francis, B., Kumarendran, K., & Mason, J. (2015) · Acta Paediatrica · doi:10.1111/apa.13001 · Cited by 28 · see citing works
Current status of fetal neurodevelopmental assessment: Four‐dimensional ultrasound study
Hata, T. (2016) · Journal of obstetrics and gynaecology research · doi:10.1111/jog.13099 · Cited by 27 · see citing works
Self-citationMiyagi, Y., Habara, T., Hirata, R., & Hayashi, N. (2020) · Artificial Intelligence in Medical Imaging · doi:10.35711/aimi.v1.i3.94 · Cited by 23 · see citing works
Self-citationMiyagi, Y., Tada, K., Yasuhi, I., Maekawa, Y., Okura, N., Kawakami, K., Yamaguchi, K. D., Ogawa, M., Kodama, T. S., Nomiyama, M., et al. (2019) · Journal of obstetrics and gynaecology research · doi:10.1111/jog.14166 · Cited by 20 · see citing works
Self-citationA Survey on Facial Expression Recognition using Machine Learning Techniques
Dixit, A. N., & Kasbe, T. (2020) · doi:10.1109/idea49133.2020.9170706 · Cited by 19 · see citing works
Potential of Artificial Intelligence for Estimating Japanese Fetal Weights.
Miyagi, Y., & Miyake, T. M. (2020) · PubMed · doi:10.18926/amo/61207 · Cited by 17 · see citing works
Self-citationFetal Consciousness: Four-dimensional Ultrasound Study
(2015) · Donald School Journal of Ultrasound in Obstetrics & Gynecology · doi:10.5005/jp-journals-10009-1434 · Cited by 16 · see citing works
HDlive and 4D Ultrasound in the Assessment of Fetal Facial Expressions
(2015) · Donald School Journal of Ultrasound in Obstetrics & Gynecology · doi:10.5005/jp-journals-10009-1388 · Cited by 15 · see citing works
Kuno, A., Yanagihara, T., Hanaoka, U., et al. (2001) · Ultrasound in Obstetrics and Gynecology · doi:10.1046/j.1469-0705.2001.abs20-7.x · Cited by 7 · see citing works
Theory and Practice of Mathematics and Computer Science Vol. 2
Fried, I., Al-Dosary, K. I. T., Shenify, M., et al. (2020) · Book Publisher International (a part of SCIENCEDOMAIN International) · doi:10.9734/bpi/tpmcs/v2 · Cited by 2 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- Hata T, Kanenishi K, AboEllail MAM, Marumo G, Kurjak A. Fetal consciousness 4D ultrasound study. Donald Sch J Ultrasound Obstet Gynecol 2015;9:471-4.
- Miyagi Y, Fujiwara K, Oda T, Miyake T, Coleman RL. Development of new method for the prediction of clinical trial results using compressive sensing of artificial intelligence. J Biostat Biometric Appl 2018;3:202.
- Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics (AISTATS), Lauderdale, USA, 2011. AISTATS; 2011. pp. 315-23.
Cited by
Sorted by citations
Artificial intelligence in obstetric ultrasound: A scoping review.
Horgan, R., Nehme, L., & Abuhamad, A. (2023) · Prenatal Diagnosis · doi:10.1002/pd.6411 · Cited by 72 · see citing works
Jost, E., Kosian, P., Jimenez Cruz, J., Albarqouni, S., Gembruch, U., Strizek, B., & Recker, F. (2023) · Journal of Clinical Medicine · doi:10.3390/jcm12216833 · Cited by 44 · see citing works
Miyagi, Y., Hata, T., Bouno, S., Koyanagi, A., & Miyake, T. (2023) · International Journal of Gynecology & Obstetrics · doi:10.1002/ijgo.14569 · Cited by 15 · see citing works
Self-citationFetal brain activity and the free energy principle
Miyagi, Y., Hata, T., & Miyake, T. (2023) · Journal of Perinatal Medicine · doi:10.1515/jpm-2023-0092 · Cited by 6 · see citing works
Self-citation
- Authors
- Miyagi, Y., Hata, T., Bouno, S., Koyanagi, A., & Miyake, T.
- Affiliations
- Saitama International Medical Center (JP)
- Kagawa University (JP)
- Department of Obstetrics and Gynecology , Miyake Clinic , Okayama , Japan
- Department of Gynecology , Miyake Ofuku Clinic , Okayama , Japan
- Corresponding author
- Miyagi, Yasunari
- Venue
- Journal of Perinatal Medicine
- Publisher
- Walter de Gruyter GmbH
- Published
- 2021-02-08
- Volume
- 49
- Issue
- 5
- Pages
- 596-603
- ISSN
- 0300-5577 (ISSN-L)
- 1619-3997
- OpenAlex
- W3126365686
- Cited by
- 34
- References
- 51
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 8 · 2023: 8 · 2022: 7 · 2021: 2
- FWCI
- 1.85
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Journal of Perinatal Medicine (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Health Sciences › Medicine › Health Informatics › Artificial Intelligence in Healthcare and Education
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Health Sciences › Medicine › Pediatrics, Perinatology and Child Health › Fetal and Pediatric Neurological Disorders
- Keywords
- fetal facial expressions, fetal brain development, prenatal ultrasound, deep learning classifier, blinking, yawning, smiling, mouthing
- MeSH terms
- Artificial Intelligence, Brain, Face, Facial Expression, Female, Fetal Movement, Fetus, Humans, Pregnancy, Ultrasonography, Prenatal, Fetal Development
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
