Vision-Based Mouth Motion Analysis in Epilepsy: A 3D Perspective
Open accessAbstract
Epilepsy monitoring involves the study of videos to assess clinical signs (semiology) to assist with the diagnosis of seizures. Recent advances in the application of vision-based approaches to epilepsy analysis have demonstrated significant potential to automate this assessment. Nevertheless, current proposed computer vision based techniques are unable to accurately quantify specific facial modifications, e.g. mouth motions, which are examined by neurologists to distinguish between seizure types. 2D approaches that analyse facial landmarks have been proposed to quantify mouth motions, however, they are unable to fully represent motions in the mouth and cheeks (ictal pouting) due to a lack of landmarks in the the cheek regions. Additionally, 2D region-based techniques based on the detection of the mouth have limitations when dealing with large pose variations, and thus make a fair comparison between samples difficult due to the variety of poses present. 3D approaches, on the other hand, retain rich information about the shape and appearance of faces, simplifying alignment for comparison between sequences. In this paper, we propose a novel network method based on a 3D reconstruction of the face and deep learning to detect and quantify mouth semiology in our video dataset of 20 seizures, recorded from patients with mesial temporal and extra-temporal lobe epilepsy. The proposed network is capable of distinguishing between seizures of both types of epilepsy. An average classification accuracy of 89% demonstrates the benefits of computer vision and deep learning for clinical applications of non-contact systems to identify semiology commonly encountered in a natural clinical setting.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Ahmedt-Aristizabal [159] achieved an average classification accuracy of 89% for differentiating two types of epileptic seizures.
“The developed network demonstrated its capability to differentiate between two types of epileptic seizures, achieving an average classification accuracy of 89%.”
From A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis (2024) · cites this paper as [159]
Cites
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318 · see citing works
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.322 · Cited by 29,137 · see citing works
Rapid object detection using a boosted cascade of simple features
Viola, P., & Jones, M. (2005) · Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001 · doi:10.1109/cvpr.2001.990517 · Cited by 18,300 · see citing works
Aggregated Residual Transformations for Deep Neural Networks
Xie, S., Girshick, R., Dollar, P., Tu, Z., & He, K. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.634 · Cited by 12,465 · see citing works
Greff, K., Srivastava, R. K., Koutnik, J., Steunebrink, B. R., & Schmidhuber, J. (2017) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2016.2582924 · Cited by 6,955 · see citing works
A morphable model for the synthesis of 3D faces
Blanz, V., & Vetter, T. R. (1999) · ACM SIGGRAPH Conference (SIGGRAPH) · doi:10.1145/311535.311556 · Cited by 4,695 · see citing works
Bulat, A., & Tzimiropoulos, G. (2017) · 2017 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.116 · Cited by 1,464 · see citing works
A 3D Face Model for Pose and Illumination Invariant Face Recognition
Paysan, P., Knothe, R., Amberg, B., Romdhani, S., & Vetter, T. R. (2009) · IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) · doi:10.1109/avss.2009.58 · Cited by 1,325 · see citing works
Bulat, A., & Tzimiropoulos, G. (2017) · Nottingham ePrints (University of Nottingham) · Cited by 1,277 · see citing works
Face Alignment Across Large Poses: A 3D Solution
Zhu, X., Lei, Z., Liu, X., Shi, H., & Li, S. Z. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.23 · Cited by 1,137 · see citing works
Joint 3D Face Reconstruction and Dense Alignment with Position Map Regression Network
Feng, Y., Wu, F., Shao, X., Wang, Y., & Zhou, X. (2018) · Lecture notes in computer science · doi:10.1007/978-3-030-01264-9_33 · Cited by 768 · see citing works
Hu, P., & Ramanan, D. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.166 · Cited by 709 · see citing works
Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network
Tran, A. T., Hassner, T., Masi, I., & Medioni, G. G. (2017) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2017.163 · Cited by 503 · see citing works
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
Tewari, A., Zollhöfer, M., Kim, H., Garrido, P., Bernard, F., Pérez, P., & Theobalt, C. (2017) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2017.401 · Cited by 398 · see citing works
Large-Pose Face Alignment via CNN-Based Dense 3D Model Fitting
Jourabloo, A., & Liu, X. (2016) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2016.454 · Cited by 302 · see citing works
Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification
Rodriguez, P., Cucurull, G., Gonzalez, J., Gonfaus, J. M., Nasrollahi, K., Moeslund, T. B., & Roca, F. X. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2662199 · Cited by 297 · see citing works
A Multiresolution 3D Morphable Face Model and Fitting Framework
Huber, P., Hu, G., Tena, R., Mortazavian, P., Koppen, W. P., Christmas, W. J., Rätsch, M., & Kittler, J. (2016) · International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP) · doi:10.5220/0005669500790086 · Cited by 227 · see citing works
Extreme 3D Face Reconstruction: Seeing Through Occlusions
Tran, A. T., Hassner, T., Masi, I., Paz, E., Nirkin, Y., & Medioni, G. G. (2018) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2018.00414 · Cited by 183 · see citing works
Dense 3D face alignment from 2D videos in real-time
Jeni, L. A., Cohn, J. F., & Takeo Kanade, A. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163142 · Cited by 179 · see citing works
A Survey on Computer Vision for Assistive Medical Diagnosis From Faces
Thevenot, J., López, M. B., & Hadid, A. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2754861 · Cited by 166 · see citing works
Semiology of epileptic seizures: A critical review
Noachtar, S., & Peters, A. S. (2009) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2009.02.029 · Cited by 145 · see citing works
Vision-based patient monitoring: a comprehensive review of algorithms and technologies
Sathyanarayana, S., Satzoda, R. K., Sathyanarayana, S., & Srikanthan, T. (2015) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-015-0328-1 · Cited by 121 · see citing works
3D-Aided Face Recognition Robust to Expression and Pose Variations
Chu, B., Romdhani, S., & Chen, L. (2014) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2014.245 · Cited by 84 · see citing works
Deep facial analysis: A new phase I epilepsy evaluation using computer vision
Ahmedt‐Aristizabal, D., Fookes, C., Nguyen, K., Denman, S., Sridharan, S., & Dionisio, S. A. (2018) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2018.02.010 · Cited by 68 · see citing works
Self-citationAhmedt‐Aristizabal, D., Fookes, C., Dionisio, S. A., Nguyen, K., Cunha, J. P. S., & Sridharan, S. (2017) · Epilepsia · doi:10.1111/epi.13907 · Cited by 67 · see citing works
Self-citationNeural network underlying ictal pouting (“chapeau de gendarme”) in frontal lobe epilepsy
Souirti, Z., Landré, E., Mellerio, C., Devaux, B. C., & Chassoux, F. (2014) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2014.07.009 · Cited by 59 · see citing works
Insulo‐opercular cortex generates oroalimentary automatisms in temporal seizures
Aupy, J., Noviawaty, I., Krishnan, B., et al. (2018) · Epilepsia · doi:10.1111/epi.14011 · Cited by 48 · see citing works
A hierarchical multimodal system for motion analysis in patients with epilepsy
Ahmedt‐Aristizabal, D., Fookes, C., Denman, S., et al. (2018) · Epilepsy & Behavior · doi:10.1016/j.yebeh.2018.07.028 · Cited by 42 · see citing works
Self-citationGuo, Z., Shen, M., Duan, L., Zhou, Y., Xiang, J., Ding, H., Chen, S., Deußen, O., & Guo, D. (2017) · IEEE International Symposium on Biomedical Imaging (ISBI) · doi:10.1109/isbi.2017.7950486 · Cited by 38 · see citing works
Vision-based absence seizure detection
Pediaditis, M., Tsiknakis, M., Koumakis, L., et al. (2012) · doi:10.1109/embc.2012.6345872 · Cited by 16 · see citing works
A survey on face modeling: building a bridge between face analysis and synthesis
Salam, H., & Séguier, R. (2016) · The Visual Computer · doi:10.1007/s00371-016-1332-y · Cited by 16 · see citing works
3D model fitting for facial expression analysis under uncontrolled imaging conditions
Maurel, P., McGonigal, A., Keriven, R., & Chauvel, P. (2008) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2008.4761736 · Cited by 14 · see citing works
Pose Normalization for Eye Gaze Estimation and Facial Attribute Description from Still Images
Egger, B., Schönborn, S., Forster, A., et al. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-11752-2_25 · Cited by 12 · see citing works
Model-free vision-based facial motion analysis in epilepsy
Pediaditis, M., Tsiknakis, M., Bologna, V., et al. (2011) · doi:10.1109/iwbe.2011.6079051 · Cited by 9 · see citing works
A Feedback Estimation Approach for Therapeutic Facial Training
Dittmar, C., Denzler, J., & Groß, H. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.26 · Cited by 7 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A Comprehensive Review on Synergy of Multi-Modal Data and AI Technologies in Medical Diagnosis
Xu, X., Li, J., Zhu, Z., Zhao, L., Wang, H., Song, C., Chen, Y., Zhao, Q., Yang, J., & Pei, Y. (2024) · Bioengineering · doi:10.3390/bioengineering11030219 · Cited by 242 · see citing works
Karacsony, T., Loesch-Biffar, A. M., Vollmar, C., Noachtar, S., & Cunha, J. P. S. (2021) · 2021 IEEE EMBS International Conference on Biomedical and Health Informatics (BHI) · doi:10.1109/bhi50953.2021.9508555 · Cited by 14 · see citing works
- Authors
- Ahmedt-Aristizabal, D., Nguyen, K., Denman, S., Sarfraz, M. S., Sridharan, S., Dionisio, S., & Fookes, C.
- Affiliations
- Queensland University of Technology (AU)
- Karlsruhe Institute of Technology (DE)
- Mater Health Services (AU)
- Venue
- 2019 41st Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC)
- Publisher
- IEEE
- Published
- 2019-07
- Volume
- 2019
- Pages
- 1625-1629
- ISSN
- 2375-7477 (ISSN-L)
- OpenAlex
- W2979833082
- Cited by
- 17
- References
- 36
- Citations by year
- 2026: 1 · 2025: 3 · 2024: 5 · 2022: 3 · 2021: 3 · 2020: 1 · 2019: 1
- FWCI
- 0.65
- Type
- conference paper
- Language
- en
- Open access
- green
- eprints.qut.edu.au
- Licence
- Locations
- Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (publishedVersion)
- PubMed (publishedVersion)
- QUT ePrints (Queensland University of Technology) (submittedVersion, cc-by-nc)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Health Sciences › Medicine › Psychiatry and Mental health › Epilepsy research and treatment
- Life Sciences › Neuroscience › Cognitive Neuroscience › EEG and Brain-Computer Interfaces
- Keywords
- epilepsy, seizure semiology, 3D face reconstruction, seizure type classification, mesial temporal lobe epilepsy, extratemporal epilepsy
- MeSH terms
- Electroencephalography, Epilepsy, Face, Humans, Mouth, Seizures
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
