The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Depression recognition based on dynamic facial and vocal expression features using partial least square regression
Abstract
Depression is a typical mood disorder, and the persons who are often in this state face the risk in mental and even physical problems. In recent years, there has therefore been increasing attention in machine based depression analysis. In such a low mood, both the facial expression and voice of human beings appear different from the ones in normal states. This paper presents a novel method, which comprehensively models visual and vocal modalities, and automatically predicts the scale of depression. On one hand, Motion History Histogram (MHH) extracts the dynamics from corresponding video and audio data to represent characteristics of subtle changes in facial and vocal expression of depression. On the other hand, for each modality, the Partial Least Square (PLS) regression algorithm is applied to learn the relationship between the dynamic features and depression scales using training data, and then predict the depression scale for an unseen one. Predicted values of visual and vocal clues are further combined at decision level for final decision. The proposed approach is evaluated on the AVEC2013 dataset and experimental results clearly highlight its effectiveness and better performance than baseline results provided by the AVEC2013 challenge organiser.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Citations
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikäinen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,592
Computer Vision: A Modern Approach
Forsyth, D. A., & Ponce, J. (2002) · Cited by 3,728
The recognition of human movement using temporal templates
Bobick, A., & Davis, J. W. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.910878 · Cited by 2,837
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300
Survey on speech emotion recognition: Features, classification schemes, and databases
Ayadi, M. M. H. E., Kamel, M. S., & Karray, F. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,139
SIMPLS: An alternative approach to partial least squares regression
de Jong, S. (1993) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(93)85002-x · Cited by 1,905
Scale for the Assessment of Negative Symptoms
Andreasen, N. C. (1982) · PsycTESTS Dataset · doi:10.1037/t12696-000 · Cited by 1,888
Research works printed but not linked: 6
- 10.5555/645312.648916 · DOI
- W. T. Freeman , W. T. Freeman , M. Roth , and M. Roth . Orientation histograms for hand gesture recognition . In IEEE International Workshop on Automatic Face and Gesture Recognition , pages 296 -- 301 , 1994 . W. T. Freeman, W. T. Freeman, M. Roth, and M. Roth. Orientation histograms for hand gesture recognition. In IEEE International Workshop on Automatic Face and Gesture Recognition, pages 296--301, 1994.
- Health & Consumer Protection Directorate General . Improving the mental health of the population: Towards a strategy on mental health for the european union , 2005 . Health & Consumer Protection Directorate General. Improving the mental health of the population: Towards a strategy on mental health for the european union, 2005.
- Health & Consumer Protection Directorate General . Mental health in the EU , 2008 . Health & Consumer Protection Directorate General. Mental health in the EU, 2008.
- S. Salmans . Depression: questions you have - answers you need . People's Medical Society , 1995 . S. Salmans. Depression: questions you have - answers you need. People's Medical Society, 1995.
- 10.5555/520809.796139 · DOI
Cited by
Sorted by citations
Items 201–223 of 223
Specific Designed Facial Expression Recognition System for Interactive Film Applications
Qin, R., Liu, J., Meng, H., & Chen, T. (2021) · Lecture notes on data engineering and communications technologies · doi:10.1007/978-3-030-70665-4_70
Self-citationShinde, P. G., & Kulkarni, R. S. (2021) · International Journal of Recent Technology and Engineering (IJRTE) · doi:10.35940/ijrte.b6402.0910321
Data-Driven Facial Expression Analysis from Live Video
Tay, W. K. (2017) · doi:10.26686/wgtn.17064395
Automated audiovisual depression analysisPreprint
Girard, J. M., & Cohn, J. F. (2018) · doi:10.31234/osf.io/zaum3
Automatic Depression Recognition Using Multi-scale Facial Behavior Dynamics
Zhu, Y. (2022) · International Conference on Information Communication and Software Engineering (ICICSE) · doi:10.1109/icicse55337.2022.9828962
Tackling Depression Detection With Deep Learning
Krishna, N. B., Reddy, R. S. V., Likhith, M., & Priya, N. L. (2024) · Advances in medical diagnosis, treatment, and care (AMDTC) book series · doi:10.4018/979-8-3693-3679-3.ch006
Tackling Depression Detection with deep learning a Hybrid Model
BalaKrishna, N., Reddy, R. S. V., Geethika, S. N. S., Likhith, M., Priya, N. L., & Harsha, S. H. (2024) · International Conference on Knowledge Engineering and Communication Systems (ICKECS) · doi:10.1109/ickecs61492.2024.10617226
Enhanced Multimodal Integration Using TriFusion Networks for Comprehensive Emotion AnalysisPreprint
Wilson, E., Patel, R., Taylor, A., & Jones, L. (2024) · Preprints.org · doi:10.20944/preprints202408.0576.v1
Cross-Cultural Automatic Depression Detection Based on Audio Signals
Mamontov, D., Zepf, S., Karpov, A. A., & Minker, W. (2024) · Lecture notes in computer science · doi:10.1007/978-3-031-77961-9_23
Attention-Based Audio Depression Recognition Integrating Handcrafted and Deep Features
Jin, C., Zhao, S., Zhang, S., Fang, Z., Xie, J., & Chen, Y. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-5084-2_14
Channel-Dcoupled and Local-Attention Network for Depression Detection
Chen, Y., Huang, Z. Y., Liu, J., Hu, M. Q., & Ren, F. (2024) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait63902.2024.11021842
“The hand-crafted-based methods manually design feature descriptors to extract the representation of depression [17,18,19].”
A Graph Attention Representation for Facial Expression Recognition
Lu, J., Shang, Y., Shao, Z., Liu, T., Ding, H., Liu, J., & Wang, X. (2025) · Communications in computer and information science · doi:10.1007/978-981-96-6951-6_3
Deep spectrotemporal network based depression severity estimation from speech
Jabbar, I., Uddin, M. A., Joolee, J. B., & Nasridinov, A. (2025) · Scientific Reports · doi:10.1038/s41598-025-29889-0
“Meng et al. 27 utilized the Motion History Histogram (MHH) method to extract dynamic vocal features, subsequently employing Partial Least Squares (PLS) regression to predict depression levels effectively from audio signals.”
Depression level assessment based on multi-layer graph neural networks applied to facial videos
Du, S., Ren, A., Xu, H., & Qiu, Y. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.132415
PROMIND: Privacy-Protected Mental Health Intelligence with Noise Defense for Depression Recognition
Pan, Y., Shen, W., & Yao, H. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3661511
Reading Faces. Using Hard Multi-Task Metric Learning for Kernel Regression
Nicolle, J. (2016) · theses.fr (ABES) · doi:10.70675/8a22a6caz95d0z4cb8zaeadz043ba01a754d
“Name Year Database Winner(s) 300W [83] 2013 300W mix [118] (academia), [131] (industry) 300W [91] 2015 300W mix [119], [116] Emotiw [25] 2013 AFEW [47] Emotiw [24] 2014 AFEW [56] AVEC [88] 2011 SEMAINE [64] (audio), [79] (video) AVEC [89] 2012 SEMAINE [75] (fully continuous), [86] (word level) AVEC [102] 2013 AViD-Cor…”
Automatic detection of visual cues associated to depression
Pampouchidou, A. (2018) · theses.fr (ABES) · doi:10.70675/98d0e5cfz8530z4299z8ae3zb6e1b6331af1 · Cited by 4
Gradient Embedding and Modal Complementarity Networks for Audio-Video Depression Level Prediction
Niu, M., Zhang, Z., Gao, C., Zeng, Y., Fan, N., & Zhang, S. (2026) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2026.3681251
Liu, X., Li, H., Zhou, T., Yang, Z., Deng, T., & Xiao, H. (2025) · Asian Conference on Artificial Intelligence Technology (ACAIT) · doi:10.1109/acait67930.2025.11521945
“In the context of bimodal depression detection, Meng et al. [42] encoded low-level descriptor (LLD) audio features and combined them with motion history histograms (MHH) to analyze facial region motion, effectively integrating audio and visual features at the decision level to improve depression…”
Beyond Accuracy: A Review of Model and Data Trustworthiness in Audio-Visual Depression Recognition
Pan, Y., Yao, H., Shen, W., & He, L. (2026) · Communications in computer and information science · doi:10.1007/978-981-95-9493-1_3
Multi-Scale Non-Stationary Attention Learning for Video-Based Depression Severity Regression
Cao, Y., Li, X., Shi, Q., & Zhang, W. (2026) · International Conference on Advanced Computational Intelligence (ICACI) · doi:10.1109/icaci69731.2026.11677792
Cao, Y., Tang, B., Li, C., Dai, B., Ling, J., Li, Y., & Liu, X. (2026) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2026.111625
Metadata
- Authors
- Meng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., & Wang, Y.
- Affiliations
- Brunel University of London (GB)
- Beihang University (CN)
- Venue
- Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
- Publisher
- ACM
- Published
- 2013-10-21
- Pages
- 21-30
- OpenAlex
- W1988088279
- Semantic Scholar
- e638499e3649fb69142208c33abf63093c96b00c
- Cited by
- 225
- References
- 47
- Citations by year
- 2026: 7 · 2025: 24 · 2024: 24 · 2023: 21 · 2022: 23 · 2021: 18 · 2020: 18 · 2019: 21 · 2018: 14 · 2017: 16 · 2016: 10 · 2015: 18 · 2014: 11
- FWCI
- 17.63
- Type
- conference paper
- Language
- en
- Open access
- closed
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- depression detection, facial expressions, vocal expression, multimodal fusion, partial least squares regression, depression severity prediction, decision fusion, affective computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
