Depression recognition based on dynamic facial and vocal expression features using partial least square regression
Abstract
Depression is a typical mood disorder, and the persons who are often in this state face the risk in mental and even physical problems. In recent years, there has therefore been increasing attention in machine based depression analysis. In such a low mood, both the facial expression and voice of human beings appear different from the ones in normal states. This paper presents a novel method, which comprehensively models visual and vocal modalities, and automatically predicts the scale of depression. On one hand, Motion History Histogram (MHH) extracts the dynamics from corresponding video and audio data to represent characteristics of subtle changes in facial and vocal expression of depression. On the other hand, for each modality, the Partial Least Square (PLS) regression algorithm is applied to learn the relationship between the dynamic features and depression scales using training data, and then predict the depression scale for an unseen one. Predicted values of visual and vocal clues are further combined at decision level for final decision. The proposed approach is evaluated on the AVEC2013 dataset and experimental results clearly highlight its effectiveness and better performance than baseline results provided by the AVEC2013 challenge organiser.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Cites
Histograms of Oriented Gradients for Human Detection
Dalal, N., & Triggs, B. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.177 · Cited by 33,584 · see citing works
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398 · see citing works
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019 · see citing works
Face Description with Local Binary Patterns: Application to Face Recognition
Ahonen, T., Hadid, A., & Pietikäinen, M. (2006) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2006.244 · Cited by 5,592 · see citing works
Computer Vision: A Modern Approach
Forsyth, D. A., & Ponce, J. (2002) · Cited by 3,728 · see citing works
The recognition of human movement using temporal templates
Bobick, A., & Davis, J. W. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.910878 · Cited by 2,830 · see citing works
Facial expression recognition based on Local Binary Patterns: A comprehensive study
Shan, C., Gong, S., & McOwan, P. W. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.08.005 · Cited by 2,300 · see citing works
Survey on speech emotion recognition: Features, classification schemes, and databases
Ayadi, M. M. H. E., Kamel, M. S., & Karray, F. (2010) · Pattern Recognition · doi:10.1016/j.patcog.2010.09.020 · Cited by 2,089 · see citing works
SIMPLS: An alternative approach to partial least squares regression
de Jong, S. (1993) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(93)85002-x · Cited by 1,905 · see citing works
Scale for the Assessment of Negative Symptoms
Andreasen, N. C. (1982) · PsycTESTS Dataset · doi:10.1037/t12696-000 · Cited by 1,878 · see citing works
Automatic analysis of facial expressions: the state of the art
Pantie, M., & Rothkrantz, L. J. M. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.895976 · Cited by 1,778 · cited 1× · see citing works
“using static features extracted from still face images [31, 37].”
MethodRecognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745 · see citing works
Speech emotion recognition using hidden Markov models
Nwe, T. L., Foo, S. W., & De Silva, L. C. (2003) · Speech Communication · doi:10.1016/s0167-6393(03)00099-2 · Cited by 920 · see citing works
Facial expression recognition from video sequences: temporal and static modeling
Cohen, I., Sebe, N., Garg, A., Chen, L. S., & Huang, T. S. (2003) · Computer Vision and Image Understanding · doi:10.1016/s1077-3142(03)00081-x · Cited by 874 · see citing works
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840 · cited 1× · see citing works
“Therefore, a nat-ural trend appears to combine the vocal modality with the visual one, claiming that both the clues are complementary to each other and the joint use improves system performance [7] [49].”
BackgroundThe role of voice quality in communicating emotion, mood and attitude
Gobl, C. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00082-1 · Cited by 790 · see citing works
Information combination operators for data fusion: a comparative review with classification
Bloch, I. (1996) · IEEE Transactions on Systems Man and Cybernetics - Part A Systems and Humans · doi:10.1109/3468.477860 · Cited by 640 · see citing works
Zhang, Z., Lyons, M., Schuster, M., & Akamatsu, S. (2002) · Proceedings Third IEEE International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.1998.670990 · Cited by 613 · see citing works
Describing the emotional states that are expressed in speech
Cowie, R., & Cornelius, R. R. (2003) · Speech Communication · doi:10.1016/s0167-6393(02)00071-7 · Cited by 602 · see citing works
Valstar, M., Schuller, B. W., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantić, M. (2013) · doi:10.1145/2512530.2512533 · Cited by 581 · see citing works
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580 · see citing works
Detecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494 · see citing works
Recognizing human facial expressions from long image sequences using optical flow
Yacoob, Y., & Davis, L. S. (1996) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.506414 · Cited by 458 · see citing works
Nonlinear feature based classification of speech under stress.
Zhou, G., Hansen, J. H. L., & Kaiser, J. F. (2001) · IEEE Transactions on Speech and Audio Processing · doi:10.1109/89.905995 · Cited by 409 · see citing works
Face recognition using active appearance models
Edwards, G. J., Cootes, T. F., & Taylor, C. (1998) · Lecture notes in computer science · doi:10.1007/bfb0054766 · Cited by 408 · see citing works
The first facial expression recognition and analysis challenge
Valstar, M. F., Jiang, B., Mehu, M., Pantic, M., & Scherer, K. (2011) · Face and Gesture 2011 · doi:10.1109/fg.2011.5771374 · Cited by 385 · see citing works
Facial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships
Tong, Y., Liao, W., & Ji, Q. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1094 · Cited by 377 · see citing works
Some observations on oral air flow during phonation
Teager, H. M. (1980) · IEEE Transactions on Acoustics Speech and Signal Processing · doi:10.1109/tassp.1980.1163453 · Cited by 344 · see citing works
Fast multiple object tracking via a hierarchical particle filter
Yang, C., Ramani Duraiswami, B., & Davis, L. S. (2005) · IEEE/CVF International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2005.95 · Cited by 315 · see citing works
Hamm, J., Kohler, C., Gur, R., & Verma, R. (2011) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2011.06.023 · Cited by 296 · see citing works
Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures
El Kaliouby, R., & Robinson, P. (2005) · 2004 Conference on Computer Vision and Pattern Recognition Workshop · doi:10.1109/cvpr.2004.427 · Cited by 282 · see citing works
AVEC 2011–The First International Audio/Visual Emotion Challenge
Schuller, B. W., Valstar, M., Eyben, F., McKeown, G. J., Cowie, R., & Pantić, M. (2011) · Lecture notes in computer science · doi:10.1007/978-3-642-24571-8_53 · Cited by 270 · see citing works
IEEE International Conference on Automatic Face and Gesture Recognition
Thippur, A., Ek, C. H., & Kjellström, H. (2013) · Cited by 257 · see citing works
Schuller, B. W., Valstar, M., Eyben, F., Cowie, R., & Pantić, M. (2012) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2388676.2388776 · Cited by 225 · see citing works
Optimal linear combination of neural networks for improving classification performance
Ueda, N. (2000) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.825759 · Cited by 205 · see citing works
Social risk and depression: Evidence from manual and automatic facial expression analysis
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., & Rosenwald, D. P. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553748 · Cited by 197 · see citing works
Automatic Recognition of Non-Acted Affective Postures
Kleinsmith, A., Bianchi‐Berthouze, N., & Steed, A. J. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2103557 · Cited by 174 · see citing works
Automatic behavior descriptors for psychological disorder analysis
Scherer, S., Stratou, G., Mahmoud, M., Boberg, J., Gratch, J., Rizzo, A., & Morency, L.-P. (2013) · 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2013.6553789 · Cited by 147 · see citing works
Real-Time Inference of Complex Mental States from Facial Expressions and Head Gestures
el Kaliouby, R., & Robinson, P. (2005) · Springer eBooks · doi:10.1007/0-387-27890-7_11 · Cited by 134 · see citing works
A Human Action Recognition System for Embedded Computer Vision Application
Meng, H., Pears, N. E., & Bailey, C. (2007) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2007.383420 · Cited by 114 · see citing works
Self-citationAutomated video-based facial expression analysis of neuropsychiatric disorders
Wang, P., Barrett, F. S., Martin, E. A., Milonova, M., Gur, R. E., Gur, R. C., Kohler, C. G., & Verma, R. (2007) · Journal of Neuroscience Methods · doi:10.1016/j.jneumeth.2007.09.030 · Cited by 90 · see citing works
Affective State Level Recognition in Naturalistic Facial and Vocal Expressions
Meng, H., & Bianchi‐Berthouze, N. (2013) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2013.2253768 · Cited by 75 · see citing works
Self-citationAudio-Visual Affect Recognition through Multi-Stream Fused HMM for HCI
Zeng, Z., Tu, J., Pianfetti, B., Liu, M., Zhang, T., Zhang, Z., Huang, T. S., & Levinson, S. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.77 · Cited by 62 · see citing works
Descriptive temporal template features for visual motion recognition
Meng, H., & Pears, N. E. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.03.003 · Cited by 49 · see citing works
Self-citationEmotion recognition by two view SVM_2K classifier on dynamic facial expression features
Meng, H., Romera‐Paredes, B., & Bianchi‐Berthouze, N. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771362 · Cited by 40 · see citing works
Self-citationMotion History Histograms for Human Action Recognition
Meng, H., Pears, N. E., Freeman, M., & Bailey, C. (2008) · Advances in computer vision and pattern recognition · doi:10.1007/978-1-84800-304-0_7 · Cited by 36 · see citing works
Self-citationEstimation and Decision Fusion: A Survey
Sinha, A., Chen, H., Danu, D. G., Kirubarajan, T., & Farooq, M. (2006) · IEEE International Conference on Engineering of Intelligent Systems (ICEIS) · doi:10.1109/iceis.2006.1703130 · Cited by 9 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Valstar, M., Schuller, B. W., Smith, K., Almaev, T. R., Eyben, F., Krajewski, J., Cowie, R., & Pantić, M. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661807 · Cited by 476 · see citing works
“127 for Valence and Arousal, respectively [20].”
Ma, X., Yang, H., Chen, Q., Huang, D., & Wang, Y. (2016) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2988257.2988267 · Cited by 300 · see citing works
Self-citationVisually Interpretable Representation Learning for Depression Recognition from Facial Images
Zhou, X., Jin, K., Shang, Y., & Guo, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2828819 · Cited by 273 · see citing works
“[8] adopted motion history histogram (MHH) [25] based feature to encode motion cue from videos, and the partial least squares (PLS) [30] was”
BackgroundMethodInfluentialAutomated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics
Zhu, Y. M., Shang, Y., Shao, Z., & Guo, G. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2650899 · Cited by 258 · see citing works
“[7] utilized Motion History Histogram (MHH) [25] to characterize motion information of each pixel in the video.”
MethodDeep learning for depression recognition with audiovisual cues: A review
He, L., Niu, M., Tiwari, P., Marttinen, P., Su, R., Jiang, J., Guo, C., Wang, H., Ding, S., Wang, Z., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.10.012 · Cited by 270 · see citing works
“Regarding modality, speech and video samples [14, 94, 113, 17, 96, 97, 99, 100, 101, 104, 121, 122, 123, 149, 150, 16, 151, 152, 153], physiological signals [100, 154, 154, 155, 156, 157, 158, 159, 160], and text [100, 108] have been employed to improve the performance of depression assessment.”
MethodSelf-citationAutomated depression analysis using convolutional neural networks from speech
He, L., & Cui, C. (2018) · Journal of Biomedical Informatics · doi:10.1016/j.jbi.2018.05.007 · Cited by 250 · see citing works
AFEW-VA database for valence and arousal estimation in-the-wild
Kossaifi, J., Tzimiropoulos, G., Todorovic, S., & Pantic, M. (2017) · Image and Vision Computing · doi:10.1016/j.imavis.2017.02.001 · Cited by 236 · see citing works
“There are several approaches to valence and arousal estimation, the most obvious of which is static regression, such as linear regression [25,37], partial least squares regression [22], or Support Vector Machine for Regression (SVR) [12,21,38], which is also frequently used as a baseline method [11,41-43].”
MethodDeep learning for human affect recognition: Insights and new developments
Rouast, P. V., Adam, M. T. P., & Chiong, R. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2890471 · Cited by 243 · see citing works
Automatic Assessment of Depression Based on Visual Cues: A Systematic Review
Pampouchidou, A., Simos, P. G., Kostas Marias, A., Mériaudeau, F., Yang, F., Pediaditis, M., & Tsiknakis, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2724035 · Cited by 224 · see citing works
“In [121] decision fusion was implemented linearly through a weighted sum according to the formula”
MethodJan, A., Meng, H., Gaus, Y. F. B. A., & Zhang, F. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2721552 · Cited by 219 · see citing works
“[11] modeled the visual and vocal cues for depression analysis.”
BackgroundMethodResultInfluentialSelf-citationMultimodal Depression Detection: Fusion Analysis of Paralinguistic, Head Pose and Eye Gaze Behaviors
Alghowinem, S., Goecke, R., Wagner, M., Epps, J., Hyett, M. P., Parker, G. B., & Breakspear, M. J. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2634527 · Cited by 204 · see citing works
“To the best of our knowledge, only a few studies have investigated multiple channels for this task [18], [19], [20], [21], [22], [23].”
BackgroundMethodResultDynamic Multimodal Measurement of Depression Severity Using Deep Autoencoding
Dibeklioğlu, H., Hammal, Z., & Cohn, J. F. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2676878 · Cited by 196 · see citing works
“Using AVEC and a few non-publicly available resources [25], audiovisual detection of depression has been proposed [26]– [33].”
Facial Expression Recognition Using Computer Vision: A Systematic Review
Canedo, D., & Neves, A. J. R. (2019) · Applied Sciences · doi:10.3390/app9214678 · Cited by 192 · see citing works
Kollias, D., & Zafeiriou, S. P. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3014171 · Cited by 23 · see citing works
“In the beginning, data and annotations were scarce, hence research relied on extracting highly engineered handcrafted features and designing ad-hoc learning strategies [40], [41], [48], [52].”
MethodAutomatic depression recognition using CNN with attention mechanism from videos
He, L., Chan, J. C.-W., & Wang, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.10.015 · Cited by 179 · see citing works
Shoumy, N. J., Ang, L.-M., Seng, K. P., Rahaman, D. M. M., & Zia, T. (2020) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2019.102447 · Cited by 171 · see citing works
Multimodal Spatiotemporal Representation for Automatic Depression Level Detection
Niu, M., Tao, J., Liu, B., Huang, J., & Lian, Z. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3031345 · Cited by 171 · see citing works
“In the works of [7], [15], [23], [24], [25], video dynamic features are generated by calculating the statistical histogram, but the histogram can not reflect the uneven distribution of features in temporal space [48].”
BackgroundMethodInfluentialA hybrid model for depression detection using deep learning
Vandana, V., Marriwala, N. K., & Chaudhary, D. (2022) · Measurement Sensors · doi:10.1016/j.measen.2022.100587 · Cited by 164 · see citing works
Video-Based Depression Level Analysis by Encoding Deep Spatiotemporal Features
Jazaery, M. A., & Guo, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2870884 · Cited by 158 · see citing works
“The speech characteristics can be useful for depression analysis [5], [6], some depression analysis methods are developed using the audio data [7], [8], [9], [10], [11], [12].”
MethodKhoo, L. S., McNaney, R., Lim, M. K., & McNaney, R. (2024) · Sensors · doi:10.3390/s24020348 · Cited by 178 · see citing works
He, L., Jiang, D., Yang, L., Pei, E., Wu, P., & Hichem Sahli, B. (2015) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2808196.2811641 · Cited by 149 · see citing works
“Among other features, Meng et al. in [17] performed Motion History Histograms (MHH) on the LLDs to extract change information of the vocal expressions.”
BackgroundSpectral Representation of Behaviour Primitives for Depression Analysis
Song, S., Jaiswal, S., Shen, L., & Valstar, M. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2970712 · Cited by 149 · see citing works
“For example, Meng et al. [29] extracted LBP and EOH as visual features and LLD as audio features, and applied Motion History Histogram (MHH) to extract dynamics from short video segments.”
BackgroundMethodMultimodal and Multiresolution Depression Detection from Speech and Facial Landmark Features
Nasir, M., Jati, A., Shivakumar, P. G., Chakravarthula, S. N., & Georgiou, P. (2016) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2988257.2988261 · Cited by 147 · see citing works
“[28] used Motion History Histogram (MHH) dynamical features [29] to recognize depression from both audio and video.”
MethodAutomated Depression Diagnosis Based on Facial Dynamic Analysis and Sparse Coding
Wen, L., Li, X., Guo, G., & Zhu, Y. M. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2414392 · Cited by 141 · see citing works
“Meng et al. utilized Motion History Histogram (MHH) to capture temporal information of each pixel in the video [6] .”
Ecologically valid long-term mood monitoring of individuals with bipolar disorder using speech
Karam, Z. N., Provost, E. M., Singh, S. P., Montgomery, J., Archer, C., Harrington, G. J., & McInnis, M. G. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854525 · Cited by 142 · see citing works
“The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. major depression (MD) [5] [6] [7] [8], autism [9] [10] [11] [12], and post-traumatic stress disorder (PTSD) [13] [14] [15] [16].”
BackgroundFan, H., Zhang, X., Xu, Y., Fang, J., Zhang, S., Zhao, X., & Yu, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102161 · Cited by 137 · see citing works
MDN: A Deep Maximization-Differentiation Network for Spatio-Temporal Depression Detection
de Melo, W. C., Granger, É., & López, M. B. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3072579 · Cited by 137 · see citing works
“The following researches on AVEC2013 dataset are relied on LPQ [34], LBP [35], Pyramid of Histogram of Gra-dients (PHOG) [36], Local Phase Quantization from Three Orthogonal Planes (LPQ-TOP) [37], and Canonical Correlation Analysis (CCA) [38].”
MethodIntegrating Deep and Shallow Models for Multi-Modal Depression Analysis—Hybrid Architectures
Yang, L., Jiang, D., & Hichem Sahli, B. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2870398 · Cited by 121 · see citing works
“Table 7 compares the performance of HDR (HDR-DCNN-DNN) with those of Bag-of-Words(BoW-DCNN-DNN) and Motion History Histograms(MHH-DCNN-DNN) [49], together with the parameters (threshold ( th ) and the number of frames ( m ) for MHH, the number of clusters ( K ) for BoW, and the number of ranges ( R…”
BackgroundMethodMulti-modal emotion analysis from facial expressions and electroencephalogram
Huang, X., Kortelainen, J., Zhao, G., Li, X., Moilanen, A., Seppänen, T., & Pietikäinen, M. (2016) · Computer Vision and Image Understanding · doi:10.1016/j.cviu.2015.09.015 · Cited by 117 · see citing works
A Deep Multiscale Spatiotemporal Network for Assessing Depression From Facial Dynamics
de Melo, W. C., Granger, É., & Hadid, A. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3021755 · Cited by 116 · see citing works
“The models in [20], [24], [28], [30], [33], [56], [58] are based on hand-engineered representations.”
BackgroundAutomatic Depression Scale Prediction using Facial Expression Dynamics and Regression
Jan, A., Meng, H., Gaus, Y. F. A., Zhang, F., & Turabzadeh, S. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661812 · Cited by 114 · see citing works
“[14] presents a novel method, which comprehensively models visual and vocal modalities with dynamic features to automatically predicts the scale of depression.”
BackgroundMethodResultInfluentialSelf-citationHe, L., Jiang, D., & Hichem Sahli, B. (2018) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2018.2877129 · Cited by 113 · see citing works
“In [48], the authors adopted Motion History Histogram (MHH) to extract dynamic features from videos.”
MethodDepression Level Prediction Using Deep Spatiotemporal Features and Multilayer Bi-LTSM
Uddin, M. A., Joolee, J. B., & Lee, Y.-K. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2970418 · Cited by 110 · see citing works
Depression Assessment by Fusing High and Low Level Features from Audio, Video, and Text
Pampouchidou, A., Simantiraki, O., Fazlollahi, A., Pediaditis, M., Manousos, D., Roniotis, A., Giannakakis, G. A., Mériaudeau, F., Simos, P. G., Kostas Marias, A., et al. (2016) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2988257.2988266 · Cited by 106 · see citing works
“Meng et al. [17] in their participation to AVEC 2013 [30], used MHIs for depression recognition, by extracting Local Binary Patterns (LBP) and Edge of Oriented Histograms from them.”
Methodde Melo, W. C., Granger, É., & Hadid, A. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756568 · Cited by 104 · see citing works
“[25] proposed a conventional method based on Motion History Histogram [26] to capture motion information of facial expressions.”
MethodDepression Detection Based on Deep Distribution Learning
de Melo, W. C., Granger, É., & Hadid, A. (2019) · Proceedings - International Conference on Image Processing · doi:10.1109/icip.2019.8803467 · Cited by 103 · see citing works
“The methods in [17, 18, 19, 20, 21] are based on handcrafted features.”
MethodSpatial–Temporal Attention Network for Depression Recognition from facial videos
Pan, Y., Shang, Y., Liu, T., Shao, Z., Guo, G., Ding, H., & Hu, Q. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121410 · Cited by 102 · see citing works
Towards Automatic Depression Detection: A BiLSTM/1D CNN-Based Model
Lin, L., Chen, X., Shen, Y., & Zhang, L. (2020) · Applied Sciences · doi:10.3390/app10238701 · Cited by 100 · see citing works
Gooding, P., & Kariotis, T. (2021) · JMIR Mental Health · doi:10.2196/24668 · Cited by 100 · see citing works
Hybrid Depression Classification and Estimation from Audio Video and Text Information
Yang, L., Hichem Sahli, B., Xia, X., Pei, E., Oveneke, M. C., & Jiang, D. (2017) · doi:10.1145/3133944.3133950 · Cited by 98 · see citing works
“[12], we propose a method to extract the dynamics from the AUs.”
MethodDeep Multi-Modal Network Based Automated Depression Severity Estimation
Uddin, M. A., Joolee, J. B., & Sohn, K.-A. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3179478 · Cited by 99 · see citing works
“Meng et al. [44] utilized the Motion History Histogram (MHH) to obtain the variations in facial and vocal expression.”
Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors
Khalid, H., Kim, M., Tariq, S., & Woo, S. S. (2021) · Proceedings of the 1st Workshop on Synthetic Multimedia - Audiovisual Deepfake Generation and Detection · doi:10.1145/3476099.3484315 · Cited by 110 · see citing works
“Recently, there has been some recent interest in multimodal learning because of the fact that multimodal outperforms single feature-based methods [25, 42, 45, 53].”
BackgroundMuscular Movement Model-Based Automatic 3D/4D Facial Expression Recognition
Zhen, Q., Huang, D., Wang, Y., & Chen, L. (2016) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2016.2557063 · Cited by 97 · see citing works
“In recent years, as a major branch of automatic face analysis, Facial Expression Recognition (FER) has attracted increasing interests in the filed of computer vision and pattern recognition due to its scientific challenges, and has shown its potential in many applications, such as psychological analysis [1], transport…”
BackgroundSelf-citationDepression Recognition Using Remote Photoplethysmography From Facial Videos
Casado, C. A., Cañellas, M. L., & López, M. B. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3238641 · Cited by 94 · see citing works
Meng, H., Bianchi‐Berthouze, N., Deng, Y. S., Cheng, J., & Cosmas, J. (2015) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2015.2418092 · Cited by 94 · see citing works
Self-citationFacial Depression Recognition by Deep Joint Label Distribution and Metric Learning
Zhou, X., Wei, Z., Xu, M., Qu, S., & Guo, G. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3022732 · Cited by 80 · see citing works
“Our DJ-LDML is first compared to 13 visual-based state of the arts [4], [7], [9], [14], [15], [17], [25], [49], [50],”
BackgroundMethodInfluentialCross-cultural detection of depression from nonverbal behaviour
Alghowinem, S., Goecke, R., Cohn, J. F., Wagner, M., Parker, G. B., & Breakspear, M. J. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163113 · Cited by 77 · see citing works
He, L., Guo, C., Tiwari, P., Pandey, H. M., & Dang, W. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22426 · Cited by 78 · see citing works
“[33] Meng H, Huang D,Wang H, Yang H, AI-Shuraifi M,Wang Y.”
Zhang, S., Zhang, X., Zhao, X., Fang, J., Niu, M., Zhao, Z., Yu, J., & Tian, Q. C. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3312263 · Cited by 74 · see citing works
“In particular, typical (LPQ)[22],MotionHistoryHis-togram(MHH)[26],LocalGaborBinaryPatternsfromThree OrthogonalPlanes(LGBP-TOP)[23],LocalBinaryPatterns FromThreeOrthogonalPlanes(LBP-TOP)[25],LocalPhase QuantizationfromThreeOrthogonalPlanes(LPQ-TOP)[24], andsoon.”
MethodEncoding Temporal Information For Automatic Depression Recognition From Facial Analysis
de Melo, W. C., Granger, É., & López, M. B. (2020) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp40776.2020.9054375 · Cited by 72 · see citing works
“Handengineered representations are employed in [13, 17, 18, 19, 20].”
Towards Robust Deep Neural Networks for Affect and Depression Recognition from Speech
Othmani, A., Kadoch, D., Bentounes, K. A., Rejaibi, E., Alfred, R., & Hadid, A. (2021) · Lecture notes in computer science · doi:10.1007/978-3-030-68790-8_1 · Cited by 74 · see citing works
“[18] spectral + energy PLS regression AVEC2013 RMSE 11.”
BackgroundGuo, W., Yang, H., Liu, Z., Xu, Y., & Hu, B. (2021) · Frontiers in Neuroscience · doi:10.3389/fnins.2021.609760 · Cited by 71 · see citing works
“Histogram of optical flow (Gupta et al., 2014), motion history histogram (MHH) (Meng et al., 2013), and space-time interest points (STIPs) (He et al., 2015) are extracted to describe the facial motion.”
MethodSelf-citationPRA-Net: Part-and-Relation Attention Network for depression recognition from facial expression
Liu, Z., Yuan, X., Li, Y., Shangguan, Z., Zhou, L., & Hu, B. (2023) · Computers in Biology and Medicine · doi:10.1016/j.compbiomed.2023.106589 · Cited by 69 · see citing works
Smart Contract Vulnerability Detection Based on Deep Learning and Multimodal Decision Fusion
Deng, W., Wei, H., Huang, T., Cao, C., Peng, Y., & Hu, X. (2023) · Sensors · doi:10.3390/s23167246 · Cited by 63 · see citing works
“Feature-based Early Emotion recognition [41–43] Decision-based Terminal Emotion recognition [44,45] Hybrid Fusion Early and Terminal Event detection [46–48]”
MethodKaya, H., Eyben, F., Salah, A. A., & Schuller, B. W. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854298 · Cited by 63 · see citing works
“[18] (10.”
Multimodal depression recognition with dynamic visual and audio cues
He, L., Jiang, D., & Hichem Sahli, B. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344581 · Cited by 63 · see citing works
“Motion History Histogram (MHH) is a descriptive representation of temporal-series, which achieved good performance in human action recognition [22] and depression analysis [23], [24].”
BackgroundAutomated audiovisual depression analysis
Girard, J. M., & Cohn, J. F. (2015) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2014.12.010 · Cited by 62 · see citing works
“[32] Meng H, Hunag D, Wang H, Yang H, Al-shuraifi M, Wang Y: Depression recognition based on dynamic facial and vocal expression features using partial least square regression.”
MethodHybrid Network Feature Extraction for Depression Assessment from Speech
Zhao, Z., Li, Q., Cummins, N., Liu, B., Wang, H., Tao, J., & Schuller, B. W. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2396 · Cited by 61 · see citing works
“Moreover, as shown in Table 2 and Table 3, although the performance of the BLSTM, DCNN, SAN and BLSTM-DCNN models is inferior to that of the proposed hybrid model, it still surpasses that of the AVEC 2013 or AVEC 2014 audio baseline and the multi-modal work proposed in [30, 31] in terms of RMSE and MAE.”
ResultSelf-citationSequential fusion of facial appearance and dynamics for depression recognition
Chen, Q., Chaturvedi, I., Ji, S., & Cambria, E. (2021) · Pattern Recognition Letters · doi:10.1016/j.patrec.2021.07.005 · Cited by 61 · see citing works
“Meng et al. [9] proposed to use motion history histogram (MHH) [23] feature to model motion in videos, and then use the partial least squares (PLS) [24] for training regression model.”
BackgroundMethodPan, Y., Shang, Y., Shao, Z., Liu, T., Guo, G., & Ding, H. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3296318 · Cited by 58 · see citing works
Detecting Depression with Word-Level Multimodal Fusion
Rohanian, M., Hough, J., & Purver, M. (2019) · Interspeech (USB) · doi:10.21437/interspeech.2019-2283 · Cited by 58 · see citing works
“However, while this work tries to predict depression based on late or early fusion methods [10, 12] or the sequential nature of their inputs [17, 18], learning the time-dependent relationships between language, visual and audio features in detecting depression is still unexplored.”
BackgroundLQGDNet: A Local Quaternion and Global Deep Network for Facial Depression Recognition
Shang, Y., Pan, Y., Jiang, X., Shao, Z., Guo, G., Liu, T., & Ding, H. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3139651 · Cited by 58 · see citing works
“Most hand-crafted features [5], [8], [18], [19], [20] require a gray-”
MethodThe SRI AVEC-2014 Evaluation System
Mitra, V., Shriberg, E. E., McLaren, M., Kathol, A., Richey, C., Vergyri, D., & Graciarena, M. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661818 · Cited by 56 · see citing works
“Several studies [17, 18, 19, 20] have used both audio and video modalities.”
Automatic nonverbal behavior indicators of depression and PTSD: the effect of gender
Stratou, G., Scherer, S. J., Gratch, J. S., & Morency, L. (2014) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-014-0161-4 · Cited by 57 · see citing works
“On the side of automatic assessment of depression there has been recent work [17,18] that focuses on the automatic diagnosis of depression from multimodal features, exploring dynamics of the face.”
BackgroundMultimodal mild depression recognition based on EEG-EM synchronization acquisition network
Zhu, J., Wang, Y., La, R., Zhan, J., Niu, J., Zeng, S., & Hu, X. (2019) · IEEE Access · doi:10.1109/access.2019.2901950 · Cited by 56 · see citing works
“[37] investigated the fusion of facial and vocal expressions, and used a weighted sum decision fusion; their result showed a slight improvement comparedwith individual channels.”
MethodSelf-citationXu, J., Güneş, H., Kusumam, K., Valstar, M., & Song, S. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3415770 · Cited by 55 · see citing works
“However, handcrafted methods [23], [26], [27], [28] which are engineered to summarize the frame-level/thin slice-level features into a video-level descriptor, generally fail to learn depressionspecific features that can be learned using deep learning methods.”
BackgroundEncoding Visual Behaviors with Attentive Temporal Convolution for Depression Prediction
Du, Z., Li, W., Huang, D., & Wang, Y. (2019) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2019.8756584 · Cited by 53 · see citing works
“Pioneering studies on this issue mostly model depressive temporal characteristics in the face area, making use of facial appearances, head poses, gazes and so forth [7] [23] [17] [1].”
BackgroundSelf-citationHe, L., Guo, C., Tiwari, P., Su, R., Pandey, H. M., & Dang, W. (2021) · International Journal of Intelligent Systems · doi:10.1002/int.22704 · Cited by 57 · see citing works
“Meng et al.(30) adopted LBP and Edge Orientation Histogram (EOH) as frame‐based features and Motion History Histogram (MHH) to represent dynamic and discriminative patterns from videos.”
MethodA Multimodal Approach for Detection and Assessment of Depression Using Text, Audio and Video
Zhang, W., Mao, K., & Chen, J. (2024) · Phenomics · doi:10.1007/s43657-023-00152-8 · Cited by 60 · see citing works
“Meng et al. (2013) proposed using MHI along with extracting Local Binary Patterns and Edge of Oriented Histograms for depression recognition in the Audio/visual emotion challenge (AVEC) 2013 Challenge.”
Multi task sequence learning for depression scale prediction from video
Chao, L., Tao, J., Yang, M., & Li, Y. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344620 · Cited by 53 · see citing works
“[11] presents a novel method, which comprehensively models visual and vocal modalities with dynamic features to automatically predicts the scale of depression.”
MethodNiu, M., Liu, B., Tao, J., & Li, Q. (2021) · Neurocomputing · doi:10.1016/j.neucom.2021.04.056 · Cited by 48 · see citing works
Reducing noisy annotations for depression estimation from facial images
He, L., Tiwari, P., Lv, C., Wu, W., & Guo, L. (2022) · Neural Networks · doi:10.1016/j.neunet.2022.05.025 · Cited by 47 · see citing works
Multimodal Fusion with Deep Neural Networks for Audio-Video Emotion RecognitionPreprint
Ortega, J. D. S., Senoussaoui, M. E. A., Granger, É., Pedersoli, M., Cardinal, P., & Koerich, A. L. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1907.03196 · Cited by 57 · see citing works
“Meng et al. [15] propose a multi-layer system using Motion History Histogram dynamical features to recognize depression from both audio and video.”
Automated screening for distress: A perspective for the future
Rana, R. K., Latif, S., Gururajan, R., Gray, A. D., Mackenzie, G., Humphris, G. M., & Dunn, J. C. (2019) · European Journal of Cancer Care · doi:10.1111/ecc.13033 · Cited by 45 · see citing works
Facial expression analysis using Decomposed Multiscale Spatiotemporal Networks
de Melo, W. C., Granger, É., & López, M. B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121276 · Cited by 43 · see citing works
A Linguistically-Informed Fusion Approach for Multimodal Depression Detection
Morales, M. R., Scherer, S. J., & Levitan, R. (2018) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w18-0602 · Cited by 43 · see citing works
“Late fusion approaches have also been investigated by some (Joshi et al., 2013; Meng et al., 2013).”
BackgroundMethodBipolar Disorder Recognition via Multi-scale Discriminative Audio Temporal Representation
Du, Z., Li, W., Huang, D., & Wang, Y. (2018) · doi:10.1145/3266302.3268997 · Cited by 42 · see citing works
“Recent studies [21] [19] show that automatic audio and video analysis techniques can advance detection of some mental disorders, e.”
BackgroundSelf-citationKaya, H., & Salah, A. A. (2014) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2647868.2654978 · Cited by 40 · see citing works
“From the table we observe that features using range functional give more information about the target variable than the mode features.”
MethodDepressioner: Facial dynamic representation for automatic depression level prediction
Niu, M., He, L., Li, Y., & Liu, B. (2022) · Expert Systems with Applications · doi:10.1016/j.eswa.2022.117512 · Cited by 39 · see citing works
Dual Attention and Element Recalibration Networks for Automatic Depression Level Prediction
Niu, M., Zhao, Z., Tao, J., Li, Y., & Schuller, B. W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3177737 · Cited by 39 · see citing works
Dual-Stream Multiple Instance Learning for Depression Detection With Facial Expression Videos
Shangguan, Z., Liu, Z., Li, G., Chen, Q., Ding, Z., & Hu, B. (2022) · IEEE Transactions on Neural Systems and Rehabilitation Engineering · doi:10.1109/tnsre.2022.3204757 · Cited by 43 · see citing works
A novel multimodal depression diagnosis approach utilizing a new hybrid fusion method
Zhang, X., Li, B., & Qi, G. (2024) · Biomedical Signal Processing and Control · doi:10.1016/j.bspc.2024.106552 · Cited by 39 · see citing works
MLM-EOE: Automatic Depression Detection via Sentimental Annotation and Multi-Expert Ensemble
Lin, Z., Wang, Y., Zhou, Y., Du, F., & Yang, Y. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3585599 · Cited by 39 · see citing works
“Earlystudies onADD oftenutilizedtraditional machinelearn-ing techniques, such as Local Binary Pattern (LBP), Edge Orientation Histogram (EOH), and Low-Level Descriptors (LLD), to represent features of individual video frames [26], [27], [28].”
Self-citationMulti-scale Temporal Modeling for Dimensional Emotion Recognition in Video
Chao, L., Tao, J., Yang, M., Li, Y., & Wen, Z. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661811 · Cited by 39 · see citing works
Du, Z., Wu, S., Huang, D., Li, W., & Wang, Y. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2940224 · Cited by 39 · see citing works
Self-citationRetracted: Diagnostic Criteria for Depression based on Both Static and Dynamic Visual Features
Pandya, D. D., Jadeja, A., Degadwala, S., & Vyas, D. (2023) · International Conference on Intelligent Data Communication Technologies and Internet of Things (IDCIoT) · doi:10.1109/idciot56793.2023.10053450 · Cited by 37 · see citing works
RetractedDesigning a framework for assisting depression severity assessment from facial image analysis
Pampouchidou, A., Kostas Marias, A., Tsiknakis, M., Simos, P. G., Yang, F., & Mériaudeau, F. (2015) · IEEE International Conference on Signal and Image Processing Applications (ICSIPA) · doi:10.1109/icsipa.2015.7412257 · Cited by 37 · see citing works
“LBP was computed for 41×windowSize images for plane 1, plus the N + M images of planes 2 and 3, for the parameter combination [8,1] and [16,2] (neighbourhood and radius respectively) [33].”
MethodShan, G., Lina, Z., & Zhang, D. (2020) · Information & Management · doi:10.1016/j.im.2020.103349 · Cited by 36 · see citing works
WavDepressionNet: Automatic Depression Level Prediction via Raw Speech Signals
Niu, M., Tao, J., Li, Y., Qin, Y., & Li, Y. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3272553 · Cited by 36 · see citing works
Automatic Depression Recognition With an Ensemble of Multimodal Spatio-Temporal Routing Features
Wang, Y., Lin, Z., Yang, C., Zhou, Y., & Yang, Y. (2025) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2025.3543226 · Cited by 36 · see citing works
“Techniques like Local Phase Quantization (LPQ) [21] and Motion History His-togram (MHH) [22] extracted facial and behavioral features for analysis in machine learning models such as Support Vector Regression (SVR) [21] and Partial Least Squares (PLS) [23].”
Self-citationMeshram, P., & Rambola, R. K. (2022) · Expert Systems · doi:10.1111/exsy.12933 · Cited by 35 · see citing works
“These neural networks have a better application in describing the visible and facial features (Meng et al., 2014; Weber et al, 2016).”
BackgroundRetractedNiu, M., Zhao, Z., Tao, J., Li, Y., & Schuller, B. W. (2022) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2022.3182658 · Cited by 35 · see citing works
Parameswaran, N. S., & Venkataraman, D. (2019) · Indonesian Journal of Electrical Engineering and Computer Science · doi:10.11591/ijeecs.v14.i1.pp503-512 · Cited by 35 · see citing works
“Features of video and audio information are separated from the video utilizing a Movement History Histogram (MHH) which represents to the qualities of minute changes that occur in face and vocal appearances of the depressed [17].”
MethodTDCA-Net: Time-Domain Channel Attention Network for Depression Detection
Cai, C., Niu, M., Liu, B., Tao, J., & Liu, X. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1176 · Cited by 33 · see citing works
“They [7, 8, 9, 10, 17, 34, 35] use hand-crafted features as input to the model.”
MethodMachot, F. A., Ali, M., Ranasinghe, S., Mosa, A. H., & Kyamakya, K. (2018) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/3197768.3201523 · Cited by 33 · see citing works
“[24] Visual and vocal modalities, Motion History Histogram (MHH), Partial Least Square (PLS), regression algorithm Scale of depression”
MethodNiu, M., Li, Y., Tao, J., Zhou, X., & Schuller, B. W. (2024) · IEEE Transactions on Circuits and Systems for Video Technology · doi:10.1109/tcsvt.2024.3382334 · Cited by 31 · see citing works
“In general, it is difficult to match the experimental performance of methods [4], [5], [47], [48], [49], [50] using hand-crafted features with those using neural networks [10], [12], [13], [35], [51], [52].”
MethodEffects of feature type, learning algorithm and speaking style for depression detection from speech
Mitra, V., & Shriberg, E. E. (2015) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2015.7178877 · Cited by 29 · see citing works
“It was demonstrated in [20] that use of both audio and video modalities improves the accuracy of a MDD detection compared to using each modality alone.”
MethodCsenet: Complex Squeeze-and-Excitation Network for Speech Depression Level Prediction
Fan, C., Lv, Z., Pei, S., & Niu, M. (2022) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp43922.2022.9746011 · Cited by 29 · see citing works
Multi-modality Hierarchical Recall based on GBDTs for Bipolar Disorder Classification
Xing, X., Cai, B., zhao, Y., Li, S., He, Z., & Fan, W. (2018) · doi:10.1145/3266302.3266311 · Cited by 28 · see citing works
“Inspired by Motion History Histogram (MHH) [17, 31], we extract the 17 key AUs MHH features fau_mhh ∈ R17×5×10=850 with 5 time intervals Mk ∈ {10, 20, 30, 40, 50} and 10 equally spaced bins Rb ∈ {−5,−4,−3, .”
MethodNoise and reverberation effects on depression detection from speech
Mitra, V., Tsiartas, A., & Shriberg, E. E. (2016) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2016.7472788 · Cited by 28 · see citing works
“Studies have also used both audio and video modalities [24, 25] for MDD detection.”
BackgroundMethod
- Authors
- Meng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., & Wang, Y.
- Affiliations
- Brunel University of London (GB)
- Beihang University (CN)
- Venue
- Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
- Publisher
- ACM
- Published
- 2013-10-21
- Pages
- 21-30
- OpenAlex
- W1988088279
- Semantic Scholar
- e638499e3649fb69142208c33abf63093c96b00c
- Cited by
- 225
- References
- 47
- Citations by year
- 2026: 7 · 2025: 24 · 2024: 24 · 2023: 21 · 2022: 23 · 2021: 18 · 2020: 18 · 2019: 21 · 2018: 14 · 2017: 16 · 2016: 10 · 2015: 18 · 2014: 11
- FWCI
- 17.63
- Type
- conference paper
- Language
- en
- Open access
- closed
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Keywords
- depression detection, facial expressions, vocal expression, multimodal fusion, partial least squares regression, depression severity prediction, decision fusion, affective computing
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
