Automatic behavior descriptors for psychological disorder analysis
Abstract
We investigate the capabilities of automatic nonverbal behavior descriptors to identify indicators of psychological disorders such as depression, anxiety, and post-traumatic stress disorder. We seek to confirm and enrich present state of the art, predominantly based on qualitative manual annotations, with automatic quantitative behavior descriptors. In this paper, we propose four nonverbal behavior descriptors that can be automatically estimated from visual signals. We introduce a new dataset called the Distress Assessment Interview Corpus (DAIC) which includes 167 dyadic interactions between a confederate interviewer and a paid participant. Our evaluation on this dataset shows correlation of our automatic behavior descriptors with specific psychological disorders as well as a generic distress measure. Our analysis also includes a deeper study of self-adaptor and fidgeting behaviors based on detailed annotations of where these behaviors occur.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Depression recognition from FEs is a very important application in analysis of psychological distress.
“Depression recognition from FEs is a very important application in analysis of psychological distress [53], [54], [55].”
From Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition: History, Trends, and Affect-Related Applications (2016) · cites this paper as [54]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910 · see citing works
(1970) · Cited by 24,983 · see citing works
An inventory for measuring clinical anxiety: Psychometric properties.
Beck, A. T., Epstein, N., Brown, G., & Steer, R. A. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.56.6.893 · Cited by 14,407 · see citing works
The PHQ-9: A New Depression Diagnostic and Severity Measure
Kroenke, K., & Spitzer, R. L. (2002) · Psychiatric Annals · doi:10.3928/0048-5713-20020901-06 · Cited by 5,898 · see citing works
Distribution Theory for Glass's Estimator of Effect Size and Related Estimators
Hedges, L. V. (1981) · Journal of Educational Statistics · doi:10.2307/1164588 · Cited by 5,695 · see citing works
Psychometric properties of the PTSD checklist (PCL)
Blanchard, E. B., Jones-Alexander, J., Buckley, T. C., & Forneris, C. A. (1996) · Behaviour Research and Therapy · doi:10.1016/0005-7967(96)00033-2 · Cited by 3,997 · see citing works
The Repertoire of Nonverbal Behavior: Categories, Origins, Usage, and Coding
EKMAN, P., & FRIESEN, W. V. (1969) · Semiotica · doi:10.1515/semi.1969.1.1.49 · Cited by 3,316 · see citing works
Russell, J. A., & Barrett, L. F. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.76.5.805 · Cited by 3,121 · see citing works
A meta-analysis of emotional reactivity in major depressive disorder
Bylsma, L. M., Morris, B. H., & Rottenberg, J. (2008) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.10.001 · Cited by 933 · see citing works
The State--Trait Anxiety Inventory, Trait version: structure and content re-examined
Bieling, P. J., Antony, M. M., & Swinson, R. P. (1998) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(98)00023-0 · Cited by 690 · see citing works
Coding gestural behavior with the NEUROGES-ELAN system
Lausberg, H., & Sloetjes, H. (2009) · Behavior Research Methods · doi:10.3758/brm.41.3.841 · Cited by 607 · see citing works
Detecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494 · see citing works
Campbell, D. G., Felker, B. L., Liu, C.-F., Yano, E. M., Kirchner, J. E., Chan, D., Rubenstein, L. V., & Chaney, E. F. (2007) · Journal of General Internal Medicine · doi:10.1007/s11606-006-0101-4 · Cited by 458 · cited 1× · see citing works
“and several findings in the literature that confirmed these co-morbidities [8], [23], we decided to combine the three measures using factor analysis to that of general distress.”
MethodAgreement and Information in the Reliability of Coding
Krippendorff, K. (2011) · Communication Methods and Measures · doi:10.1080/19312458.2011.568376 · Cited by 451 · see citing works
3D Constrained Local Model for rigid and non-rigid facial tracking
Baltrušaitis, T., Robinson, P., & Morency, L. (2012) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2012.6247980 · Cited by 320 · see citing works
Self-citationToward Practical Smile Detection
Whitehill, J., Littlewort, G. C., Fasel, I., Bartlett, M. S., & Movellan, J. R. (2009) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2009.42 · Cited by 310 · see citing works
Nonverbal behavior in clinician—patient interaction
Hall, J. A., Harrigan, J. A., & Rosenthal, R. (1995) · Applied and Preventive Psychology · doi:10.1016/s0962-1849(05)80049-6 · Cited by 309 · see citing works
Elhai, J. D., Carvalho, L. D. F., Miguel, F. K., Palmieri, P. A., Primi, R., & Frueh, B. C. (2010) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2010.11.003 · Cited by 115 · see citing works
Nonverbal cues for depression.
Waxer, P. H. (1974) · Journal of Abnormal Psychology · doi:10.1037/h0036706 · Cited by 109 · see citing works
Impact of depression on response to comedy: A dynamic facial coding analysis.
Reed, L. I., Sayette, M. A., & Cohn, J. F. (2007) · Journal of Abnormal Psychology · doi:10.1037/0021-843x.116.4.804 · Cited by 100 · see citing works
Marshall, G. N., Schell, T. L., & Miles, J. N. V. (2010) · Journal of Abnormal Psychology · doi:10.1037/a0018477 · Cited by 97 · see citing works
Morency, L., Whitehill, J., & Movellan, J. R. (2008) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/afgr.2008.4813429 · Cited by 97 · see citing works
Self-citationArbisi, P. A., Kaler, M. E., Kehle‐Forbes, S., Erbes, C. R., Polusny, M. A., & THURAS, P. D. (2012) · Psychological Assessment · doi:10.1037/a0028014 · Cited by 71 · see citing works
Nonverbal interaction of patients and therapists during psychiatric interviews.
Fairbanks, L. A., McGuire, M. T., & Harris, C. J. (1982) · Journal of Abnormal Psychology · doi:10.1037//0021-843x.91.2.109 · Cited by 63 · see citing works
Major Depression: Behavioral Markers of Depression and Recovery
SCHELDE, J. T. M. (1998) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199803000-00001 · Cited by 55 · see citing works
Major Depression: Behavioral Parameters of Depression and Recovery
SCHELDE, J. T. M. (1998) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-199803000-00002 · Cited by 54 · see citing works
Nonverbal attunement between depressed patients and an interviewer predicts subsequent improvement
Geerts, E., Bouhuys, N., & van den Hoofdakker, R. H. (1996) · Journal of Affective Disorders · doi:10.1016/0165-0327(96)00037-7 · Cited by 47 · see citing works
Nonverbal Social Skills and Psychopathology
Pérez, J. E., & Riggio, R. E. (2003) · Oxford University Press eBooks · doi:10.1093/med:psych/9780195141092.003.0002 · Cited by 44 · see citing works
The Effect of Two Training Procedures on Observer Agreement and Variability of Behavior Ratings
Wildman, B. G., Erickson, M. T., & Kent, R. N. (1975) · Child Development · doi:10.1111/j.1467-8624.1975.tb03342.x · Cited by 42 · see citing works
Bouhuys, A. L., & van den Hoofdakker, R. H. (1991) · Journal of Affective Disorders · doi:10.1016/0165-0327(91)90093-8 · Cited by 35 · see citing works
Recording system bias in direct observational methodology
Harris, F. C., & Lahey, B. B. (1982) · Clinical Psychology Review · doi:10.1016/0272-7358(82)90029-0 · Cited by 32 · see citing works
Kirsch, A., & Brunnhuber, S. (2007) · Psychopathology · doi:10.1159/000104779 · Cited by 30 · see citing works
Social Signal Interpretation (SSI)
Wagner, J., Lingenfelser, F., Bee, N., & André, E. (2011) · Künstliche Intell. · doi:10.1007/s13218-011-0115-x · Cited by 17 · see citing works
Examining nonverbal shame markers among post-pregnancy women with maltreatment histories
Menke, R. A. (2011) · Human Biology · Cited by 8 · see citing works
Flagged first, then by citations
2 of the 36 references couldn’t be matched to a known paper.
Cited by
Sorted by citations
A review of depression and suicide risk assessment using speech analysis
Cummins, N., Scherer, S. J., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T. F. (2015) · Speech Communication · doi:10.1016/j.specom.2015.03.004 · Cited by 1,007 · see citing works
Self-citationValstar, M., Schuller, B. W., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantić, M. (2013) · doi:10.1145/2512530.2512533 · Cited by 581 · see citing works
(2014) · doi:10.5555/2615731.2617415 · Cited by 553 · see citing works
Self-citationCorneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Valstar, M., Schuller, B. W., Smith, K., Almaev, T. R., Eyben, F., Krajewski, J., Cowie, R., & Pantić, M. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661807 · Cited by 476 · see citing works
Ringeval, F., Schuller, B., Valstar, M., Cummins, N., Cowie, R., Tavabi, L., Schmitt, M., Alisamir, S., Amiriparian, S., Messner, E.-M., et al. (2019) · Proceedings of the 9th International on Audio/Visual Emotion Challenge and Workshop · doi:10.1145/3347320.3357688 · Cited by 409 · see citing works
Deep learning for depression recognition with audiovisual cues: A review
He, L., Niu, M., Tiwari, P., Marttinen, P., Su, R., Jiang, J., Guo, C., Wang, H., Ding, S., Wang, Z., et al. (2021) · Information Fusion · doi:10.1016/j.inffus.2021.10.012 · Cited by 270 · see citing works
Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Hammal, Z., & Rosenwald, D. P. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2013.12.007 · Cited by 255 · see citing works
Automatic Assessment of Depression Based on Visual Cues: A Systematic Review
Pampouchidou, A., Simos, P. G., Kostas Marias, A., Mériaudeau, F., Yang, F., Pediaditis, M., & Tsiknakis, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2724035 · Cited by 224 · see citing works
Meng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., & Wang, Y. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512532 · Cited by 225 · see citing works
Jan, A., Meng, H., Gaus, Y. F. B. A., & Zhang, F. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2721552 · Cited by 219 · see citing works
Hire me: Computational Inference of Hirability in Employment Interviews Based on Nonverbal Behavior
Nguyen, L. S., Frauendorfer, D., Mast, M. S., & Gática-Pérez, D. (2014) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2014.2307169 · Cited by 186 · see citing works
The social signal interpretation (SSI) framework
Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., & André, E. (2013) · ACM International Conference on Multimedia (ACM MM) · doi:10.1145/2502081.2502223 · Cited by 174 · see citing works
Cicero - Towards a Multimodal Virtual Audience Platform for Public Speaking Training
Batrinca, L., Stratou, G., Shapiro, A., Morency, L., & Scherer, S. J. (2013) · Lecture notes in computer science · doi:10.1007/978-3-642-40415-3_10 · Cited by 157 · see citing works
Self-citationInvestigating voice quality as a speaker-independent indicator of depression and PTSD
Scherer, S. J., Stratou, G., Gratch, J. S., & Morency, L. (2013) · Interspeech (USB) · doi:10.21437/interspeech.2013-240 · Cited by 141 · see citing works
Self-citationDiagnosis of depression by behavioural signals
Cummins, N., Joshi, J., Dhall, A., Sethu, V., Goecke, R., & Epps, J. (2013) · doi:10.1145/2512530.2512535 · Cited by 138 · see citing works
Scherer, S. J., Lucas, G., Gratch, J. S., Rizzo, A., & Morency, L. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2440264 · Cited by 122 · see citing works
Self-citationAutomatic audiovisual behavior descriptors for psychological disorder analysis
Scherer, S. J., Stratou, G., Lucas, G., Mahmoud, M., Boberg, J., Gratch, J. S., Rizzo, A., & Morency, L. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.06.001 · Cited by 123 · see citing works
Self-citationSpeech‐based markers for posttraumatic stress disorder in US veterans
Marmar, C. R., Brown, A. D., Qian, M., LASKA, E. M., Siegel, C., Li, M., Amara, D. A., Tsiartas, A., Richey, C., Smith, J. G., et al. (2019) · Depression and Anxiety · doi:10.1002/da.22890 · Cited by 129 · see citing works
The distress analysis interview corpus of human and computer interviews
Gratch, J., Artstein, R., Lucas, G., Stratou, G., Scherer, S., Nazarian, A., Wood, R., Boberg, J., DeVault, D., Marsella, S., et al. (2014) · Proceedings of the Language Resources and Evaluation Conference · doi:10.63317/3o7bccg9xequ · Cited by 599 · see citing works
Self-citationMultitask Representation Learning for Multimodal Estimation of Depression Level
Qureshi, S. A., Saha, S., Hasanuzzaman, M., & Dias, G. H. (2019) · IEEE Intelligent Systems · doi:10.1109/mis.2019.2925204 · Cited by 112 · see citing works
Multimodal Prediction of Affective Dimensions and Depression in Human-Computer Interactions
Gupta, R., Malandrakis, N., Xiao, B., Guha, T., Segbroeck, M. V., Black, M., Potamianos, A., & Narayanan, S. S. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661810 · Cited by 110 · see citing works
Automatic Depression Scale Prediction using Facial Expression Dynamics and Regression
Jan, A., Meng, H., Gaus, Y. F. A., Zhang, F., & Turabzadeh, S. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661812 · Cited by 114 · see citing works
Zou, B., Han, J., Wang, Y., Liu, R., Zhao, S., Feng, L., Lyu, X., & Ma, H. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3181210 · Cited by 114 · see citing works
AVEC 2016 - Depression, Mood, and Emotion Recognition Workshop and ChallengePreprint
Valstar, M., Gratch, J. S., Schuller, B. W., Ringeval, F., Lalanne, D., Torres, M. T., Scherer, S. J., Stratou, G., Cowie, R., & Pantić, M. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.01600 · Cited by 121 · see citing works
Self-citationA Brief Review of Artificial Intelligence Applications and Algorithms for Psychiatric Disorders
Liu, G.-D., Li, Y., Zhang, W., & Zhang, L. (2019) · Engineering · doi:10.1016/j.eng.2019.06.008 · Cited by 102 · see citing works
Wang, J. Z., Zhao, S., Wu, C., Adams, R. B., Newman, M. G., Shafir, T., & Tsachor, R. P. (2023) · Proceedings of the IEEE · doi:10.1109/jproc.2023.3273517 · Cited by 98 · see citing works
Automated Face Analysis for Affective Computing
Calvo, R. A., D’Mello, S. K., Gratch, J. S., Kappas, A., Cohn, J. F., & la Torre, F. D. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199942237.013.020 · Cited by 82 · see citing works
Self-citationCross-cultural detection of depression from nonverbal behaviour
Alghowinem, S., Goecke, R., Cohn, J. F., Wagner, M., Parker, G. B., & Breakspear, M. J. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163113 · Cited by 77 · see citing works
Depression Estimation Using Audiovisual Features and Fisher Vector Encoding
Jain, V., Crowley, J. L., Dey, A. K., & Lux, A. (2014) · Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2661806.2661817 · Cited by 78 · see citing works
Automatic Nonverbal Behavior Indicators of Depression and PTSD: Exploring Gender Differences
Stratou, G., Scherer, S. J., Gratch, J. S., & Morency, L. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.31 · Cited by 77 · see citing works
Self-citationAudiovisual behavior descriptors for depression assessment
Scherer, S. J., Stratou, G., & Morency, L. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522886 · Cited by 73 · see citing works
Self-citationGrafsgaard, J. F., Wiggins, J. B., Vail, A. K., Boyer, K. E., Wiebe, E., & Lester, J. C. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663264 · Cited by 72 · see citing works
A Cross-modal Review of Indicators for Depression Detection Systems
Morales, M. R., Scherer, S. J., & Levitan, R. (2017) · Workshop on Computational Linguistics and Clinical Psychology (CLPsych) · doi:10.18653/v1/w17-3101 · Cited by 71 · see citing works
Self-citationInvestigating the speech characteristics of suicidal adolescents
Scherer, S. J., Pestian, J. P., & Morency, L. (2013) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2013.6637740 · Cited by 63 · see citing works
Self-citationMultiSense—Context-Aware Nonverbal Behavior Analysis Framework: A Psychological Distress Use Case
Stratou, G., & Morency, L. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2614300 · Cited by 56 · see citing works
Self-citationJiang, Z., Harati, S., Crowell, A., Mayberg, H. S., Nemati, S., & Clifford, G. D. (2021) · IEEE Transactions on Biomedical Engineering · doi:10.1109/tbme.2020.3010472 · Cited by 60 · see citing works
Facial Action Unit Detection Using Active Learning and an Efficient Non-linear Kernel Approximation
Sénéchal, T., McDuff, D., & el Kaliouby, R. (2015) · IEEE International Conference on Computer Vision Workshops · doi:10.1109/iccvw.2015.11 · Cited by 58 · see citing works
Automatic nonverbal behavior indicators of depression and PTSD: the effect of gender
Stratou, G., Scherer, S. J., Gratch, J. S., & Morency, L. (2014) · Journal on Multimodal User Interfaces · doi:10.1007/s12193-014-0161-4 · Cited by 57 · see citing works
Self-citationMultimodal assessment of depression from behavioral signals
Cohn, J. F., Cummins, N., Epps, J., Goecke, R., Joshi, J., & Scherer, S. J. (2018) · ACM eBooks · doi:10.1145/3107990.3108004 · Cited by 55 · see citing works
Self-citationAn End-to-End Model for Detection and Assessment of Depression Levels using Speech
Srimadhur, N. S., & Lalitha, S. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.04.003 · Cited by 54 · see citing works
Albuquerque, L., Valente, A. R., Teixeira, A. J. S., de Figueiredo, D. M. P., Sá‐Couto, P., & Oliveira, C. (2021) · PLoS ONE · doi:10.1371/journal.pone.0248842 · Cited by 55 · see citing works
How Can Affect Be Detected and Represented in Technological Support for Physical Rehabilitation?
Olugbade, T. A., Singh, A., Bianchi-Berthouze, N., Marquardt, N., Aung, M., & Williams, A. C. D. C. (2019) · ACM Trans. Comput. Hum. Interact. · doi:10.1145/3299095 · Cited by 43 · see citing works
Verbal indicators of psychological distress in interactive dialogue with a virtual human
DeVault, D., Georgila, K., Artstein, R., Morbini, F., Traum, D. R., Scherer, S. J., Rizzo, A., & Morency, L. (2013) · Cited by 44 · see citing works
Self-citationAutomated screening for distress: A perspective for the future
Rana, R. K., Latif, S., Gururajan, R., Gray, A. D., Mackenzie, G., Humphris, G. M., & Dunn, J. C. (2019) · European Journal of Cancer Care · doi:10.1111/ecc.13033 · Cited by 45 · see citing works
Facial expression analysis using Decomposed Multiscale Spatiotemporal Networks
de Melo, W. C., Granger, É., & López, M. B. (2023) · Expert Systems with Applications · doi:10.1016/j.eswa.2023.121276 · Cited by 43 · see citing works
Yadav, S., Chauhan, J., Sain, J. P., Thirunarayan, K., Sheth, A., & Schumm, J. A. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.61 · Cited by 44 · see citing works
Detecting depression in speech: Comparison and combination between different speech types
Long, H., Guo, Z., Wu, X., Hu, B. X., Liu, Z., & Cai, H. (2017) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm.2017.8217802 · Cited by 38 · see citing works
Designing a framework for assisting depression severity assessment from facial image analysis
Pampouchidou, A., Kostas Marias, A., Tsiknakis, M., Simos, P. G., Yang, F., & Mériaudeau, F. (2015) · IEEE International Conference on Signal and Image Processing Applications (ICSIPA) · doi:10.1109/icsipa.2015.7412257 · Cited by 37 · see citing works
Interpreting Depression From Question-Wise Long-Term Video Recording of SDS Evaluation
Xie, W., Liang, L., Lu, Y., Wang, C., Shen, J., Luo, H., & Liu, X. (2021) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2021.3092628 · Cited by 44 · see citing works
Attention-based acoustic feature fusion network for depression detection
Xu, X. D., Wang, Y., Wei, X., Wang, F., & Zhang, X. (2024) · Neurocomputing · doi:10.1016/j.neucom.2024.128209 · Cited by 35 · see citing works
Human vision inspired feature extraction for facial expression recognition
Sadeghi, H., & Raie, A. A. (2019) · Multimedia Tools and Applications · doi:10.1007/s11042-019-07863-z · Cited by 35 · see citing works
Color Analysis of Facial Skin: Detection of Emotional State
Ramírez, G. A., Fuentes, O., Crites, S. L., Jimenez, M., & Ordoñez, J. (2014) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2014.76 · Cited by 33 · see citing works
Deep peak-neutral difference feature for facial expression recognition
Chen, J., Xu, R., & Liu, L. (2018) · Multimedia Tools and Applications · doi:10.1007/s11042-018-5909-5 · Cited by 32 · see citing works
Wörtwein, T., & Scherer, S. J. (2017) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2017.8273573 · Cited by 32 · see citing works
Self-citationBirnbaum, M. L., Abrami, A., Heisig, S., Ali, A. F., Arenare, E., Agurto, C., Lu, N. Y., Kane, J. M., & Cecchi, G. (2022) · JMIR Mental Health · doi:10.2196/24699 · Cited by 31 · see citing works
Context-based signal descriptors of heart-rate variability for anxiety assessment
Chatterjee, M., Stratou, G., Scherer, S. J., & Morency, L. (2014) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp.2014.6854278 · Cited by 30 · see citing works
Self-citationKishimoto, T., Takamiya, A., Liang, K., Funaki, K., Fujita, T., Kitazawa, M., Yoshimura, M., Tazawa, Y., Horigome, T., Eguchi, Y., et al. (2020) · Contemporary Clinical Trials Communications · doi:10.1016/j.conctc.2020.100649 · Cited by 30 · see citing works
Rizzo, A., Scherer, S. J., DeVault, D., Gratch, J. S., Artstein, R., Hartholt, A., Lucas, G., Marsella, S., Morbini, F., Nazarian, A., et al. (2014) · ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) · Cited by 28 · see citing works
Self-citationBehavioral Sentiment Analysis of Depressive States
Esposito, A. A., Raimo, G., Maldonato, M. N., Vogel, C. M., Conson, M., & Cordasco, G. (2020) · IEEE International Conference on Cognitive Infocommunications · doi:10.1109/coginfocom50765.2020.9237856 · Cited by 28 · see citing works
Towards an affective interface for assessment of psychological distress
Lucas, G., Gratch, J. S., Scherer, S. J., Boberg, J., & Stratou, G. (2015) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2015.7344622 · Cited by 26 · see citing works
Self-citationRecommendations for the Use of ICT in Elderly Populations with Affective Disorders
Gros, A., Bensamoun, D., Manera, V., Fabre, R., Zacconi-Cauvin, A.-M., Thümmler, S., Benoît, M., Robert, P. H., & David, R. (2016) · Frontiers in Aging Neuroscience · doi:10.3389/fnagi.2016.00269 · Cited by 28 · see citing works
A Multimodal Context-based Approach for Distress Assessment
Ghosh, S., Chatterjee, M., & Morency, L. (2014) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2663204.2663274 · Cited by 27 · see citing works
Self-citationGender-Aware Estimation of Depression Severity Level in a Multimodal Setting
Oureshi, S. A., Dias, G. H., Saha, S., & Hasanuzzaman, M. (2021) · International Joint Conference on Neural Networks (IJCNN) · doi:10.1109/ijcnn52387.2021.9534330 · Cited by 26 · see citing works
Let Your Body Speak: Communicative Cue Extraction on Natural Interaction Using RGBD Data
Marcos-Ramiro, A., Pizarro, D., Marrón-Romera, M., & Gática-Pérez, D. (2015) · IEEE Transactions on Multimedia · doi:10.1109/tmm.2015.2464152 · Cited by 25 · see citing works
Extended Study on the Use of Vocal Tract Variables to Quantify Neuromotor Coordination in Depression
Seneviratne, N., Williamson, J. R., Lammert, A. C., Quatieri, T. F., & Espy-Wilson, C. (2020) · Interspeech (USB) · doi:10.21437/interspeech.2020-2758 · Cited by 24 · see citing works
Speech Based Depression Severity Level Classification Using a Multi-Stage Dilated CNN-LSTM Model
Seneviratne, N., & Espy-Wilson, C. (2021) · Interspeech (USB) · doi:10.21437/interspeech.2021-1967 · Cited by 24 · see citing works
Modeling Dyadic and Group Impressions with Intermodal and Interperson Features
Okada, S., Nguyen, L. S., Aran, O., & Gática-Pérez, D. (2019) · ACM Transactions on Multimedia Computing Communications and Applications · doi:10.1145/3265754 · Cited by 24 · see citing works
A multimodal adaptive session manager for physical rehabilitation exercising
Tsiakas, K., Huber, M., & Makedon, F. S. (2015) · International Conference on PErvasive Technologies Related to Assistive Environments (PETRA) · doi:10.1145/2769493.2769507 · Cited by 22 · see citing works
Onie, S., Li, X., Liang, M., Sowmya, A., & Larsen, M. E. (2021) · JMIR Mental Health · doi:10.2196/27663 · Cited by 21 · see citing works
(2016) · doi:10.1145/2988257.2988265 · Cited by 26 · see citing works
Intra- and Interpersonal Functions of Head Motion in Emotion Communication
Hammal, Z., & Cohn, J. F. (2014) · doi:10.1145/2666253.2666258 · Cited by 19 · see citing works
Yu, Z., Scherer, S., DeVault, D., Gratch, J. S., Stratou, G., Morency, L., & Cassell, J. (2013) · Cited by 19 · see citing works
Self-citationDecoding depressive disorder using computer vision
Singh, J., & Goyal, G. (2020) · Multimedia Tools and Applications · doi:10.1007/s11042-020-10128-9 · Cited by 19 · see citing works
Qureshi, S. A., Hasanuzzaman, M., Saha, S., & Dias, G. H. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.07656 · Cited by 19 · see citing works
Looking at the Body: Automatic Analysis of Body Gestures and Self-Adaptors in Psychological Distress
Lin, W., Orton, I., Li, Q., Pavarini, G., & Mahmoud, M. (2021) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2021.3101698 · Cited by 17 · see citing works
Self-citationAutomatic multimodal descriptors of rhythmic body movement
Mahmoud, M., Morency, L., & Robinson, P. (2013) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/2522848.2522895 · Cited by 17 · see citing works
Self-citationToward Visual Behavior Markers of Suicidal Ideation
Eigbe, N., Baltrušaitis, T., Morency, L., & Pestian, J. P. (2018) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2018.00085 · Cited by 16 · see citing works
Self-citationAn historical overview of artificial intelligence for diagnosis of major depressive disorder
Liu, H., Wu, H., Yang, Z., Ren, Z., Dong, Y., Zhang, G., & Li, M. D. (2024) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1417253 · Cited by 16 · see citing works
A Demonstration of Dialogue Processing in SimSensei Kiosk
Morbini, F., DeVault, D., Georgila, K., Artstein, R., Traum, D. R., & Morency, L. (2014) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.3115/v1/w14-4334 · Cited by 15 · see citing works
Self-citationGamification to Improve Adherence to Clinical Treatment Advice
Richards, D., & Caldwell, P. H. Y. (2015) · Advances in medical technologies and clinical practice book series · doi:10.4018/978-1-4666-9522-1.ch004 · Cited by 15 · see citing works
Deep Learning for Depression Recognition with Audiovisual Cues: A ReviewPreprint
He, L., Niu, M., Tiwari, P., Marttinen, P., Su, R., Jiang, J., Guo, C., Wang, H., Ding, S., Wang, Z., et al. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.00610 · Cited by 15 · see citing works
Gupta, R. K., & Sinha, R. (2023) · IEEE Access · doi:10.1109/access.2023.3297091 · Cited by 14 · see citing works
An Emerging Decision Authority
Töniges, T., Ötting, S. K., Wrede, B., Maier, G. W., & Sagerer, G. (2016) · Elsevier eBooks · doi:10.1016/b978-0-12-803801-7.00026-2 · Cited by 14 · see citing works
Local Second-Order Gradient Cross Pattern for Automatic Depression Detection
Niu, M., Tao, J., & Liu, B. (2019) · International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW) · doi:10.1109/aciiw.2019.8925158 · Cited by 14 · see citing works
Multimodal analysis of vocal collaborative search: a public corpus and results
McDuff, D., Thomas, P., Czerwinski, M. P., & Craswell, N. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3136813 · Cited by 11 · see citing works
Hinduja, S., Darzi, A., Ertuğrul, I. Ö., Provenza, N. R., Gadot, R., Storch, E. A., Sheth, S. A., Goodman, W. K., & Cohn, J. F. (2024) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2024.3395117 · Cited by 10 · see citing works
Domain Adversarial Training for Mitigating Gender Bias in Speech-based Mental Health Detection
Kim, J.-W., Yoon, H., Oh, W., Jung, D., Yoon, S.-H., Kim, D.-J., Lee, D., Lee, S.-Y., & Yang, C.-M. (2025) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc58623.2025.11253190 · Cited by 10 · see citing works
Detecting emotional reactions to videos of depression
Zhu, X., Gedeon, T., Caldwell, S., & Jones, R. (2019) · IEEE International Conference on Intelligent Engineering Systems (INES) · doi:10.1109/ines46365.2019.9109519 · Cited by 9 · see citing works
Computer-Based PTSD Assessment in VR Exposure Therapy
Tavabi, L., Poon, A., Rizzo, A., & Soleymani, M. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-59990-4_32 · Cited by 8 · see citing works
Self-citationAutomatic Analysis of Bodily Social Signals
Poppe, R. W. (2017) · Cambridge University Press eBooks · doi:10.1017/9781316676202.012 · Cited by 8 · see citing works
Automatic Detection of Self-Adaptors for Psychological Distress
Lin, W., Orton, I., Liu, M., & Mahmoud, M. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00032 · Cited by 8 · see citing works
Self-citationTowards Identifying Fine-Grained Depression Symptoms from Memes
Yadav, S., Caragea, C., Zhao, C., Kumari, N. P., Solberg, M. A., & Sharma, T. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.495 · Cited by 6 · see citing works
A Multi-Frame Rate Network with Attention Mechanism for Depression Severity Estimation
Wang, R., Guo, J. M., Wang, J., He, L., & Yang, Y. (2023) · IEEE International Conference on Bioinformatics and Biomedicine (BIBM) · doi:10.1109/bibm58861.2023.10385423 · Cited by 6 · see citing works
Speech Analysis as a Tool for Detection and Monitoring of Medical Conditions: A review
Igraś-Cybulska, M., Hemmerling, D., Ziółko, M., Datka, W., Stogowska, E., Kucharski, M., Rzepka, R., & Ziółko, B. (2023) · Archives of Acoustics · doi:10.24425/aoa.2023.146640 · Cited by 7 · see citing works
Gamification to Improve Adherence to Clinical Treatment Advice
Richards, D., & Caldwell, P. H. Y. (2017) · IGI Global eBooks · doi:10.4018/978-1-5225-1928-7.ch005 · Cited by 6 · see citing works
The Nonverbal Visual Cues to Deception
Liang, J., Kaiyun, L. I., Qu, F., Yu-Hsin, C. H. E. N., Yan, W., & Fu, X. (2014) · Advances in Psychological Science · doi:10.3724/sp.j.1042.2014.00995 · Cited by 5 · see citing works
Deep Learning Based Generalized Models for Depression Classification.Preprint
Seneviratne, N., & Espy-Wilson, C. (2020) · arXiv (Cornell University) · Cited by 5 · see citing works
Deep multi-task learning based detection of correlated mental disorders using audio modality
Gupta, R. K., & Sinha, R. (2024) · Computer Speech & Language · doi:10.1016/j.csl.2024.101710 · Cited by 4 · see citing works
- Authors
- Scherer, S., Stratou, G., Mahmoud, M., Boberg, J., Gratch, J., Rizzo, A., & Morency, L.-P.
- Affiliations
- University of Southern California (US)
- Venue
- 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG)
- Publisher
- IEEE
- Published
- 2013-04
- Pages
- 1-8
- ISSN
- 1541-5058 (ISSN-L)
- OpenAlex
- W1981231682
- Cited by
- 147
- References
- 36
- Citations by year
- 2026: 2 · 2025: 5 · 2024: 7 · 2023: 15 · 2022: 8 · 2021: 12 · 2020: 14 · 2019: 19 · 2018: 5 · 2017: 8 · 2016: 8 · 2015: 10 · 2014: 20 · 2013: 14
- FWCI
- 23.59
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- International Conference on Automatic Face and Gesture Recognition (publishedVersion)
- ENLIGHTEN (Jurnal Bimbingan dan Konseling Islam) (submittedVersion)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Social Sciences › Psychology › Social Psychology › Mental Health via Writing
- Social Sciences › Psychology › Social Psychology › Deception detection and forensic psychology
- Keywords
- depression, anxiety, post-traumatic stress disorder, dyadic interaction, visual signals
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review