Diagnosis of depression by behavioural signals: a multimodal approach
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual's level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Projections of Global Mortality and Burden of Disease from 2002 to 2030
Mathers, C. D., & Loncar, D. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030442 · Cited by 11,386
Opensmile: the munich versatile and fast open-source audio feature extractor
Eyben, F., Wöllmer, M., & Schuller, B. (2010) · Proceedings of the 18th ACM international conference on Multimedia · doi:10.1145/1873951.1874246 · Cited by 2,642 · cited 1×
“The experimental settings (unless otherwise stated) of the classification system were as follows: a 39-dimensional feature vector was formed from thirteen MFCCs, including C 0 , extracted using the openSMILE toolkit [42], every 10ms using a 25ms window.”
Emotion recognition in human-computer interaction
Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W., & Taylor, J. G. (2001) · IEEE Signal Processing Magazine · doi:10.1109/79.911197 · Cited by 2,603
Depression: Perspectives from Affective Neuroscience
Davidson, R. J., Pizzagalli, D., Nitschke, J. B., & Putnam, K. (2002) · Annual Review of Psychology · doi:10.1146/annurev.psych.53.100901.135148 · Cited by 1,138
Psychomotor symptoms of depression
(1997) · American Journal of Psychiatry · doi:10.1176/ajp.154.1.4 · Cited by 662 · cited 2×
“This leads to depressed speech having longer pauses, decreased utterance length and a reduced speech rate [14], [15].”
Acoustical properties of speech as indicators of depression and suicidal risk
France, D. J., Shiavi, R. G., Silverman, S., Silverman, M., & Wilkes, M. (2000) · IEEE Transactions on Biomedical Engineering · doi:10.1109/10.846676 · Cited by 504
Detecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494
Nicolaou, M. A., Gunes, H., & Pantic, M. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.9 · Cited by 450 · cited 2×
“Recent studies have shown that audio cues are often better at recognizing arousal, whilst visual cues correlate better with valence [38].”
Bi-modal emotion recognition from expressive face and body gestures
Gunes, H., & Piccardi, M. (2007) · Journal of Network and Computer Applications · doi:10.1016/j.jnca.2006.09.007 · Cited by 382
Mundt, J. C., Snyder, P. J., Cannizzaro, M. S., Chappie, K., & Geralts, D. S. (2007) · Journal of Neurolinguistics · doi:10.1016/j.jneuroling.2006.04.001 · Cited by 380 · cited 2×
“In practice, these tests are subjective in nature, requiring a large degree of clinical training to produce acceptable results, and can be conducted only infrequently [4].”
Cited by
A review of depression and suicide risk assessment using speech analysis
Cummins, N., Scherer, S., Krajewski, J., Schnieder, S., Epps, J., & Quatieri, T. F. (2015) · Speech Communication · doi:10.1016/j.specom.2015.03.004 · Cited by 1,003
Self-citationDepAudioNet: An Efficient Deep Model for Audio based Depression Classification
Ma, X., Yang, H., Chen, Q., Huang, D., & Wang, Y. (2016) · Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2988257.2988267 · Cited by 298
Visually Interpretable Representation Learning for Depression Recognition from Facial Images
Zhou, X., Jin, K., Shang, Y., & Guo, G. (2020) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2018.2828819 · Cited by 271
Automated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics
Zhu, Y., Shang, Y., Shao, Z., & Guo, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2650899 · Cited by 257
Deep learning for depression recognition with audiovisual cues: A review
He, L., Niu, M., Tiwari, P., Marttinen, P., Su, R., Jiang, J., Guo, C., Wang, H., Ding, S., Wang, Z., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.10.012 · Cited by 253
Metadata
- Authors
- Cummins, N., Joshi, J., Dhall, A., Sethu, V., Goecke, R., & Epps, J.
- Venue
- Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge
- Publisher
- ACM
- Published
- 2013-10-21
- Pages
- 11-20
- OpenAlex
- W2027192823
- Cited by
- 139
- References
- 12
- FWCI
- 14.53
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Diagnosis of depression by behavioural signals: a multimodal approach. https://app.cochat.ai/papers/10.1145/2512530.2512535
- The paper
- Cummins, N., Joshi, J., Dhall, A., Sethu, V., Goecke, R., & Epps, J. (2013). Diagnosis of depression by behavioural signals: a multimodal approach. Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge. https://doi.org/10.1145/2512530.2512535
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.