The DOI matches its Crossref record; 50 of the 50 references it cites were found in a registry or an index.
Visually Interpretable Representation Learning for Depression Recognition from Facial Images
Abstract
Recent evidence in mental health assessment have demonstrated that facial appearance could be highly indicative of depressive disorder. While previous methods based on the facial analysis promise to advance clinical diagnosis of depressive disorder in a more efficient and objective manner, challenges in visual representation of complex depression pattern prevent widespread practice of automated depression diagnosis. In this paper, we present a deep regression network termed DepressNet to learn a depression representation with visual explanation. Specifically, a deep convolutional neural network equipped with a global average pooling layer is first trained with facial depression data, which allows for identifying salient regions of input image in terms of its severity score based on the generated depression activation map (DAM). We then propose a multi-region DepressNet, with which multiple local deep regression models for different face regions are jointly leaned and their responses are fused to improve the overall recognition performance. We evaluate our method on two benchmark datasets, and the results show that our method significantly boosts state-of-the-art performance of the visual-based depression recognition. Most importantly, the DAM induced by our learned deep model may help reveal the visual depression pattern on faces and understand the insights of automated depression diagnosis.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Deep Residual Learning for Image Recognition
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.90 · Cited by 235,474
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Visualizing and understanding convolutional networks
Zeiler, M. D., & Fergus, R. (2014) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-10590-1_53 · Cited by 15,839
Multiresolution gray-scale and rotation invariant texture classification with local binary patterns
Ojala, T., Pietikainen, M., & Maenpaa, T. (2002) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2002.1017623 · Cited by 15,398
Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex
Hubel, D. H., & Wiesel, T. N. (1962) · The Journal of Physiology · doi:10.1113/jphysiol.1962.sp006837 · Cited by 13,680
A tutorial on support vector regression
Smola, A. J., & Schölkopf, B. (2004) · Statistics and Computing · doi:10.1023/b:stco.0000035301.49549.88 · Cited by 13,387
Vos, T., Abajobir, A. A., Abate, K. H., Abbafati, C., Abbas, K. M., Abd-Allah, F., Abdulkader, R. S., Abdulle, A. M., Abebo, T. A., Abera, S. F., et al. (2017) · The Lancet · doi:10.1016/s0140-6736(17)32154-2 · Cited by 13,220
TensorFlow: A system for large-scale machine learning
Abadi, M., Barham, P., Chen, J., Chen, Z., Davis, A., Dean, J., Devin, M., Ghemawat, S., Irving, G., Isard, M., et al. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1605.08695 · Cited by 11,822
Learning Deep Features for Discriminative Localization
Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., & Torralba, A. (2016) · 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2016.319 · Cited by 11,428
Identity mappings in deep residual networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-46493-0_38 · Cited by 10,741
Identity Mappings in Deep Residual Networks
He, K., Zhang, X., Ren, S., & Sun, J. (2016) · arXiv · doi:10.48550/arxiv.1603.05027 · Cited by 10,665
Partial least-squares regression: a tutorial
Geladi, P., & Kowalski, B. R. (1986) · Analytica Chimica Acta · doi:10.1016/0003-2670(86)80028-9 · Cited by 6,992
DeepFace: Closing the Gap to Human-Level Performance in Face Verification
Taigman, Y., Yang, M., Ranzato, M., & Wolf, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.220 · Cited by 6,542
Vos, T., Barber, R. M., Bell, B., Bertozzi-Villa, A., Biryukov, S., Bolliger, I., Charlson, F., Davis, A., Degenhardt, L., Dicker, D., et al. (2015) · The Lancet · doi:10.1016/s0140-6736(15)60692-4 · Cited by 6,414
Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients
Beck, A. T., Steer, R. A., Ball, R., & Ranieri, W. F. (1996) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6703_13 · Cited by 5,965
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
Simonyan, K., Vedaldi, A., & Zisserman, A. (2013) · arXiv · doi:10.48550/arxiv.1312.6034 · Cited by 5,370
Learning realistic human actions from movies
Laptev, I., Marszalek, M., Schmid, C., & Rozenfeld, B. (2008) · 2008 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2008.4587756 · Cited by 3,444
Dlib-ml: A Machine Learning Toolkit
King, D. E. (2009) · doi:10.5555/1577069.1755843 · Cited by 2,777
Belmaker, R. H., & Agam, G. (2008) · New England Journal of Medicine · doi:10.1056/nejmra073096 · Cited by 2,166
SIMPLS: An alternative approach to partial least squares regression
de Jong, S. (1993) · Chemometrics and Intelligent Laboratory Systems · doi:10.1016/0169-7439(93)85002-x · Cited by 1,905 · cited 1×
“[8] adopted motion history histogram (MHH) [25] based feature to encode motion cue from videos, and the partial least squares (PLS) [30] was”
MethodUnderstanding deep image representations by inverting them
Mahendran, A., & Vedaldi, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7299155 · Cited by 1,827
Learning Face Representation from Scratch
Yi, D., Lei, Z., Liao, S., & Li, S. Z. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1411.7923 · Cited by 1,521
Reliable Crowdsourcing and Deep Locality-Preserving Learning for Expression Recognition in the Wild
Li, S., Deng, W., & Du, J. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.277 · Cited by 1,470
Representing shape with a spatial pyramid kernel
Bosch, A., Zisserman, A., & Munoz, X. (2007) · Proceedings of the 6th ACM international conference on Image and video retrieval · doi:10.1145/1282280.1282340 · Cited by 1,399
Contractive Auto-Encoders: Explicit Invariance During Feature Extraction
Rifai, S., Vincent, P., Muller, X., Glorot, X., & Bengio, Y. (2011) · International Conference on Machine Learning · Cited by 1,190
Disentangled Representation Learning GAN for Pose-Invariant Face Recognition
Tran, L., Yin, X., & Liu, X. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2017.141 · Cited by 1,005
AVEC 2013: the continuous audio/visual emotion and depression recognition challenge
Valstar, M., Schuller, B., Smith, K., Eyben, F., Jiang, B., Bilakhia, S., Schnieder, S., Cowie, R., & Pantic, M. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512533 · Cited by 581
Detecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494
AVEC 2014: 3D Dimensional Affect and Depression Recognition Challenge
Valstar, M., Schuller, B., Smith, K., Almaev, T., Eyben, F., Krajewski, J., Cowie, R., & Pantic, M. (2014) · Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2661806.2661807 · Cited by 472
Learning Compact Binary Face Descriptor for Face Recognition
Lu, J., Liong, V. E., Zhou, X., & Zhou, J. (2015) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2015.2408359 · Cited by 399
Self-citationDepAudioNet: An Efficient Deep Model for Audio based Depression Classification
Ma, X., Yang, H., Chen, Q., Huang, D., & Wang, Y. (2016) · Proceedings of the 6th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2988257.2988267 · Cited by 298
Detecting Depression Severity from Vocal Prosody
Yang, Y., Fairbairn, C., & Cohn, J. F. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.38 · Cited by 277
Automated Depression Diagnosis Based on Deep Networks to Encode Facial Appearance and Dynamics
Zhu, Y., Shang, Y., Shao, Z., & Guo, G. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2650899 · Cited by 257
Self-citationNonverbal social withdrawal in depression: Evidence from manual and automatic analyses
Girard, J. M., Cohn, J. F., Mahoor, M. H., Mavadati, S. M., Hammal, Z., & Rosenwald, D. P. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2013.12.007 · Cited by 255
Meng, H., Huang, D., Wang, H., Yang, H., AI-Shuraifi, M., & Wang, Y. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512532 · Cited by 225 · cited 6×
“[8] adopted motion history histogram (MHH) [25] based feature to encode motion cue from videos, and the partial least squares (PLS) [30] was”
BackgroundMethodInfluentialJan, A., Meng, H., Gaus, Y. F. B. A., & Zhang, F. (2018) · IEEE Transactions on Cognitive and Developmental Systems · doi:10.1109/tcds.2017.2721552 · Cited by 219
Multimodal Depression Detection: Fusion Analysis of Paralinguistic, Head Pose and Eye Gaze Behaviors
Alghowinem, S., Goecke, R., Wagner, M., Epps, J., Hyett, M., Parker, G., & Breakspear, M. (2018) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2634527 · Cited by 204
Local Gabor Binary Patterns from Three Orthogonal Planes for Automatic Facial Expression Recognition
Almaev, T. R., & Valstar, M. F. (2013) · 2013 Humaine Association Conference on Affective Computing and Intelligent Interaction · doi:10.1109/acii.2013.65 · Cited by 201
Vocal biomarkers of depression based on motor incoordination
Williamson, J. R., Quatieri, T. F., Helfer, B. S., Horwitz, R., Yu, B., & Mehta, D. D. (2013) · Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge · doi:10.1145/2512530.2512531 · Cited by 183
Wang, Z., & Yang, J. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1703.10757 · Cited by 144
Automated Depression Diagnosis Based on Facial Dynamic Analysis and Sparse Coding
Wen, L., Li, X., Guo, G., & Zhu, Y. M. (2015) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2015.2414392 · Cited by 142
Self-citationDiagnosis of depression by behavioural signals: a multimodal approach
Cummins, N., Joshi, J., Dhall, A., Sethu, V., Goecke, R., & Epps, J. (2013) · doi:10.1145/2512530.2512535 · Cited by 140
Automatic Depression Scale Prediction using Facial Expression Dynamics and Regression
Jan, A., Meng, H., Gaus, Y. F. A., Zhang, F., & Turabzadeh, S. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661812 · Cited by 115
Ensemble CCA for Continuous Emotion Prediction
Kaya, H., Çilli, F., & Salah, A. A. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661814 · Cited by 100
Pérez-Espinosa, H., Escalante, H. J., Villaseñor-Pineda, L., Montes-y-Gómez, M., Pinto, D., & Reyes‐Meza, V. (2014) · International Workshop on Audio/Visual Emotion Challenge (AVEC) · doi:10.1145/2661806.2661815 · Cited by 60
Emotion Recognition and Depression Diagnosis by Acoustic and Visual Features: A Multimodal Approach
Sidorov, M., & Minker, W. (2014) · Proceedings of the 4th International Workshop on Audio/Visual Emotion Challenge · doi:10.1145/2661806.2661816 · Cited by 56
Descriptive temporal template features for visual motion recognition
Meng, H., & Pears, N. E. (2009) · Pattern Recognition Letters · doi:10.1016/j.patrec.2009.03.003 · Cited by 51
Cited by
Deep Facial Expression Recognition: A Survey
Li, S., & Deng, W. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.2981446 · Cited by 1,753
Deep learning for depression recognition with audiovisual cues: A review
He, L., Niu, M., Tiwari, P., Marttinen, P., Su, R., Jiang, J., Guo, C., Wang, H., Ding, S., Wang, Z., et al. (2022) · Information Fusion · doi:10.1016/j.inffus.2021.10.012 · Cited by 253
Fan, H., Zhang, X., Xu, Y., Fang, J., Zhang, S., Zhao, X., & Yu, J. (2024) · Information Fusion · doi:10.1016/j.inffus.2023.102161 · Cited by 137
A channel-spatial-temporal attention-based network for vibration-based damage detection
Liao, S., Liu, H., Yang, J., & Ge, Y. (2022) · Information Sciences · doi:10.1016/j.ins.2022.05.042 · Cited by 31
Metadata
- Authors
- Zhou, X., Jin, K., Shang, Y., & Guo, G.
- Venue
- IEEE Transactions on Affective Computing
- Published
- 2020-07-01
- Volume
- 11
- Issue
- 3
- Pages
- 542-552
- ISSN
- 1949-3045 (ISSN-L)
- 2371-9850
- OpenAlex
- W2800833042
- Cited by
- 271
- References
- 50
- FWCI
- 19.13
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.