The DOI matches its Crossref record, with a small difference in title or year; 69 of the 69 references it cites were found in a registry or an index.
Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification
Open accessAbstract
Pain is an unpleasant feeling that has been shown to be an important factor for the recovery of patients. Since this is costly in human resources and difficult to do objectively, there is the need for automatic systems to measure it. In this paper, contrary to current state-of-the-art techniques in pain assessment, which are based on facial features only, we suggest that the performance can be enhanced by feeding the raw frames to deep learning models, outperforming the latest state-of-the-art results while also directly facing the problem of imbalanced data. As a baseline, our approach first uses convolutional neural networks (CNNs) to learn facial features from VGG_Faces, which are then linked to a long short-term memory to exploit the temporal relation between video frames. We further compare the performances of using the so popular schema based on the canonically normalized appearance versus taking into account the whole image. As a result, we outperform current state-of-the-art area under the curve performance in the UNBC-McMaster Shoulder Pain Expression Archive Database. In addition, to evaluate the generalization properties of our proposed methodology on facial motion recognition, we also report competitive results in the Cohn Kanade+ facial expression database.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“More advanced face frontalization involving the approximation of 3D shape is useful when dealing with 3D head pose variation [104], [105].”
From Deep learning for human affect recognition: Insights and new developments (2021) · cites this paper as [105]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Very Deep Convolutional Networks for Large-Scale Image Recognition
Simonyan, K., & Zisserman, A. (2014) · arXiv · doi:10.48550/arxiv.1409.1556 · Cited by 114,148
ImageNet classification with deep convolutional neural networks
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017) · Communications of the ACM · doi:10.1145/3065386 · Cited by 112,155
Hochreiter, S., & Schmidhuber, J. (1997) · Neural Computation · doi:10.1162/neco.1997.9.8.1735 · Cited by 102,318
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946
ImageNet: A large-scale hierarchical image database
Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009) · 2009 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2009.5206848 · Cited by 65,333
Going deeper with convolutions
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., & Rabinovich, A. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) · doi:10.1109/cvpr.2015.7298594 · Cited by 48,562
Learning representations by back-propagating errors
Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986) · Nature · doi:10.1038/323533a0 · Cited by 31,892
Backpropagation Applied to Handwritten Zip Code Recognition
LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D. (1989) · Neural Computation · doi:10.1162/neco.1989.1.4.541 · Cited by 11,891
Learning Deep Architectures for AI
Bengio, Y. (2009) · Foundations and Trends® in Machine Learning · doi:10.1561/2200000006 · Cited by 7,330
3D Convolutional Neural Networks for Human Action Recognition
Ji, S., Xu, W., Yang, M., & Yu, K. (2012) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2012.59 · Cited by 6,313
Research works printed but not linked: 1
- Ekman, 2002, Facial Action Coding System—The Manual on CD-ROM
Cited by
Sorted by citations
Facial Expression Analysis under Partial Occlusion
Zhang, L., Verma, B., Tjondronegoro, D., & Chandran, V. (2018) · ACM Computing Surveys · doi:10.1145/3158369 · Cited by 1,338
Khare, S. K., Blanes‐Vidal, V., Nadimi, E. S., & Acharya, U. R. (2023) · Information Fusion · doi:10.1016/j.inffus.2023.102019 · Cited by 508
Attention mechanism-based CNN for facial expression recognition
Li, J., Jin, K., Zhou, D., Kubota, N., & Ju, Z. (2020) · Neurocomputing · doi:10.1016/j.neucom.2020.06.014 · Cited by 368
Graph-Based Deep Learning for Medical Diagnosis and Analysis: Past, Present and Future
Ahmedt‐Aristizabal, D., Armin, M. A., Denman, S., Fookes, C., & Petersson, L. (2021) · Sensors · doi:10.3390/s21144758 · Cited by 298
Automatic Recognition Methods Supporting Pain Assessment: A Survey
Werner, P., Lopez-Martinez, D., Walter, S., Al-Hamadi, A. K., Gruss, S., & Picard, R. W. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2946774 · Cited by 262
Metadata
- Authors
- Rodriguez, P., Cucurull, G., Gonzalez, J., Gonfaus, J. M., Nasrollahi, K., Moeslund, T. B., & Roca, F. X.
- Affiliations
- Computer Vision Center, Universitat Autònoma de Barcelona, Barcelona, Spain
- Universitat Autònoma de Barcelona (ES)
- Aalborg University (DK)
- Venue
- IEEE Transactions on Cybernetics
- Published
- 2022-05
- Volume
- 52
- Issue
- 5
- Pages
- 3314-3324
- ISSN
- 2168-2267 (ISSN-L)
- 2168-2275
- OpenAlex
- W2587128043
- Cited by
- 297
- References
- 69
- Citations by year
- 2026: 12 · 2025: 28 · 2024: 33 · 2023: 32 · 2022: 26
- FWCI
- 43.48
- Type
- journal article
- Language
- en
- Open access
- green
- vbn.aau.dk
- Licence
- Locations
- IEEE Transactions on Cybernetics (publishedVersion)
- PubMed (publishedVersion)
- VBN Forskningsportal (Aalborg Universitet) (acceptedVersion, other-oa)
- Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona) (submittedVersion, public-domain)
- Funders
- Spanish Project (TIN2015-65464-R)
- Generalitat de Catalunya (2016FI_B 01163)
- COST Action IC1307 iV&L Net (European Network on Integrating Vision and Language) through COST
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- pain assessment, facial expression recognition, long short-term memory networks, deep pain, video sequences, temporal modeling, class imbalance
- MeSH terms
- Emotions, Facial Expression, Humans, Memory, Short-Term, Pain, Neural Networks, Computer
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
