Cited by 297Emotion and Mood RecognitionFace recognition and analysisFace and Expression Recognition

Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification

Open access
Rodriguez, Pau, Cucurull, Guillem, Gonzalez, Jordi, Gonfaus, Josep M., Nasrollahi, Kamal, Moeslund, Thomas B., Roca, F. Xavier·2022·IEEE Transactions on Cybernetics

Abstract

Pain is an unpleasant feeling that has been shown to be an important factor for the recovery of patients. Since this is costly in human resources and difficult to do objectively, there is the need for automatic systems to measure it. In this paper, contrary to current state-of-the-art techniques in pain assessment, which are based on facial features only, we suggest that the performance can be enhanced by feeding the raw frames to deep learning models, outperforming the latest state-of-the-art results while also directly facing the problem of imbalanced data. As a baseline, our approach first uses convolutional neural networks (CNNs) to learn facial features from VGG_Faces, which are then linked to a long short-term memory to exploit the temporal relation between video frames. We further compare the performances of using the so popular schema based on the canonically normalized appearance versus taking into account the whole image. As a result, we outperform current state-of-the-art area under the curve performance in the UNBC-McMaster Shoulder Pain Expression Archive Database. In addition, to evaluate the generalization properties of our proposed methodology on facial motion recognition, we also report competitive results in the Cohn Kanade+ facial expression database.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1109/tcyb.2017.2662199

Cites

69 of 69 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • More advanced face frontalization involving approximation of 3D shape is useful when dealing with 3D head pose variation.

    “More advanced face frontalization involving the approximation of 3D shape is useful when dealing with 3D head pose variation [104], [105].”

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Rodriguez, P., Cucurull, G., Gonzalez, J., Gonfaus, J. M., Nasrollahi, K., Moeslund, T. B., & Roca, F. X.
Affiliations
  • Computer Vision Center, Universitat Autònoma de Barcelona, Barcelona, Spain
  • Universitat Autònoma de Barcelona (ES)
  • Aalborg University (DK)
Venue
IEEE Transactions on Cybernetics
Published
2022-05
Volume
52
Issue
5
Pages
3314-3324
ISSN
  • 2168-2267 (ISSN-L)
  • 2168-2275
OpenAlex
W2587128043
Cited by
297
References
69
Citations by year
2026: 12 · 2025: 28 · 2024: 33 · 2023: 32 · 2022: 26 · 2021: 43 · 2020: 46 · 2019: 42 · 2018: 28 · 2017: 6
FWCI
43.48
Type
journal article
Language
en
Open access
Locations
  • IEEE Transactions on Cybernetics (publishedVersion)
  • PubMed (publishedVersion)
  • VBN Forskningsportal (Aalborg Universitet) (acceptedVersion, other-oa)
  • Dipòsit Digital de Documents de la UAB (Universitat Autònoma de Barcelona) (submittedVersion, public-domain)
Funders
  • Spanish Project (TIN2015-65464-R)
  • Generalitat de Catalunya (2016FI_B 01163)
  • COST Action IC1307 iV&L Net (European Network on Integrating Vision and Language) through COST
Topics
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
Keywords
pain assessment, facial expression recognition, long short-term memory networks, deep pain, video sequences, temporal modeling, class imbalance
MeSH terms
Emotions, Facial Expression, Humans, Memory, Short-Term, Pain, Neural Networks, Computer

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review