The painful face – Pain expression recognition using active appearance models
Open accessAbstract
Pain is typically assessed by patient self-report. Self-reported pain, however, is difficult to interpret and may be impaired or in some circumstances (i.e., young children and the severely ill) not even possible. To circumvent these problems behavioral scientists have identified reliable and valid facial indicators of pain. Hitherto, these methods have required manual measurement by highly skilled human observers. In this paper we explore an approach for automatically recognizing acute pain without the need for human observers. Specifically, our study was restricted to automatically detecting pain in adult patients with rotator cuff injuries. The system employed video input of the patients as they moved their affected and unaffected shoulder. Two types of ground truth were considered. Sequence-level ground truth consisted of Likert-type ratings by skilled observers. Frame-level ground truth was calculated from presence/absence and intensity of facial actions previously associated with pain. Active appearance models (AAM) were used to decouple shape and appearance in the digitized face images. Support vector machines (SVM) were compared for several representations from the AAM and of ground truth of varying granularity. We explored two questions pertinent to the construction, design and development of automatic pain detection systems. First, at what level (i.e., sequence- or frame-level) should datasets be labeled in order to obtain satisfactory automatic pain detection performance? Second, how important is it, at both levels of labeling, that we non-rigidly register the face?
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Seel, N. M. (2012) · Springer eBooks · doi:10.1007/978-1-4419-1428-6_5195 · Cited by 19,525 · see citing works
Robust Real-Time Face Detection
Viola, P., & Jones, M. J. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000013087.49260.fb · Cited by 14,097 · see citing works
The expression of the emotions in man and animals.
Darwin, C. D. (1872) · J. Murray eBooks · doi:10.1037/10001-000 · Cited by 12,120 · see citing works
Russell, E. L., Chiang, L. H., & Braatz, R. D. (2000) · Advances in industrial control · doi:10.1007/978-1-4471-0409-4_3 · Cited by 7,938 · see citing works
Ekman, P., & Friesen, W. V. (2019) · PsycTESTS Dataset · doi:10.1037/t27734-000 · Cited by 6,019 · see citing works
Cootes, T. F., Edwards, G. J., & Taylor, C. J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.927467 · Cited by 5,409 · see citing works
A Practical Guide to Support Vector Classication
Hsu, C., Chang, C.-C., & Lin, C. (2008) · Cited by 5,070 · see citing works
Benjamin, S. (1994) · Journal of Psychosomatic Research · doi:10.1016/0022-3999(94)90012-4 · Cited by 2,006 · see citing works
Li, S. Z., Jain, A. K., Deng, J., & service), S. (. (2011) · Springer eBooks · doi:10.1007/978-0-85729-932-1 · Cited by 1,849 · see citing works
Active Appearance Models Revisited
Matthews, I., & Baker, S. (2004) · International Journal of Computer Vision · doi:10.1023/b:visi.0000029666.37597.d3 · Cited by 1,538 · see citing works
Williams, A. C. D. C. (1995) · Journal of Psychosomatic Research · doi:10.1016/0022-3999(95)90016-0 · Cited by 1,475 · see citing works
Sierles, F. S. (1982) · Springer eBooks · doi:10.1007/978-94-011-7973-7_4 · Cited by 992 · see citing works
Li, S. Z., & Jain, A. K. (2005) · Springer eBooks · doi:10.1007/b138828 · Cited by 791 · see citing works
Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2005.297 · Cited by 632 · see citing works
Pantic, M., & Patras, I. (2006) · IEEE Transactions on Systems, Man and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2005.859075 · Cited by 580 · see citing works
The consistency of facial expressions of pain: a comparison across modalities
Prkachin, K. M. (1992) · Pain · doi:10.1016/0304-3959(92)90213-u · Cited by 532 · see citing works
Self-citationHandbook of methods in nonverbal behavior research
Scherer, K. R., & Ekman, P. (1982) · Cited by 515 · see citing works
Prkachin, K. M., & Solomon, P. E. (2008) · Pain · doi:10.1016/j.pain.2008.04.010 · Cited by 472 · see citing works
Self-citationSimple pain rating scales hide complex idiosyncratic meanings
Williams, A. C. D. C., Davies, H. T. O., & Chadury, Y. (2000) · Pain · doi:10.1016/s0304-3959(99)00299-7 · Cited by 450 · see citing works
Facial Action Recognition for Facial Expression Analysis From Static Face Images
Pantić, M., & Rothkrantz, L. J. M. (2004) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2004.825931 · Cited by 419 · see citing works
Tian, Y., Takeo Kanade, A., & Cohn, J. F. (2005) · Springer eBooks · doi:10.1007/0-387-27257-7_12 · Cited by 407 · see citing works
Self-citationObserver-Based Measurement of Facial Expression with the Facial Action Coding System
Cohn, J. F., Ambadar, Z., & Ekman, P. (2007) · doi:10.1093/oso/9780195169157.003.0014 · Cited by 398 · see citing works
Self-citationFacial Action Unit Recognition by Exploiting Their Dynamic and Semantic Relationships
Tong, Y., Liao, W., & Ji, Q. (2007) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2007.1094 · Cited by 377 · see citing works
The Science of Emotion: Research and Tradition in the Psychology of Emotion
Cornelius, R. R. (1995) · Medical Entomology and Zoology · Cited by 302 · see citing works
THE TIMING OF FACIAL MOTION IN POSED AND SPONTANEOUS SMILES
Cohn, J. F., & Schmidt, K. L. (2004) · International Journal of Wavelets Multiresolution and Information Processing · doi:10.1142/s021969130400041x · Cited by 301 · see citing works
Self-citationFully Automatic Facial Action Recognition in Spontaneous Behavior
Bartlett, M. S., Littlewort, G. C., Frank, M. G., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2006) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fgr.2006.55 · Cited by 295 · see citing works
The facial expression of pain.
Craig, K. D., Prkachin, K. M., & GRUNAU, R. E. (2001) · Cited by 259 · see citing works
Self-citationTakeo Kanade, A. (2005) · Lecture notes in computer science · doi:10.1007/11564386_1 · Cited by 226 · see citing works
Gibson, R. (2002) · Cited by 195 · see citing works
Machine learning methods for fully automatic recognition of facial expressions and facial actions
Bartlett, M. S., Littlewort, G. C., Lainscsek, C. S. M., Fasel, I., & Movellan, J. R. (2005) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2004.1398364 · Cited by 168 · see citing works
Spontaneous vs. posed facial behavior: automatic analysis of brow actions
Valstar, M. F., Pantic, M., Ambadar, Z., & Cohn, J. F. (2006) · Proceedings of the 8th international conference on Multimodal interfaces · doi:10.1145/1180995.1181031 · Cited by 168 · see citing works
Self-citationInvestigating Spontaneous Facial Action Recognition through AAM Representations of the Face
Lucey, S., Bilal, A., & F., J. (2007) · Face Recognition · doi:10.5772/4841 · Cited by 161 · see citing works
Self-citationHuman computing and machine understanding of human behavior
Pantić, M., Pentland, A., Nijholt, A., & Huang, T. S. (2006) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1180995.1181044 · Cited by 153 · see citing works
Littlewort, G. C., Bartlett, M. S., & Lee, K. (2007) · International Conference on Multimodal Interfaces · doi:10.1145/1322192.1322198 · Cited by 152 · see citing works
Encoding and decoding of pain expressions: a judgement study
Prkachin, K. M., Berzins, S., & Mercer, S. R. (1994) · Pain · doi:10.1016/0304-3959(94)90206-2 · Cited by 125 · see citing works
Self-citationAshraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K., Solomon, P., & Theobald, B. J. (2007) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/1322192.1322197 · Cited by 106 · see citing works
Self-citation2D vs. 3D Deformable Face Models: Representational Power, Construction, and Real-Time Fitting
Matthews, I. A., Xiao, J., & Baker, S. (2007) · International Journal of Computer Vision · doi:10.1007/s11263-007-0043-2 · Cited by 92 · see citing works
Detecting Facial Actions and their Temporal Segments in Nearly Frontal-View Face Image Sequences
Pantić, M., & Patras, I. (2006) · Conference proceedings/Conference proceedings - IEEE International Conference on Systems, Man, and Cybernetics · doi:10.1109/icsmc.2005.1571665 · Cited by 92 · see citing works
Flagged first, then by citations
Research works printed but not linked: 3
- [3]Ashraf AB Lucey S Cohn J Chen T Ambadar Z Prkachin K Solomon P The Painfule Face – Pain expression recognition using active appearance models Proceedings of the 9th international conference on Multimodal interfaces (ICMI) 9 14 2007
- [9]Cornelius RR The science of emotion Prentice Hall Upper Saddler River, New Jersey 1996
- [13]Ekman P Friesen WV Facial action coding system Consulting Psychologists Press Palo Alto, CA 1978
Cited by
Sorted by citations
Facing Imbalanced Data--Recommendations for the Use of Performance Metrics
Jeni, L. A., Cohn, J. F., & de la Torre, F. N. (2013) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2013.47 · Cited by 828 · see citing works
Self-citationPainful data: The UNBC-McMaster shoulder pain expression archive database
Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., & Matthews, I. A. (2011) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2011.5771462 · Cited by 617 · see citing works
Self-citationAffective Body Expression Perception and Recognition: A Survey
Kleinsmith, A., & Bianchi-Berthouze, N. (2013) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2012.16 · Cited by 646 · see citing works
Automatic facial expression recognition using features of salient facial patches
Happy, S., & Routray, A. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2014.2386334 · Cited by 513 · see citing works
Costa, E. D., Minero, M., Lebelt, D., Stucke, D., Canali, E., & Leach, M. C. (2014) · PLoS ONE · doi:10.1371/journal.pone.0092281 · Cited by 585 · see citing works
Corneanu, C. A., Simon, M. O., Cohn, J. F., & Guerrero, S. E. (2016) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2016.2515606 · Cited by 527 · see citing works
Self-citationDavis, K. D., Aghaeepour, N., Ahn, A. H., Angst, M. S., Borsook, D., Brenton, A., Burczynski, M. E., Crean, C. S., Edwards, R. R., Gaudilliere, B. L., et al. (2020) · Nature Reviews Neurology · doi:10.1038/s41582-020-0362-2 · Cited by 499 · see citing works
Detecting depression from facial actions and vocal prosody
Cohn, J. F., Kruez, T. S., Matthews, I., Yang, Y., Nguyen, M. H., Padilla, M. T., Zhou, F., & De la Torre, F. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349358 · Cited by 494 · see citing works
Self-citationFace alignment by coarse-to-fine shape searching
Zhu, S., Li, C., Loy, C. C., & Tang, X. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7299134 · Cited by 488 · see citing works
Gunes, H., & Schuller, B. (2013) · Image and Vision Computing · doi:10.1016/j.imavis.2012.06.016 · Cited by 360 · see citing works
Automatic Analysis of Facial Actions: A Survey
Martínez, B., Valstar, M., Jiang, B., & Pantić, M. (2017) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2017.2731763 · Cited by 354 · see citing works
Pain assessment in elderly adults with dementia
Hadjistavropoulos, T., Herr, K., Prkachin, K. M., Craig, K. D., Gibson, S. J., Lukas, A., & Smith, J. H. (2014) · The Lancet Neurology · doi:10.1016/s1474-4422(14)70103-6 · Cited by 319 · see citing works
Self-citationFully Automatic Recognition of the Temporal Phases of Facial Actions
Valstar, M., & Pantić, M. (2011) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2011.2163710 · Cited by 304 · see citing works
Deep Pain: Exploiting Long Short-Term Memory Networks for Facial Expression Classification
Rodriguez, P., Cucurull, G., Gonzalez, J., Gonfaus, J. M., Nasrollahi, K., Moeslund, T. B., & Roca, F. X. (2022) · IEEE Transactions on Cybernetics · doi:10.1109/tcyb.2017.2662199 · Cited by 297 · see citing works
Meta-Analysis of the First Facial Expression Recognition Challenge
Valstar, M. F., Mehu, M., Jiang, B., Pantic, M., & Scherer, K. (2012) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2200675 · Cited by 297 · see citing works
Automatically Detecting Pain in Video Through Facial Action Units
Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., & Prkachin, K. M. (2011) · IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics) · doi:10.1109/tsmcb.2010.2082525 · Cited by 267 · see citing works
Self-citationAutomatic Recognition Methods Supporting Pain Assessment: A Survey
Werner, P., Lopez-Martinez, D., Walter, S., Al-Hamadi, A. K., Gruss, S., & Picard, R. W. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2946774 · Cited by 250 · see citing works
Aung, M. H., Kaltwang, S., Romera‐Paredes, B., Martínez, B., Singh, A., Cella, M., Valstar, M., Meng, H., Kemp, A. H., Shafizadeh, M., et al. (2015) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2015.2462830 · Cited by 219 · see citing works
BAUM-1: A Spontaneous Audio-Visual Face Database of Affective and Mental States
Zhalehpour, S., Onder, O., Akhtar, Z., & Erdem, Ç. E. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2553038 · Cited by 220 · see citing works
Schuller, B. W., Müller, R., Eyben, F., Gast, J., Hörnler, B., Wöllmer, M., Rigoll, G., Hoethker, A., & Konosu, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2009.02.013 · Cited by 194 · see citing works
Automatic detection of pain intensity
Hammal, Z., & Cohn, J. F. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388688 · Cited by 185 · see citing works
Self-citationAssessing Pain by Facial Expression: Facial Expression as Nexus
Prkachin, K. M. (2009) · Pain Research and Management · doi:10.1155/2009/542964 · Cited by 183 · see citing works
Self-citationA Survey on Computer Vision for Assistive Medical Diagnosis From Faces
Thevenot, J., López, M. B., & Hadid, A. (2017) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2017.2754861 · Cited by 166 · see citing works
Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., Chew, S. W., & Matthews, I. A. (2011) · Image and Vision Computing · doi:10.1016/j.imavis.2011.12.003 · Cited by 166 · see citing works
Self-citationAutomatic Pain Assessment with Facial Activity Descriptors
Werner, P., Al-Hamadi, A. K., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., & Traue, H. C. (2016) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2016.2537327 · Cited by 166 · see citing works
Karnati, M., Seal, A., Bhattacharjee, D., Yazidi, A., & Krejcar, O. (2023) · IEEE Transactions on Instrumentation and Measurement · doi:10.1109/tim.2023.3243661 · Cited by 176 · see citing works
Enhanced deep learning algorithm development to detect pain intensity from facial expression images
Bargshady, G., Zhou, X., Deo, R. C., Soar, J., Whittaker, F., & Wang, H. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113305 · Cited by 162 · see citing works
Prediction and Localization of Student Engagement in the Wild
Kaur, A., Mustafa, A., Mehta, L., & Dhall, A. (2018) · International Conference on Digital Image Computing: Techniques and Applications (DICTA) · doi:10.1109/dicta.2018.8615851 · Cited by 160 · see citing works
The racially diverse affective expression (RADIATE) face stimulus set
Conley, M. I., Dellarco, D. V., Rubien-Thomas, E., Cohen, A. O., Cervera, A., Tottenham, N., & Casey, B. J. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.04.066 · Cited by 151 · see citing works
Automated Assessment of Children’s Postoperative Pain Using Computer Vision
Sikka, K., Ahmed, A. A., Díaz, D., Goodwin, M. S., Craig, K. D., Bartlett, M. S., & Huang, J. S. (2015) · PEDIATRICS · doi:10.1542/peds.2015-0029 · Cited by 137 · see citing works
Advances, Challenges, and Opportunities in Automatic Facial Expression Recognition
Martínez, B., & Valstar, M. (2016) · Springer eBooks · doi:10.1007/978-3-319-25958-1_4 · Cited by 136 · see citing works
de la Torre, F. N., & Cohn, J. F. (2011) · Springer eBooks · doi:10.1007/978-0-85729-997-0_19 · Cited by 136 · see citing works
Self-citationAutomatically detecting pain using facial actions
Lucey, P., Cohn, J. F., Lucey, S., Matthews, I., Sridharan, S., & Prkachin, K. M. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349321 · Cited by 126 · see citing works
Self-citationRecurrent Convolutional Neural Network Regression for Continuous Pain Intensity Estimation in Video
Zhou, J. L., Hong, X., Su, F., & Zhao, G. (2016) · IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2016.191 · Cited by 125 · see citing works
A Review of Automated Pain Assessment in Infants: Features, Classification Tasks, and Databases
Zamzmi, G., Kasturi, R., Goldgof, D. B., Zhi, R., Ashmeade, T. L., & Sun, Y. (2017) · IEEE Reviews in Biomedical Engineering · doi:10.1109/rbme.2017.2777907 · Cited by 122 · see citing works
Vision-based patient monitoring: a comprehensive review of algorithms and technologies
Sathyanarayana, S., Satzoda, R. K., Sathyanarayana, S., & Srikanthan, T. (2015) · Journal of Ambient Intelligence and Humanized Computing · doi:10.1007/s12652-015-0328-1 · Cited by 121 · see citing works
Sajjad, M., Nasir, M., Ullah, F. U. M., Muhammad, K., Sangaiah, A. K., & Baik, S. W. (2018) · Information Sciences · doi:10.1016/j.ins.2018.07.027 · Cited by 113 · see citing works
Automated Facial Expression Recognition System
Ryan, A. H., Cohn, J. F., Lucey, S., Saragih, J., Lucey, P., la Torre, F. D., & Rossi, A. (2009) · IEEE International Carnahan Conference on Security Technology (ICCST) · doi:10.1109/ccst.2009.5335546 · Cited by 106 · see citing works
Self-citationZhang, X., Li, W., Chen, X., & Lu, S. (2018) · Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies · doi:10.1145/3161414 · Cited by 100 · see citing works
Machine analysis of facial behaviour: naturalistic and dynamic behaviour
Pantić, M. (2009) · Philosophical Transactions of the Royal Society B Biological Sciences · doi:10.1098/rstb.2009.0135 · Cited by 103 · see citing works
Automatic Detection of Pain from Facial Expressions: A Survey
Hassan, T., Seuß, D., Wollenberg, J., Weitz, K., Miriam Kunz, A., Lautenbacher, S., Garbas, J.-U., & Schmid, U. (2020) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2019.2958341 · Cited by 98 · see citing works
GARP-face: Balancing privacy protection and utility preservation in face de-identification
Du, L., Yi, M., Blasch, E., & Ling, H. (2014) · IEEE International Joint Conference on Biometrics (IJCB) · doi:10.1109/btas.2014.6996249 · Cited by 95 · see citing works
Methods for Person-Centered Continuous Pain Intensity Assessment From Bio-Physiological Channels
Kächele, M., Thiam, P., Amirian, M., Schwenker, F., & Palm, G. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2535962 · Cited by 93 · see citing works
Personalized Automatic Estimation of Self-Reported Pain Intensity from Facial Expressions
Martinez, D. L., Rudovic, O., & Picard, R. W. (2017) · 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2017.286 · Cited by 88 · see citing works
From the Lab to the Real World: Affect Recognition Using Multiple Cues and Modalities
Güneş, H., Piccardi, M., & Pantić, M. (2008) · doi:10.5772/6180 · Cited by 93 · see citing works
Emotion recognition and its application in software engineering
Kołakowska, A., Landowska, A., Szwoch, M., Szwoch, W., & Wróbel, M. R. (2013) · Conference on Human System Interactions · doi:10.1109/hsi.2013.6577877 · Cited by 87 · see citing works
Automated Face Analysis for Affective Computing
Calvo, R. A., D’Mello, S. K., Gratch, J. S., Kappas, A., Cohn, J. F., & la Torre, F. D. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199942237.013.020 · Cited by 82 · see citing works
Self-citationAdaptive confidence learning for the personalization of pain intensity estimation systems
Kächele, M., Amirian, M., Thiam, P., Werner, P., Walter, S., Palm, G., & Schwenker, F. (2016) · Evolving Systems · doi:10.1007/s12530-016-9158-4 · Cited by 82 · see citing works
Facial Expression Recognition: A Review of Trends and Techniques
Ekundayo, O., & Viriri, S. (2021) · IEEE Access · doi:10.1109/access.2021.3113464 · Cited by 85 · see citing works
Ethical Issues in Affective Computing
Cowie, R. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199942237.013.006 · Cited by 78 · see citing works
Artificial intelligence to evaluate postoperative pain based on facial expression recognition
Fontaine, D., Vielzeuf, V., Genestier, P., Limeux, P., Santucci-Sivolotto, S., Mory, E., Darmon, N., Lantéri‐Minet, M., Mokhtar, M., Laine, M., et al. (2022) · European Journal of Pain · doi:10.1002/ejp.1948 · Cited by 78 · see citing works
Finka, L. R., Luna, S. P. L., Brondani, J. T., Tzimiropoulos, Y., McDonagh, J., Farnworth, M. J., Ruta, M., & Mills, D. S. (2019) · Scientific Reports · doi:10.1038/s41598-019-46330-5 · Cited by 75 · see citing works
Tavakolian, M., & Hadid, A. (2019) · International Journal of Computer Vision · doi:10.1007/s11263-019-01191-3 · Cited by 72 · see citing works
Pain detection through shape and appearance features
Khan, R. A., Meyer, A., Konik, H., & Bouakaz, S. (2013) · IEEE International Conference on Multimedia and Expo (ICME) · doi:10.1109/icme.2013.6607608 · Cited by 72 · see citing works
Sensor Technologies to Manage the Physiological Traits of Chronic Pain: A Review
Naranjo-Hernández, D., Reina‐Tosina, J., & Roa, L. M. (2020) · Sensors · doi:10.3390/s20020365 · Cited by 73 · see citing works
A New Dataset for Facial Motion Analysis in Individuals With Neurological Disorders
Bandini, A., Rezaei, S., Guarín, D. L., Kulkarni, M., Lim, D., Boulos, M. I., Zinman, L. H., Yunusova, Y., & Taati, B. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3019242 · Cited by 73 · see citing works
Automatic Facial Expression Analysis A Survey
Sumathi, C. P. (2012) · International Journal of Computer Science & Engineering Survey · doi:10.5121/ijcses.2012.3604 · Cited by 70 · see citing works
Egede, J., Valstar, M., & Martínez, B. (2017) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2017.87 · Cited by 70 · see citing works
Ensemble neural network approach detecting pain intensity from facial expressions
Bargshady, G., Zhou, X., Deo, R. C., Soar, J., Whittaker, F., & Wang, H. (2020) · Artificial Intelligence in Medicine · doi:10.1016/j.artmed.2020.101954 · Cited by 71 · see citing works
Multimodal Data Fusion for Person-Independent, Continuous Estimation of Pain Intensity
Kächele, M., Thiam, P., Amirian, M., Werner, P., Walter, S., Schwenker, F., & Palm, G. (2015) · Communications in computer and information science · doi:10.1007/978-3-319-23983-5_26 · Cited by 68 · see citing works
In the Pursuit of Effective Affective Computing: The Relationship Between Features and Registration
Chew, S. W., Lucey, P., Lucey, S., Saragih, J., Cohn, J. F., Matthews, I., & Sridharan, S. (2012) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2194485 · Cited by 64 · see citing works
Self-citationSemi-supervised learning of multi-factor models for face de-identification
Ralph Gross, A., Sweeney, L., la Torre, F. D., & Baker, S. (2008) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2008.4587369 · Cited by 65 · see citing works
Weakly supervised pain localization using multiple instance learning
Sikka, K., Dhall, A., & Bartlett, M. S. (2013) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2013.6553762 · Cited by 62 · see citing works
Costa, E. D., Pascuzzo, R., Leach, M. C., Dai, F., Lebelt, D., Vantini, S., & Minero, M. (2018) · PLoS ONE · doi:10.1371/journal.pone.0200339 · Cited by 60 · see citing works
Classification and weakly supervised pain localization using multiple segment representation
Sikka, K., Dhall, A., & Bartlett, M. S. (2014) · Image and Vision Computing · doi:10.1016/j.imavis.2014.02.008 · Cited by 58 · see citing works
Advances in Behavioral Science Using Automated Facial Image Analysis and Synthesis [Social Sciences
Cohn, J. F. (2010) · IEEE Signal Processing Magazine · doi:10.1109/msp.2010.938102 · Cited by 56 · see citing works
Self-citationCohn, J. F., & Sayette, M. A. (2010) · Behavior Research Methods · doi:10.3758/brm.42.4.1079 · Cited by 55 · see citing works
Self-citationZhu, Y., la Torre, F. D., Cohn, J. F., & Zhang, Y. (2011) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2011.10 · Cited by 53 · see citing works
Self-citationAn Approach for Automatic Pain Detection through Facial Expression
Roy, S. D., Bhowmik, M. K., Saha, P., & Ghosh, A. K. (2016) · Procedia Computer Science · doi:10.1016/j.procs.2016.04.072 · Cited by 53 · see citing works
Visual data of facial expressions for automatic pain detection
Virrey, R. A., Silva, L. C. D., Petra, M. I., & Abas, P. E. (2019) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2019.03.023 · Cited by 52 · see citing works
A Comprehensive and Context-Sensitive Neonatal Pain Assessment Using Computer Vision
Zamzmi, G., Pai, C.-Y., Goldgof, D. B., Kasturi, R., Ashmeade, T. L., & Sun, Y. (2019) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2019.2926710 · Cited by 52 · see citing works
A Survey on Recent Advances in Social Robotics
Youssef, K., Said, S., Kork, S. A., & Beyrouthy, T. (2022) · Robotics · doi:10.3390/robotics11040075 · Cited by 51 · see citing works
Li, J., Ngai, G., Leong, H., & Chan, S. (2016) · SIAP · doi:10.1145/3015297.3015301 · Cited by 45 · see citing works
Arif-Rahu, M., Grap, M. J., Cohn, J. F., Munro, C. L., Lyon, D., & Sessler, C. N. (2013) · American Journal of Critical Care · doi:10.4037/ajcc2013705 · Cited by 50 · see citing works
Self-citationA Joint Deep Neural Network Model for Pain Recognition from Face
Bargshady, G., Soar, J., Zhou, X., Deo, R. C., Whittaker, F., & Wang, H. (2019) · International Conference on Computer and Communication Systems (ICCCS) · doi:10.1109/ccoms.2019.8821779 · Cited by 50 · see citing works
Bargshady, G., Zhou, X., Deo, R. C., Soar, J., Whittaker, F., & Wang, H. (2020) · Applied Soft Computing · doi:10.1016/j.asoc.2020.106805 · Cited by 53 · see citing works
BAUM-2: a multilingual audio-visual affective face database
Erdem, Ç. E., Turan, C., & Aydın, Z. (2014) · Multimedia Tools and Applications · doi:10.1007/s11042-014-1986-2 · Cited by 49 · see citing works
EMOPAIN Challenge 2020: Multimodal Pain Evaluation from Facial and Bodily Expressions
Egede, J., Song, S., Olugbade, T., Wang, C., Williams, A. C. D. C., Meng, H., Aung, M. H., Lane, N. D., Valstar, M., & Bianchi‐Berthouze, N. (2020) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg47880.2020.00078 · Cited by 49 · see citing works
A novel approach for pain intensity detection based on facial feature deformations
Rathee, N., & Ganotra, D. (2015) · Journal of Visual Communication and Image Representation · doi:10.1016/j.jvcir.2015.09.007 · Cited by 46 · see citing works
Multilayer Architectures for Facial Action Unit Recognition
Wu, T., Butko, N. J., Ruvolo, P., Whitehill, J., Bartlett, M. S., & Movellan, J. R. (2012) · IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) · doi:10.1109/tsmcb.2012.2195170 · Cited by 45 · see citing works
DeepFaceLIFT: Interpretable Personalized Models for Automatic Estimation of Self-Reported PainPreprint
Liu, D., Peng, F., Shea, A., Ognjen, Rudovic, & Picard, R. W. (2017) · AffComp@IJCAI · doi:10.48550/arxiv.1708.04670 · Cited by 54 · see citing works
Spatiotemporal analysis of RGB-D-T facial images for multimodal pain level recognition
Irani, R., Nasrollahi, K., Simon, M. O., Corneanu, C. A., Escalera, S., Bahnsen, C., Lundtoft, D. H., Moeslund, T. B., Pedersen, T. L., Klitgaard, M.-L., et al. (2015) · 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) · doi:10.1109/cvprw.2015.7301341 · Cited by 42 · see citing works
HybNet: a hybrid network structure for pain intensity estimation
Huang, Y., Qing, L., Xu, S., Wang, L., & Peng, Y. (2021) · The Visual Computer · doi:10.1007/s00371-021-02056-y · Cited by 41 · see citing works
Pain Intensity Evaluation through Facial Action Units
Zafar, Z., & Khan, N. A. (2014) · Proceedings - International Conference on Pattern Recognition/Proceedings/International Conference on Pattern Recognition · doi:10.1109/icpr.2014.803 · Cited by 41 · see citing works
Ghosh, A. K., Umer, S., Khan, M. K., Rout, R. K., & Dhara, B. C. (2022) · Cluster Computing · doi:10.1007/s10586-022-03552-z · Cited by 42 · see citing works
Leveraging Multi-Modal Sensing for Mobile Health: A Case Review in Chronic Pain
Aung, M. H., Alquaddoomi, F., Hsieh, C.-K., Rabbi, M., Yang, L., Pollak, J., Estrin, D. L., & Choudhury, T. (2016) · IEEE Journal of Selected Topics in Signal Processing · doi:10.1109/jstsp.2016.2565381 · Cited by 38 · see citing works
Learning realistic facial expressions from web images
Yu, K., Wang, Z., Li, Z., Wang, J., Chi, Z., & Feng, D. D. (2013) · Pattern Recognition · doi:10.1016/j.patcog.2013.01.032 · Cited by 36 · see citing works
Multi-instance Hidden Markov Model for facial expression recognition
Wu, C., Wang, S., & Ji, Q. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163116 · Cited by 34 · see citing works
Pain detection from facial characteristics using supervised descent method
Neshov, N. N., & Manolova, A. H. (2015) · IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS) · doi:10.1109/idaacs.2015.7340738 · Cited by 36 · see citing works
Objectively Measuring Pain Using Facial Expression: Is the Technology Finally Ready?
Dawes, T. R., Eden-Green, B., Rosten, C., Giles, J., Governo, R. J. M., Marcelline, F., & Nduka, C. C. (2018) · Pain Management · doi:10.2217/pmt-2017-0049 · Cited by 36 · see citing works
Unobtrusive Pain Monitoring in Older Adults With Dementia Using Pairwise and Contrastive Training
Rezaei, S., Moturu, A., Zhao, S., Prkachin, K. M., Hadjistavropoulos, T., & Taati, B. (2020) · IEEE Journal of Biomedical and Health Informatics · doi:10.1109/jbhi.2020.3045743 · Cited by 36 · see citing works
Self-citationUsing Facially Expressive Robots to Calibrate Clinical Pain Perception
Moosaei, M., Das, S. K., Popa, D. O., & Riek, L. D. (2017) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1145/2909824.3020216 · Cited by 35 · see citing works
3D model-based continuous emotion recognition
Chen, H., Li, J., Zhang, F., Li, Y., & Wang, H. (2015) · IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings - IEEE Computer Society Conference on Computer Vision and Pattern Recognition/Proceedings · doi:10.1109/cvpr.2015.7298793 · Cited by 34 · see citing works
Automated and objective action coding of facial expressions in patients with acute facial palsy
Haase, D., Minnigerode, L., Volk, G. F., Denzler, J., & Guntinas‐Lichius, O. (2014) · European Archives of Oto-Rhino-Laryngology · doi:10.1007/s00405-014-3385-8 · Cited by 35 · see citing works
Delaherche, E., Chétouani, M., Bigouret, F., Xavier, J., Plaza, M., & Cohen, D. (2013) · Research in Autism · doi:10.1016/j.rasd.2013.02.003 · Cited by 35 · see citing works
Soladié, C., Salam, H., Pelachaud, C., Stoiber, N., & Séguier, R. (2012) · Proceedings of the 14th ACM international conference on Multimodal interaction · doi:10.1145/2388676.2388782 · Cited by 33 · see citing works
On pain assessment from facial videos using spatio-temporal local descriptors
Yang, R., Shujun, T., López, M. B., Boutellaa, E., Peng, J., Feng, X., & Hadid, A. (2016) · International Conference on Image Processing Theory, Tools and Applications (IPTA) · doi:10.1109/ipta.2016.7820930 · Cited by 33 · see citing works
Abbas, A., & Chalup, S. K. (2017) · INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION · doi:10.1145/3136755.3143010 · Cited by 32 · see citing works
Dynamic cascades with bidirectional bootstrapping for spontaneous facial action unit detection
Zhu, Y., la Torre, F. D., Cohn, J. F., & Zhang, Y. (2009) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii.2009.5349603 · Cited by 30 · see citing works
Self-citationZhi, R., Zamzmi, G., Goldgof, D. B., Ashmeade, T. L., & Sun, Y. (2018) · Journal of Clinical Medicine · doi:10.3390/jcm7070173 · Cited by 31 · see citing works
- Authors
- Ashraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K. M., & Solomon, P. E.
- Affiliations
- University of Pittsburgh (US)
- Corresponding author
- Cohn, Jeffrey F.
- Venue
- Image and Vision Computing
- Publisher
- Elsevier BV
- Published
- 2009-11
- Volume
- 27
- Issue
- 12
- Pages
- 1788-1796
- ISSN
- 0262-8856 (ISSN-L)
- PMC
- PMC3402903
- OpenAlex
- W2090495691
- Cited by
- 355
- References
- 38
- Citations by year
- 2026: 1 · 2025: 9 · 2024: 14 · 2023: 7 · 2022: 11 · 2021: 29 · 2020: 28 · 2019: 31 · 2018: 31 · 2017: 26 · 2016: 33 · 2015: 25 · 2014: 28 · 2013: 20 · 2012: 21
- FWCI
- 21.67
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Image and Vision Computing (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Europe PMC (PubMed Central) (acceptedVersion)
- PubMed Central (acceptedVersion)
- National University of Singapore (submittedVersion)
- Funders
- University of Northern British Columbia
- McMaster University
- Canadian Institutes of Health Research (MOP77799)
- National Institute of Mental Health (MH-51435, R01MH051435)
- Topics
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- active appearance models, rotator cuff injury, facial action units, support vector machine, ground truth annotation, video-based pain assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
