Emotional Expression Classification Using Time-Series Kernels
Open accessAbstract
Estimation of facial expressions, as spatio-temporal processes, can take advantage of kernel methods if one considers facial landmark positions and their motion in 3D space. We applied support vector classification with kernels derived from dynamic time-warping similarity measures. We achieved over 99% accuracy - measured by area under ROC curve - using only the 'motion pattern' of the PCA compressed representation of the marker point vector, the so-called shape parameters. Beyond the classification of full motion patterns, several expressions were recognized with over 90% accuracy in as few as 5-6 frames from their onset, about 200 milliseconds.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Previous FER methods treat the texture or landmark modality independently, using only one modality for FER.
“Mostly, all previous work treats the texture or landmark modality independently, where only the texture or landmark modality is employed for FER.”
From Multimodal learning for facial expression recognition (2015) · cites this paper as [2]
The Discussion and Summary (p. 6) says this paper uses shape information without texture and contrasts it with a method that exploits textural features. It proposes combining the approaches, but does not establish that most previous FER work used only one modality. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303 · see citing works
Self-citationFacial expression recognition using Gabor motion energy filters
Wu, T., Bartlett, M. S., & Movellan, J. R. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543267 · Cited by 126 · see citing works
Flagged first, then by citations
Research works printed but not linked: 19
- [1][1] K. Bousmalis, L.-P. Morency, and M. Pantic. Modeling hidden dynamics of multimodal cues for spontaneous agreement and disagreement recognition. In Proceedings of the Ninth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2011) , pages 746-752, 2011. 2 -
- [2][2] K. Bousmalis, S. Zafeiriou, L.-P. Morency, and M. Pantic. Infinite hidden conditional random fields for human behavior analysis. IEEE Transactions on Neural Networks and Learning Systems , 24(1):170-177, 2013. 2 -
- [3][3] C.-C. Chang and C.-J. Lin. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology , 2:27:1-27:27, 2011. Software available at http://www.csie.ntu.edu.tw/˜cjlin/libsvm . 4 -
- [4][4] M. Cuturi. Fast global alignment kernels. In Proceedings of the International Conference on Machine Learning , 2011. 2, 3, 4 -
- [5][5] M. Cuturi, J.-P. Vert, . Birkenes, and T. Matsui. A kernel for time series based on global alignments. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2007) , volume 2, pages 413416, 2007. 2, 3 -
- [6][6] J. R. Deller, J. G. Proakis, and J. H. Hansen. Discrete-time processing of speech signals . IEEE New York, NY, USA:, 2000. 2 -
- [7][7] B. Haasdonk and C. Bahlmann. Learning with distance substitution kernels. In Pattern Recognition , pages 220-227. Springer, 2004. 3 -
- [8][8] N. J. Higham. Computing the nearest correlation matrix a problem from finance. IMA Journal of Numerical Analysis , 22(3):329-343, 2002. 3, 5 -
- [9][9] L. A. Jeni, J. M. Girard, J. F. Cohn, and F. De La Torre. Continuous AU intensity estimation using localized, sparse facial feature space. In 2nd International Workshop on Emotion Representation, Analysis and Synthesis in Continuous Time and Space (EmoSPACE) , 2013. 6 -
- [10][10] L. A. Jeni, A. L˝ orincz, T. Nagy, Z. Palotai, J. Seb˝ ok, Z. Szab´ o, and D. Tak´ acs. 3d shape estimation in video sequences provides high precision evaluation of facial expressions. Image and Vision Computing , 30(10):785-795, 2012. 6 -
- [11][11] T. Kanade, J. F. Cohn, and Y. Tian. Comprehensive database for facial expression analysis. In Proceedings of the Fourth IEEE International Conference on Automatic Face and Gesture Recognition (FG 2000) , pages 46-53, 2000. 5 -
- [14][14] I. Matthews and S. Baker. Active appearance models revisited. International Journal of Computer Vision , 60(2):135164, 2004. 2 -
- [15][15] K.-R. M¨ uller, A. J. Smola, G. R¨ atsch, B. Sch¨ olkopf, J. Kohlmorgen, and V. Vapnik. Predicting time series with support vector machines. In Artificial Neural NetworksICANN'97 , pages 999-1004. Springer, 1997. 2 -
- [16][16] T. Preis, J. J. Schneider, and H. E. Stanley. Switching processes in financial markets. Proceedings of the National Academy of Sciences , 108(19):7674-7678, 2011. 1 -
- [17][17] T. Rakthanmanon, B. Campana, A. Mueen, G. Batista, B. Westover, Q. Zhu, J. Zakaria, and E. Keogh. Searching and mining trillions of time series subsequences under dynamic time warping. In Proceedings of the 18th ACM SIGKDD international conference on Knowledge discovery and data mining , pages 262-270. ACM, 2012. 7 -
- [18][18] H. Sakoe and S. Chiba. Dynamic programming algorithm optimization for spoken word recognition. Acoustics, Speech and Signal Processing, IEEE Transactions on , 26(1):43-49, 1978. 2 -
- [19][19] J. M. Saragih, S. Lucey, and J. F. Cohn. Deformable model fitting by regularized landmark mean-shift. International Journal of Computer Vision , 91(2):200-215, 2011. 2, 3 -
- [20][20] M. F. Valstar and M. Pantic. Combined support vector machines and hidden markov models for modeling facial action temporal dynamics. In Human-Computer Interaction , pages 118-127. Springer, 2007. 2 -
- [22][22] P. Yang, Q. Liu, and D. N. Metaxas. Boosting encoded dynamic features for facial expression recognition. Pattern Recognition Letters , 30(2):132-139, 2009. 2, 6
Cited by
Sorted by citations
Multimodal learning for facial expression recognition
Zhang, W., Zhang, Y., Ma, L., Guan, J., & Gong, S. (2015) · Pattern Recognition · doi:10.1016/j.patcog.2015.04.012 · Cited by 100 · see citing works
Dynamic facial expression recognition by joint static and multi-time gap transition classification
Dapogny, A., Bailly, K., & Dubuisson, S. (2015) · 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG) · doi:10.1109/fg.2015.7163111 · Cited by 24 · see citing works
- Authors
- Lorincz, A., Jeni, L. A., Szabo, Z., Cohn, J. F., & Kanade, T.
- Affiliations
- Carnegie Mellon University (US)
- ORCID
- Cohn, Jeffrey F.
- Venue
- 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops
- Publisher
- IEEE
- Published
- 2013-06
- Pages
- 889-895
- OpenAlex
- W2150137812
- Cited by
- 45
- References
- 2
- Citations by year
- 2025: 1 · 2022: 1 · 2021: 1 · 2020: 3 · 2019: 3 · 2018: 3 · 2017: 6 · 2016: 9 · 2015: 11 · 2014: 5 · 2013: 2
- FWCI
- 12.25
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- ELTE Digital Institutional Repository (EDIT) (Eötvös Loránd University) (submittedVersion, other-oa)
- UCL Discovery (University College London) (submittedVersion)
- arXiv (Cornell University)
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face recognition and analysis
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Face and Expression Recognition
- Keywords
- facial expression recognition, dynamic time warping, support vector machine classification, shape parameter, movement patterns, spatio-temporal processes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
