The DOI matches its Crossref record; 4 of the 4 references it cites were found in a registry or an index.
Facial Expression Recognition Using Hierarchical Features With Deep Comprehensive Multipatches Aggregation Convolutional Neural Networks
Abstract
Facial expression recognition (FER) has long been a challenging task in computer vision. In this paper, we propose a novel method, named deep comprehensive multipatches aggregation convolutional neural networks (CNNs), to solve the FER problem. The proposed method is a deep-based framework, which mainly consists of two branches of the CNN. One branch extracts local features from image patches while the other extracts holistic features from the whole expressional image. In the model, local features depict expressional details and holistic features characterize the high-level semantic information of an expression. We aggregate both local and holistic features before making classification. These two types of hierarchical features represent expressions in different scales. Compared with most current methods with single type of feature, the model can represent expressions more comprehensively. Additionally, in the training stage, a novel pooling strategy named expressional transformation-invariant pooling is proposed for handling nuisance variations, such as rotations, noises, etc. Extensive experiments are conducted on the famous the Extended Cohn-Kanade (CK+) dataset and the Japanese Female Facial Expression (JAFFE) database expression datasets, where the recognition results obtained.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Lucey, P., Cohn, J. F., Kanade, T., Saragih, J., Ambadar, Z., & Matthews, I. (2010) · 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition - Workshops · doi:10.1109/cvprw.2010.5543262 · Cited by 4,303
Recognizing action units for facial expression analysis
Tian, Y.-I., Kanade, T., & Cohn, J. F. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.908962 · Cited by 1,745
Social signal processing: Survey of an emerging domain
Vinciarelli, A., Pantic, M., & Bourlard, H. (2009) · Image and Vision Computing · doi:10.1016/j.imavis.2008.11.007 · Cited by 842
Joint Fine-Tuning in Deep Neural Networks for Facial Expression Recognition
Jung, H., Lee, S., Yim, J., Park, S., & Kim, J. (2015) · 2015 IEEE International Conference on Computer Vision (ICCV) · doi:10.1109/iccv.2015.341 · Cited by 781 · cited 4×
“In this paper, we compare the recognition accuracy of our model with several competitive approaches, including conventional methods (LAP [44], MPSD [45] and MCSPL [30]) and deep-based methods (DTAN [37], DTGN [37] and 3DCNN-DAP [46]).”
BackgroundMethodInfluential
Research works printed but not linked: 47
- babenko, 0, Aggregating deep convolutional features for image retrieval, Proc Int Conf Comput Vis, 1269
- xu, 0, Facial expression recognition based on transfer learning from deep convolutional networks, Proc 11th Int Conf Natural Comput, 702?708
- 10.1109/IST.2015.7294547 · DOI
- 10.1109/CVPR.2016.369 · DOI · Paper page
- 10.1109/ICAPR.2015.7050661 · DOI
- 10.1109/TCYB.2014.2354351 · DOI · Paper page
- 10.1109/IJCNN.2015.7280539 · DOI
- 10.1109/CVPR.2014.233 · DOI · Paper page
- 10.1109/WACV.2016.7477450 · DOI · Paper page
- 10.1109/TMM.2013.2266635 · DOI
- 10.1109/FG.2011.5771366 · DOI · Paper page
- 10.1109/TMM.2016.2598092 · DOI · Paper page
- 10.1109/CCST.2009.5335546 · DOI
- mehrabian, 2007, Nonverbal Communication
- 10.1109/ICSMC.1997.633250 · DOI
- 10.1016/j.patcog.2013.10.010 · DOI
- 10.1109/IIH-MSP.2015.33 · doi:10.1109/iih-msp.2015.33
- 10.1109/CISP.2010.5647898 · DOI
- 10.1109/TMM.2016.2629282 · DOI
- 10.1109/CVPR.2005.297 · DOI · Paper page
- 10.4218/etrij.10.1510.0132 · DOI · Paper page
- 10.1109/TAFFC.2014.2346515 · DOI
- 10.1109/ICIP.2012.6467434 · DOI
- 10.1117/1.JEI.24.2.023015 · DOI
- majumder, 2016, Automatic facial expression recognition system using deep network-based data fusion, IEEE Trans Cybern, 99, 1
- 10.1109/CVPR.2016.38 · DOI · Paper page
- 10.1109/ICCV.2015.119 · DOI · Paper page
- ekman, 1978, The Facial Action Coding System A Technique for The Measurement of Facial Movement
- tian, 0, Recognizing upper face action units for facial expression analysis, Proc Int Conf Comput Vis Pattern Recognit, 294
- 10.1109/TPAMI.2007.1094 · DOI · Paper page
- 10.1109/FGR.2006.55 · DOI · Paper page
- 10.1109/ICSMC.2004.1398367 · DOI
- 10.5244/C.26.119 · DOI
- 10.1016/j.imavis.2012.01.006 · DOI
- 10.1109/IIH-MSP.2015.54 · doi:10.1109/iih-msp.2015.54
- ekman, 1978, The Facial Action Coding System A Technique for The Measurement of Facial Movement
- vural, 0, Automated drowsiness detection for improved driversafety comprehensive databases for facial expression analysis, Proc Int Conf Autom Technol, 1, 96
- izard, 1971, The Face of Emotion, 1
- 0, Recognition of face expressions using local principal texture pattern, Proc 19th IEEE Int Conf Image Process
- 10.1109/TAFFC.2014.2386334 · DOI · Paper page
- liu, 0, Deeply learning deformable facial action parts model for dynamic expression analysis, Proc Computer Vision, 1749
- 10.1109/TIP.2014.2331141 · DOI
- 10.1109/IJCNN.2012.6252678 · DOI
- 10.1109/ICCVW.2015.12 · DOI · Paper page
- 10.1109/AFGR.1998.670949 · DOI · Paper page
- zhong, 0, Learning active facial patches for expression analysis, Proc IEEE Conf Comput Vis Pattern Recognit, 2562
- 10.1023/B:VISI.0000013087.49260.fb · DOI · Paper page
Cited by
Zhao, Z., Liu, Q., & Wang, S. (2021) · IEEE Transactions on Image Processing · doi:10.1109/tip.2021.3093397 · Cited by 375
Kim, J.-H., Kim, B.-G., Roy, P. P., & Jeong, D.-M. (2019) · IEEE Access · doi:10.1109/access.2019.2907327 · Cited by 303
Multimodal Emotion Recognition Using a Hierarchical Fusion Convolutional Neural Network
Zhang, Y., Cheng, C., & Zhang, Y. (2021) · IEEE Access · doi:10.1109/access.2021.3049516 · Cited by 84
PAtt-Lite: Lightweight Patch and Attention MobileNet for Challenging Facial Expression Recognition
Ngwe, J. L., Lim, K. M., Lee, C. P., Ong, T. S., & Alqahtani, A. (2024) · IEEE Access · doi:10.1109/access.2024.3407108 · Cited by 45
Sharifnejad, M., Shahbahrami, A., Akoushideh, A., & Hassanpour, R. Z. (2021) · IET Image Processing · doi:10.1049/ipr2.12037 · Cited by 19
“Convolutional kernels of FRRCNN are improved, which results in generating less redundant features and yields a more compact representation of an image.”
MethodInfluential
Metadata
- Authors
- Xie, S., & Hu, H.
- Venue
- IEEE Transactions on Multimedia
- Published
- 2019-01
- Volume
- 21
- Issue
- 1
- Pages
- 211-220
- ISSN
- 1520-9210 (ISSN-L)
- 1941-0077
- OpenAlex
- W2805781993
- Cited by
- 214
- References
- 4
- FWCI
- 25.30
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- Funders
- National Natural Science Foundation of China (61673402, 61273270, 60802069)
- Natural Science Foundation of Guangdong (2017A030311029, 2016B010109002, 2015B090912001, 2016B010123005, 2017B090909005)
- Science and Technology Program of Guangzhou (201704020180, 201604020024)
- Fundamental Research Funds for the Central Universities of China
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Facial Expression Recognition Using Hierarchical Features With Deep Comprehensive Multipatches Aggregation Convolutional Neural Networks. https://app.cochat.ai/papers/10.1109/tmm.2018.2844085
- The paper
- Xie, S., & Hu, H. (2019). Facial Expression Recognition Using Hierarchical Features With Deep Comprehensive Multipatches Aggregation Convolutional Neural Networks. IEEE Transactions on Multimedia. https://doi.org/10.1109/tmm.2018.2844085
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.