Recognizing expressions from face and body gesture by temporal normalized motion and appearance features

Manual feature fusion techniques do not take into consideration the inner correlation between face expression and body motion.

“Both works use the same solution based on manual feature fusion, which does not take into consideration the inner correlation between face expression and the body motion, but fused both of these features using a methodical scheme.”

The restrictions of each feature extraction technique are accumulated when features are fused into one feature vector, reducing the capability of generalization of models.

“The works of Chen et al. (2013) and Gunes and Piccardi (2009) use several different feature extraction techniques, and perform the feature fusion by uniting all features in one feature vector.The restrictions of each technique are accumulated, reducing the capability of generalization of their models.”

The CNN has competitive results compared to the models of Chen et al. (2013) and Gunes and Piccardi (2009), and the MCCNN implementation outperformed these models.

“This shows that the CNN has competitive results, when compared to the models of Chen et al. (2013) and Gunes and Piccardi (2009), and when using the MCCNN implementation, it was possible to outperform these models.”

The proposed model improves the accuracy by more than 6% compared to Chen et al. (2013).

“The proposed model improves the accuracy by more than 6% compared to Chen et al. (2013).”

Our model achieves higher accuracy for the experiments with Face expression and Multimodal experiments, compared with the results reported in Chen et al. (2013).

“Our model achieves higher accuracy for the experiments with Face expression and Multimodal experiments, compared with the results reported in Chen et al. (2013).”

Chen et al. (2013) apply skin-color segmentation, head and hand tracking, and generate motion history images to represent body movement.

“In the work of Chen et al. (2013) they apply skin-color segmentation, head and hand tracking techniques to create a segmented image.After that, they generate motion history images to represent the body movement.”

In Chen et al. (2013), Gunes and Piccardi (2009), and our study, the combination of modalities achieved the highest accuracy in automatic emotional state recognition.

“In the studies of Chen et al. (2013) and Gunes and Piccardi (2009), and also ours, the combination of modalities achieved the highest accuracy in an automatic emotional state recognition task.”

The manual feature fusion approach relies on different feature extraction techniques, each having its own constraints.

“This approach relies on different feature extraction techniques, each one of them having its own constraints.”

In the Multimodal experiment, MCCNN increased accuracy by almost 10% compared to Gunes and Piccardi (2009) and more than 15% compared to Chen et al. (2013).

“Our strategy presented best results, and increased the accuracy by almost 10 % compared to Gunes and Piccardi (2009) and more than 15 % in comparison with Chen et al. (2013).”

In the Body Motion experiment, MCCNN provided worse results than Chen et al. (2013) and Gunes and Piccardi (2009).

“For the Body Motion experiment, our model provided worse results than the ones reported in the studies of Chen et al. (2013) and Gunes and Piccardi (2009).”

In the Facial Expression experiment, MCCNN achieved higher accuracy than Chen et al. (2013) and Gunes and Piccardi (2009).

“Our model was measured with a higher accuracy when compared with the approaches of Chen et al. (2013) and Gunes and Piccardi (2009), showing that our feature learning is more reliable.”

The MCCNN architecture achieves better results than the state-of-the-art approaches.

“The MCCNN architecture is shown to achieve better results than the state-of-the-art approaches.”

Chen et al. (2013) use 53 face landmarks to extract features and classify them with an SVM.

“In the research of Chen et al. (2013), they use 53 face landmarks to extract some features and classify them with an SVM.”

In both Chen et al. (2013) and Gunes and Piccardi (2009), a manual feature fusion is implemented.

“In both the studies of Chen et al. (2013) and Gunes and Piccardi (2009), a manual feature fusion is implemented.”