The DOI matches its Crossref record; 62 of the 63 references it cites were found in a registry or an index; none of the 66 sources its claims cite is checked for support yet.
Survey on AI-Based Multimodal Methods for Emotion Detection
Open accessAbstract
Automatic emotion recognition constitutes one of the great challenges providing new tools for more objective and quicker diagnosis, communication and research. Quick and accurate emotion recognition may increase possibilities of computers, robots, and integrated environments to recognize human emotions, and response accordingly to them a social rules. The purpose of this paper is to investigate the possibility of automated emotion representation, recognition and prediction its state-of-the-art and main directions for further research. We focus on the impact of emotion analysis and state of the arts of multimodal emotion detection. We present existing works, possibilities and existing methods to analyze emotion in text, sound, image, video and physiological signals. We also emphasize the most important features for all available emotion recognition modes. Finally, we present the available platform and outlines the existing projects, which deal with multimodal emotion analysis.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
This survey reviews AI-based multimodal emotion detection across text, sound, image/video, and physiological/motor signals, summarizing methods, datasets, tools, and projects. It reports that automated emotion analysis is feasible but still limited by context, language, real-world noise, and the need for personalization.
Key findings
- The survey covers emotion analysis in text, sound, image/video, and physiological/motor signals; it highlights multimodal challenges of inter-modality and intra-modality dynamics and cites datasets such as MOSI, ICT-MMMO, and MOUD, plus methods like Tensor Fusion Network and LSTM-based contextual feature extraction.
- Speech emotion recognition results include a chunk-level classwise averaged recognition rate of almost 70% for the 4-class problem anger, emphatic, neutral, and motherese on the FAU Aibo corpus; Greek speech emotion classification reached up to 75.15%, while Basque reached 80.05% and Spanish 74.82%.
- Physiological/motor emotion recognition reports include classification rates up to 92% using HRV and PRV fusion (sensitivity 95%, specificity 83.33%), prior general recognition rates from 57.77% to 85.46%, 84.7% by Jang et al., EEG average overall accuracy of 77.14% with MLPNN and 72.92% with kNN, highest gait accuracy of 80.5%, 54.5% for combining heart rate variability, respiration frequency, and facial expressions, and 61.39% for the best i2b2/VA/Cincinnati challenge team.
- Related projects and tools include openSMILE, openEAR (capable of generating >500 k features in real time with RTF < 0.1), ASC-Inclusion, INTERSPEECH ComParE, IntraFace, Microsoft Azure Emotion API, and Ekman micro-expression training tools.
Methods
- Design: a survey/state-of-the-art review of automated emotion representation, recognition, and prediction, with emphasis on multimodal emotion detection.
- Data or sample: no primary experimental sample; the paper reviews existing works, datasets/corpora, platforms, and projects, including MOSI, ICT-MMMO, MOUD, FAU Aibo, and MAHNOB-HCI.
- Main analysis: narrative synthesis organized by modality—text, sound, image/video, physiological and motor signals—followed by related projects and future directions.
Limitations
- Stated: limitations of current knowledge and experience concern tools for automatic non-invasive emotion measurement and analysis; key requirements include portability, non-intrusiveness, and low price, and further research is needed.
- Stated: speech emotion analysis highly depends on language; real-world factors such as background noise and non-speech vocalizations degrade results from controlled environments, and algorithms must be personalized to each person to be reliable.
- Evident from the text: as a survey, it provides no new experimental validation or systematic meta-analysis, and the authors note they have addressed only some of the most important issues.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
52 Not checked yet · 24 No basis given · 13 Own interpretation
None of the sources cited for this claim have been checked yet.
In the paper
“Several works on depression have emerged.They are based on social networks: Twitter [23,24] and Facebook [25,26].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [23] Wang, W., Hernandez, I., Newman, D.A., He, J., Bian, J.: Twitter analysis: studying US weekly trends in work stress and emotion. Appl. Psychol. 65(2), 355–378 (2016) source
- Not checked yetcites [24] Reece, A.G., Reagan, A.J., Lix, K.L., Dodds, P.S., Danforth, C.M., Langer, E.J.: Forecasting the onset and course of mental illness with Twitter data (Unpublished manuscript). https://arxiv.org/pdf/1608.07740.pdf
- Not checked yetcites [25] Park, J., Lee, D.S., Shablack, H., et al.: When perceptions defy reality: the relationships between depression and actual and perceived Facebook social support. J. Affect. Disord. 200, 37–44 (2016) source
- Not checked yetcites [26] Burke, M., Develin, M.: Once more with feeling: supportive responses to social sharing on Facebook. In: Proceedings of the ACM 2016 Conference on Computer Supported Cooperative Work, pp. 1462–1474 (2016) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Several multimodal datasets include sentiment annotations.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [6] Zadeh, A., Zellers, R., Pincus, E., Morency, L.P.: Multimodal sentiment intensity analysis in videos: facial gestures and verbal messages. IEEE Intell. Syst. 31(6), 82–88 (2016). https://doi.org/10.1109/mis.2016.94 source
- Not checked yetcites [7] Wöllmer, M., et al.: YouTube movie reviews: sentiment analysis in an audio-visual context. IEEE Intell. Syst. 28(3), 46–53 (2013) source
- Not checked yetcites [8] Perez-Rosas, V., Mihalcea, R., Morency, L.P.: Utterance-level multimodal sentiment analysis. In: ACL, vol. 1, pp. 973–982 (2013)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Zadeh et al. introduced the first multimodal dataset (MOSI) with opinion-level sentiment intensity annotations and studying the prototypical interaction patterns between facial gestures and spoken words when inferring sentiment intensity.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [6] Zadeh, A., Zellers, R., Pincus, E., Morency, L.P.: Multimodal sentiment intensity analysis in videos: facial gestures and verbal messages. IEEE Intell. Syst. 31(6), 82–88 (2016). https://doi.org/10.1109/mis.2016.94 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“To solve this problem, Zadeh introduced the Tensor Fusion Network (TFN), which combine the intra-modality and inter-modality models.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [9] Zadeh, A., Chen, M., Poria, S., Cambria, E., Morency, L.P.: Tensor fusion network for multimodal sentiment analysis, arXiv:1707.07250 . In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 7–11 September 2017, Copenhagen, Denmark, pp. 1103–1114. Association for Computational Linguistics source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Zadeh et al. introduced the first multimodal dataset (MOSI) with opinion-level sentiment intensity annotations and studying the prototypical interaction patterns between facial gestures and spoken words when inferring sentiment intensity.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [6] Zadeh, A., Zellers, R., Pincus, E., Morency, L.P.: Multimodal sentiment intensity analysis in videos: facial gestures and verbal messages. IEEE Intell. Syst. 31(6), 82–88 (2016). https://doi.org/10.1109/mis.2016.94 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Inter-modality dynamics is modeled with a new multimodal fusion approach, named Tensor Fusion, which explicitly aggregates unimodal, bimodal and trimodal interactions.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [9] Zadeh, A., Chen, M., Poria, S., Cambria, E., Morency, L.P.: Tensor fusion network for multimodal sentiment analysis, arXiv:1707.07250 . In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 7–11 September 2017, Copenhagen, Denmark, pp. 1103–1114. Association for Computational Linguistics source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The studies conducted by Albert Mehrabian in 1967 established the 7%-38%-55% rule, also known as the “3V rule”: 7% of the communication is verbal, 38% of the communication is vocal and 55% of the communication is visual [1].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 1 Introduction
What the sources say
- Not checked yetcites [1] Mehrabian, A., Ferris, S.R.: Inference of attitudes from nonverbal communication in two channels. J. Consult. Psychol. 31(3), 248 (1967) source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“The third one, the dynamics, play the most important role for emotion face analysis [44].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
What the sources say
- Not checked yetcites [44] Jack, R.E., Schyns, P.G.: The human face as a dynamic tool for social communication. Curr. Biol. Rev. 25(14), R621–R634 (2015). https://doi.org/10.1016/j.cub.2015.05.052 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Best results have been obtained on the chunk level where a classwise averaged recognition rate of almost 70% for the 4-class problem anger, emphatic, neutral and motherese has been achieved.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
What the sources say
- Not checked yetcites [35] Steidl, S.: Automatic classification of emotion-related user states in spontaneous children’s speech. Ph.D. thesis, Erlangen (2009)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These factors will make the real emotion recognition trained on the data from the controlled environment unsuccessful.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 55
- It may state two separate claims as one. 21
In the paper
“Interesting work is realized by Poria et al., who developed a LTSM-based network to extract contextual features from the video for multimodal sentiment analysis [10].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [10] Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., Morency, L.P.: Context-dependent sentiment analysis in user-generated videos. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 873–883 (2017) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“For other examples of data sets we can cite ICT-MMMO [7] and MOUD [8] datasets.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [7] Wöllmer, M., et al.: YouTube movie reviews: sentiment analysis in an audio-visual context. IEEE Intell. Syst. 28(3), 46–53 (2013) source
- Not checked yetcites [8] Perez-Rosas, V., Mihalcea, R., Morency, L.P.: Utterance-level multimodal sentiment analysis. In: ACL, vol. 1, pp. 973–982 (2013)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is not yet possible to find out an ideal case to marking the opinion in a text written by different users, because the text does not follow the rules.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.1 Emotion in Text
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“With an average of 500 million messages sent per day, the platform seems ideal for tracking opinions on various subjects.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Therefore, it is impossible to schedule every possible case.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.1 Emotion in Text
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Moreover, very often the same phrase can be considered as positive for one person and negative for another one.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.1 Emotion in Text
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, automated interpretation is complicated by embedded links, abbreviations and misspellings.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, on the Internet, everyone uses his or her own style and vocabulary, what adds extra difficulty to the task.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.1 Emotion in Text
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the current growth of artificial intelligence, machine learning and natural language processing, driven by new technological possibilities, it is possible to automate the analysis of vast amounts of publicly published data.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Several authors have been interested in the use of emoticons to complete the sentiment analysis.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [27] Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. J. CS224N Proj. Rep., Stanford 1, 12 (2009)
- Not checked yetcites [28] Liu, K.L., Li, W.J., Guo, M.: Emoticon smoothed language models for Twitter sentiment analysis. In: AAAI (2012)
- Not checked yetcites [29] Węgrzyn-Wolska, K., Bougueroua, L., Yu, H., Zhong, J.: Explore the effects of emoticons on Twitter sentiment analysis. In: Proceedings of Third International Conference on Computer Science & Engineering (CSEN 2016), 27–28 August 2016, Dubai, UAE source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Moreover, opinions and sentiments are highly sensitive to the context and the field in which they are used: the same string might be positive in one context and negative in another.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.1 Emotion in Text
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“They proposed a new computational representation, called multimodal dictionary, based on a language-gesture study and evaluated the proposed approach in a speaker-independent model for sentiment intensity prediction [6].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [6] Zadeh, A., Zellers, R., Pincus, E., Morency, L.P.: Multimodal sentiment intensity analysis in videos: facial gestures and verbal messages. IEEE Intell. Syst. 31(6), 82–88 (2016). https://doi.org/10.1109/mis.2016.94 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Additionally, they presented a multimodal sentiment analysis framework, which includes sets of relevant features for text and visual data, as well as a simple technique for fusing the features extracted from different modalities [11].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [11] Poria, S., Cambria, E., Howard, N., Huang, G.B., Hussain, A.: Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing 174(Part A), 50–59 (2016). https://doi.org/10.1016/j.neucom.2015.01.095 . ISSN 0925-2312 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Contextual changes of polarity indicating words, such as negation, sarcasm as well as weak syntactic structures make it troublesome for both machines and humans to safely determine polarity of messages.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Text Analysis
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“The automation of sentiment detection on these social networks has gained attention for various purposes [18][19][20].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [18] Ghiassi, M., Skinner, J., Zimbra, D.: Twitter brand sentiment analysis: a hybrid system using n-gram analysis and dynamic artificial neural network. Expert Syst. Appl. 40(16), 6266–6282 (2013) source
- Not checked yetcites [19] Zhou, X., Tao, X., Yong, J., Yang, Z.: Sentiment analysis on tweets for social events. In: Proceedings of the 2013 IEEE 17th International Conference on Computer Supported Cooperative Work in Design, CSCWD 2013, 27–29 June 2013, pp. 557–562 (2013) source
- Not checked yetcites [20] Salathé, M., Vu, D.Q., Khandelwal, S., Hunter, D.R.: The dynamics of health behavior sentiments on a large online social network. EPJ Data Sci. 2, 4 (2013). https://doi.org/10.1140/epjds16 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Intra-modality dynamics is modeled through three Modality Embedding Subnetworks, for language, visual and acoustic modalities, respectively [9].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Multimodal emotion analysis becomes very popular research domain.
What the sources say
- Not checked yetcites [9] Zadeh, A., Chen, M., Poria, S., Cambria, E., Morency, L.P.: Tensor fusion network for multimodal sentiment analysis, arXiv:1707.07250 . In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, 7–11 September 2017, Copenhagen, Denmark, pp. 1103–1114. Association for Computational Linguistics source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“According to the Kepios study [17], the number of active users of social networks increased by 13% in 2017 to reach 3.3 billion users in April 2018.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [17] Kepios: Digital in 2018, essential insights into internet, social media, mobile, and ecommerce use around the world, April 2018. https://kepios.com/data/
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The AutoEmotive (MIT Media Lab) is a prototype equipped with sensors and a camera placed on the steering wheel [5].This vehicle measures the level of stress and fatigue of the driver.When the need arises, he puts a background music, changes the temperature and light in the vehicle interior, or still proposes to follow a less stressful journey.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 1 Introduction
What the sources say
- Not checked yetcites [5] Hernandez, J., et al.: AutoEmotive: bringing empathy to the driving experience to manage stress. In: DIS 2014, 21–25 June 2014, Vancouver, BC, Canada. ACM (2014). http://dx.doi.org/10.1145/2598784.2602780 . 978-1-4503-2903-3/14/06 source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The first study in the field of emotion detection was born during the sixties/seventies.The most prominent example is that of mood rings [2].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 1 Introduction
What the sources say
- Not checked yetcites [2] Mood Ring Monitors Your State of Mind, Chicago Tribune, 8 October 1975, at C1: Ring Buyers Warm Up to Quartz Jewelry That Is Said to Reflect Their Emotions. The Wall Street Journal, 14 October 1975, at p. 16; and “A Ring Around the Mood Market”, The Washington Post, 24 November 1975, at B9
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The frequent expression of negative emotion words on social media has been linked to depression.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [22] Seabrook, E.M., Kern, M.L., Fulcher, B.D., Rickard, N.S.: Predicting depression from language-based emotion dynamics: longitudinal analysis of Facebook and Twitter status updates. J. Med. Internet Res. 20(5), e168 (2018). https://doi.org/10.2196/jmir.9267 source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Furthermore, the very short format messages facilitate classification since short messages rarely discuss more than one topic.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Presented approach may be successful, but the limitations of the current knowledge and experience still concern tools for automatic non-invasive emotion measurement and analysis.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 5 Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds.
In the paper
“Their experiences confirmed that the human judgement is still the best one, with the highest recognition rates, followed by stick-figure models, point-light displays, and then XFace animations.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
What the sources say
- Not checked yetcites [48] Afzal, S., Sezgin, T.M., Gao, Y., Robinson, P.: Perception of emotional expressions in different representations using facial feature points. IEEE (2009). 978-1-4244-4799 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The author in [35] focused on mono-modal systems with speech as only input channel.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
What the sources say
- Not checked yetcites [35] Steidl, S.: Automatic classification of emotion-related user states in spontaneous children’s speech. Ph.D. thesis, Erlangen (2009)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It allows to communicate even complex social concepts, in various contexts, thus may be regarded not only supplementary, but also as basic way of emotion recognition.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.3 Emotion in Image and Video
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“He proposed FAU Aibo Emotion Corpus, which is a speech corpus with naturally occurring emotion-related states.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
What the sources say
- Not checked yetcites [35] Steidl, S.: Automatic classification of emotion-related user states in spontaneous children’s speech. Ph.D. thesis, Erlangen (2009)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Thus, facial expressions as a noninvasive method are used in behavioral science and in clinical.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“To reduce the complexity of automatic affective inference, measurement and interpretation of facial expressions, Ekman and Friesen developed in 1978 special system for objectively measuring facial movement; the Facial Action Coding System (FACS) [45].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
What the sources say
- Not checked yetcites [45] Ekman, P., Friesen, W., Hager, J.: Facial action coding system: Research Nexus. Network Research Information, Salt Lake City (2002)
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There are three main dimensions of facial variation: morphology, complexion and dynamics.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Authors in [27] utilize Twitter API to get training data that contain emoticons like :) and :(.They use these emoticons as noisy labels.Tweets with :) are thought to be positive training data and tweets with :( are thought to be negative training data.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [27] Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. J. CS224N Proj. Rep., Stanford 1, 12 (2009)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Emotion analysis of speech is possible; however, it highly depends of the language.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Accuracy
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“The authors in [29] explored the influence of emoticons on TSA.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [29] Węgrzyn-Wolska, K., Bougueroua, L., Yu, H., Zhong, J.: Explore the effects of emoticons on Twitter sentiment analysis. In: Proceedings of Third International Conference on Computer Science & Engineering (CSEN 2016), 27–28 August 2016, Dubai, UAE source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Eleven emotion-related states are labeled on the word level.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
What the sources say
- Not checked yetcites [35] Steidl, S.: Automatic classification of emotion-related user states in spontaneous children’s speech. Ph.D. thesis, Erlangen (2009)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“As first two dimensions, morphology and complexion, deals with static dimensions, there are very useful for facial recognition but their role for emotion analyze is not significant.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There are many factors such as background noise and not speech voice like a laugh, a whimper, a cry, a sigh, etc., which greatly aggravate the results obtained in a controlled environment.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Not only basic emotions can be analyzed, but also the cognitive assessment (interpretation of stimuli in the environment) and their physiological response.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 5 Conclusion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The face is equipped with a large number of independent muscles, which can be activated at different levels of intensity.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“First, the point-based display was created from the output of a commercial automatic face-tracker [49] on a black background.Second one, stick figure models, which connecting the automatically tracked feature-points using straight lines and sketching eyes using typical shape.The last one, 3D XFace Animation was a 3D animated facial expressions and displays created using XFace open source toolkit.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
What the sources say
- Not checked yetcites [48] Afzal, S., Sezgin, T.M., Gao, Y., Robinson, P.: Perception of emotional expressions in different representations using facial feature points. IEEE (2009). 978-1-4244-4799 source
- Not checked yetcites [49] http://www.nevenvision.com . Licensed from Google Inc.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“The recognition of emotions based on the voice has been studied for decades [31][32][33][34].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
What the sources say
- Not checked yetcites [31][32][33][34]
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We showed that automated emotion analysis is possible and can be very useful for improving the exactness of the computer/machine reaction and making possible to anticipate the emotional state of the interlocutor more quickly.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 5 Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“FACS, based on a system originally developed by a Swedish anatomist named Hjortsjö [46] became the standard for identifying any movement of the face.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
What the sources say
- Not checked yetcites [45] Ekman, P., Friesen, W., Hager, J.: Facial action coding system: Research Nexus. Network Research Information, Salt Lake City (2002)
- Not checked yetcites [46] Hjorztsjö, C.H.: Man’s face and mimic language (1969). https://books.google.com/books/about/Man_s_Face_and_Mimic_Laguage.html?id=BakQAQAAIAAJ
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
In the paper
“Later, Ekman and Sejnowski studied also computer based facial measurements [47].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
What the sources say
- Not checked yetcites [47] Ekman, P., Huang, T.S., Sejnowski, T.J., et al.: Final report to NSF of the planning workshop on facial expression understanding, vol. 378. Human Interaction Laboratory, University of California, San Francisco (1993)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The eyes and mouth have high importance to emotion detection; therefore, to successfully recognize an emotion, the observations mostly rely on the eye and mouth regions.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Furthermore, the actions of eyes and mouth allowed grouping the expressions in a continuous space, ranging from sadness and fear (reliance on the eyes) to disgust and happiness (mouth).”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Consequently, the face is capable of generating a high number of complex dynamic facial expression patterns.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“FACS is a common standard to systematically categorize and to index the physical expression of emotions.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“FAU Aibo is a corpus of spontaneous, emotionally colored speech of German children at the age of 10 to 13 years interacting with the Sony Robot Aibo.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
What the sources say
- Not checked yetcites [35] Steidl, S.: Automatic classification of emotion-related user states in spontaneous children’s speech. Ph.D. thesis, Erlangen (2009)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Combining these observations with facial AUs increase knowledge about the areas involved in displaying each emotion.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“In [28], authors present the ESLAM (Emoticon Smoothed LAnguage Models) which combine fully supervised methods and distantly supervised methods.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Social Networks
What the sources say
- Not checked yetcites [28] Liu, K.L., Li, W.J., Guo, M.: Emoticon smoothed language models for Twitter sentiment analysis. In: AAAI (2012)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Automatic emotion analysis requires advanced recognition and modeling, very often based on artificial intelligence systems.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 5 Conclusion
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper may give no reason why it holds.
In the paper
“They are common to all humans confirming Darwin's evolutionary thesis.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.3 Emotion in Image and Video
What the sources say
- Not checked yetcites [43] Ekman, P.: Facial expression and emotion. Am. Psychol. 48, 384–392 (1993) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, the majority of such a system are speech-oriented.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Nonverbal behavior constitutes useful means of communication in addition to spoken language.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.3 Emotion in Image and Video
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is possible to register the facial information automatically in real-time, without requiring any specialized equipment except a simple video camera.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4 Automatic Facial Emotion
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“Recent research has shown that under certain circumstances multimodal emotion recognition is possible even in real time [37].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Audio Analysis: Background
What the sources say
- Not checked yetcites [37] Kukolja, D., Popović, S., Horvat, M., Kovač, B., Ćosić, K.: Comparative analysis of emotion estimation methods based on physiological measurements for real-time applications. Int. J. Hum.-Comput. Stud. 72(10), 717–727 (2014) source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Voice-based emotion recognition system relying on audio input has low requirements for hardware.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.2 Emotion Detection in the Sound
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
In the paper
“They showed that human gait reflects the walker's emotional state.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Physiological Signals
What the sources say
- Not checked yetcites [62] Li, S., Cui, L., Zhu, C., Li, B., Zhao, N., Zhu, T.: Emotion recognition using Kinect motion capture data of human gaits. PeerJ 4, e2364 (2016) source
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Such an approach may cause quicker development of more user-friendly systems and environments supporting everyday life and work.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 5 Conclusion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“that more higher set of analyzed signals and more recognized emotions provides better results [64][65][66].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Accuracy
What the sources say
- Not checked yetcites [64] Khezri, M., Firoozabadi, M., Sharafat, A.R.: Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals. Comput. Methods Programs Biomed. 122(2), 149–164 (2015) source
- Not checked yetcites [65] Gouizi, K., Bereksi Reguig, F., Maaoui, C.: Emotion recognition from physiological signals. J. Med. Eng. Technol. 35(6–7), 300–307 (2011) source
- Not checked yetcites [66] Verma, G.K., Tiwary, U.S.: Multimodal fusion framework: a multiresolution approach for emotion classification and recognition from physiological signals. Neuroimage 102(Part 1), 162–172 (2014) source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“Results of the 2011 i2b2/VA/Cincinnati Medical Natural Language Processing Challenge showed that the best of 24 participants' teams achieved accuracy of 61.39% [67].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Accuracy
What the sources say
- Not checked yetcites [67] Yang, H., Willis, A., de Roeck, A., Nuseibeh, B.: A hybrid model for automatic emotion recognition in suicide notes. Biomed. Inform. Insights 5(Suppl. 1), 17–30 (2012)
plainn = 24- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Novel intelligent systems may be friendlier, preventing the computers from acting inappropriately.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 5 Conclusion
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Several popular commercial packages offer specific facial image analysis tasks, including facial expression recognition, facial attribute analysis, and face tracking.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4.1 Existing Tools for Automatic Facial Emotion Recognition
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“Özerdem and Polat used EEG signal -resultant average overall accuracy was 77.14% (using MLPNN) and 72.92% (using kNN) [56].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Accuracy
What the sources say
- Not checked yetcites [56] Wei, W., Jia, Q., Feng, Y., Chen, G.: Emotion recognition based on weighted fusion strategy of multichannel physiological signals. Comput. Intell. Neurosci. 2018, 9 (2018). 5296523 source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper relies on this without support.
One of the most challenges in multimodal emotion analysis is to model the interactions between language, visual and acoustic behaviors that change the observation of the expressed emotion (inter-modality dynamics).
“One of the most challenges in multimodal emotion analysis is to model the interactions between language, visual and acoustic behaviors that change the observation of the expressed emotion (named the inter-modality dynamics).”
The paper relies on this without support.
A second challenge in multimodal emotion analysis (intra-modality dynamics) is to efficiently explore emotion on highly expressive nature modality.
“A second challenge in multimodal emotion analysis (named intra-modality dynamics) is to efficiently explore emotion, not only on one but on highly expressive nature modality (ex.spoken language where proper language structure is often ignored, video and acoustic modalities which are expressed through both space and time.”
The paper relies on this without support.
Traditional emotion recognition and classification associate the 'value' to at least several basic emotions such as happiness, joy, neutrality, surprise, sadness, disgust, anger, or fear.
“Traditional emotion recognition and classification associate the “value” to at least several basic emotions such as happiness, joy, neutrality, surprise, sadness, disgust, anger, or fear.”
The paper relies on this without support.
Multimodal emotion analysis has become a very popular research domain.
“Multimodal emotion analysis becomes very popular research domain.”
The paper relies on this without support.
The inability of today's systems to recognize, express and feel emotions limits their ability to act intelligently and interact naturally with humans.
“The inability of today's systems to recognize, express and feel emotions limits their ability to act intelligently and interact naturally with humans.”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“In a survey on multimodal approaches for emotion detection, Marechal et al. (2019) show that affective state can be extracted both from the user's pose (Zacharatos et al., 2014), physiological features like blood pressure (Shu et al., 2018), and from facial features (Ekman, 1993).”
From Modeling Feedback in Interaction With Conversational Agents—A Review (2022) · cites this paper as Marechal et al., 2019
In the citing paper
“Recently, the valuable works of [32, 33] considered several multimodal practices that apply only to specific multimodal use cases and applications, such as emotion recognition [32], human activity and context recognition [33].”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [32]
In the citing paper
“but to the best of our knowledge, there is no recent work that directly addresses the latest advances in deep multimodal learning particularly for the computer vision community.”
From A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets (2022) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4.1 Existing Tools for Automatic Facial Emotion Recognition
“IntraFace (IF) [50], a publicly available software package offering:”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“50. Amira, T., Dan, I., Az-Eddine, B., et al.: Monitoring chronic disease at home using connected devices.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Physiological Signals
“Research by Jang et al. used ECG, EDA, SKT and more sophistical machine learning (ML) algorithms [58]:”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“58. Li, S., Cui, L., Zhu, C., Li, B., Zhao, N., Zhu, T.: Emotion recognition using Kinect motion capture data of human gaits.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Accuracy
“Research by Goshvarpour et al. using HRV and PRV, and fusion rules (feature level, decision level) showed classification rates improved up to 92% (sensitivity: 95%, specificity: 83.33%) [63].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“63. Yang, H., Willis, A., de Roeck, A., Nuseibeh, B.: A hybrid model for automatic emotion recognition in suicide notes.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.4.1 Existing Tools for Automatic Facial Emotion Recognition
“The Emotion API (Microsoft Azure) Microsoft Azure proposes on-line API to recognize emotions in images and in videos [51].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“51. Shu, L., et al.: A review of emotion recognition using physiological signals.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > 3.5 Emotion Detected by the Physiological and Motor Signals
“As an example, we can cite work done by Amira et al. [54] as a good example of using the emotion analyze for healthcare purpose.”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“54. Jang, E.H., Park, B.J., Park, M.S., Kim, S.H., Sohn, J.H.: Analysis of physiological signals for recognition of boredom, pain, and surprise emotions.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Physiological Signals
“Kortelainen et al. showed results of combining physiological signals (heart rate variability parameters, respiration frequency) and facial expressions were studied [59].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“59. Goshvarpour, A., Abbasi, A.: Goshvarpour, A: Fusion of heart rate variability and pulse rate variability for emotion recognition using lagged poincare plots.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Physiological Signals
“The whole framework of emotion recognition based on physiological signals has recently been described by [55].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“55. Kortelainen, J., Tiinanen, S., Huang, X., Li, X., Laukka, S., Pietikäinen, M., Seppänen, T.: Multimodal emotion recognition by combining physiological signals and facial expressions: a preliminary study.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Physiological Signals
“Decision-level weight fusion strategy for emotion recognition in multichannel physiological signals using MAHNOB-HCI database has been recently described [56].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“56. Zacharatos, H., Gatzoulis, C., Chrysanthou, Y.L.: Automatic emotion recognition based on body movement analysis: a survey.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Physiological Signals
“Özerdem and Polat used EEG signal, discrete wavelet transform and machine learning techniques (multilayer perceptron neural network -MLPNN, and k-nearest neighbors -kNN algorithms) [57].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“57. Tsui, W.H., Lee, P., Hsiao, T.C.: The effect of emotion on keystroke: an experimental study using facial feedback hypothesis.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Accuracy
“Results of the 2011 i2b2/VA/Cincinnati Medical Natural Language Processing Challenge showed that the best of 24 participants' teams achieved accuracy of 61.39% [67].”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“67. Schuller, B., et al.: Affective and behavioural computing: lessons learnt from the first computational paralinguistics challenge.”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Survey on AI-Based Multimodal Methods for Emotion Detection > Micro Expressions Training Tool
“Ekman has created several training tools [53] to enhance understanding of emotions and relationships:”
§ Survey on AI-Based Multimodal Methods for Emotion Detection > References
“53. Özerdem, M.S., Polat, H.: Emotion recognition based on EEG features in movie clips with channel selection.”
Citations
Cites
Picard, R. W. (1997) · MIT Press · doi:10.7551/mitpress/1140.001.0001 · Cited by 7,382
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,957
Opinion mining and sentiment analysis
Pang, B., & Lee, L. (2008) · Foundations and Trends® in Information Retrieval · doi:10.1561/1500000011 · Cited by 6,372
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis lectures on human language technologies · doi:10.2200/s00416ed1v01y201204hlt016 · Cited by 3,995
Ekman, P. (1993) · American Psychologist · doi:10.1037/0003-066x.48.4.384 · Cited by 3,441
Sentiment Analysis and Opinion Mining
Zhang, L., & Liu, B. (2017) · Springer eBooks · doi:10.1007/978-1-4899-7687-1_907 · Cited by 3,307
Toward machine emotional intelligence: analysis of affective physiological state
Picard, R. W., Vyzas, E., & Healey, J. (2001) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/34.954607 · Cited by 2,334
Tensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d17-1115 · Cited by 2,102
Inference of attitudes from nonverbal communication in two channels.
Mehrabian, A., & Ferris, S. R. (1967) · Journal of Consulting Psychology · doi:10.1037/h0024648 · Cited by 952
Context-Dependent Sentiment Analysis in User-Generated Videos
Poria, S., Cambria, E., Hazarika, D., Majumder, N., Zadeh, A., & Morency, L.-P. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p17-1081 · Cited by 939
A review of emotion recognition using physiological signals
Shu, L., Xie, J., Yang, M., Li, Z., Li, Z., Liao, D., Xu, X., & Yang, X. (2018) · Sensors · doi:10.3390/s18072074 · Cited by 923
Multimodal Sentiment Intensity Analysis in Videos: Facial Gestures and Verbal Messages
Zadeh, A., Zellers, R., Pincus, E., & Morency, L.-P. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.94 · Cited by 852
Analysis of emotion recognition using facial expressions, speech and multimodal information
Busso, C., Deng, Z., Yildirim, S., Bulut, M., Lee, C. M., Kazemzadeh, A., Lee, S., Neumann, U., & Narayanan, S. (2004) · Proceedings of the 6th international conference on Multimodal interfaces · doi:10.1145/1027933.1027968 · Cited by 840
Short text classification in twitter to improve information filtering
Sriram, B., Fuhry, D., Demir, E., Ferhatosmanoğlu, H., & Demirbaş, M. (2010) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/1835449.1835643 · Cited by 734
Dellaert, F., Polzin, T., & Waibel, A. (1996) · International Conference on Spoken Language Processing (ICSLP) · doi:10.1109/icslp.1996.608022 · Cited by 578
Fusing audio, visual and textual clues for sentiment analysis from multimodal content
Poria, S., Cambria, E., Howard, N., Huang, G.-B., & Hussain, A. (2016) · Neurocomputing · doi:10.1016/j.neucom.2015.01.095 · Cited by 534 · cited 1×
“Additionally, they presented a multimodal sentiment analysis framework, which includes sets of relevant features for text and visual data, as well as a simple technique for fusing the features extracted from different modalities [11].”
BackgroundGhiassi, M., Skinner, J., & Zimbra, D. (2013) · Expert Systems with Applications · doi:10.1016/j.eswa.2013.05.057 · Cited by 485
OpenEAR — Introducing the munich open-source emotion and affect recognition toolkit
Eyben, F., Wollmer, M., & Schuller, B. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349350 · Cited by 418
The Human Face as a Dynamic Tool for Social Communication
Jack, R. E., & Schyns, P. G. (2015) · Current Biology · doi:10.1016/j.cub.2015.05.052 · Cited by 405
Youtube movie reviews: Sentiment analysis in an audio-visual context
Wollmer, M., Weninger, F., Knaup, T., Schuller, B., Sun, C., Sagae, K., & Morency, L.-P. (2013) · IEEE Intelligent Systems · doi:10.1109/mis.2013.34 · Cited by 403
Forecasting the onset and course of mental illness with Twitter data
Reece, A. G., Reagan, A. J., Lix, K. L. M., Dodds, P. S., Danforth, C. M., & Langer, E. J. (2017) · Scientific Reports · doi:10.1038/s41598-017-12961-9 · Cited by 401
Verma, G. K., & Tiwary, U. S. (2014) · NeuroImage · doi:10.1016/j.neuroimage.2013.11.007 · Cited by 354
Emoticon Smoothed Language Models for Twitter Sentiment Analysis
Liu, K.-L., Li, W.-J., & Guo, M. (2021) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v26i1.8353 · Cited by 255
(2013) · Cited by 236
Emotion Recognition based on Phoneme Classes
Lee, C. M., Yildirim, S., Bulut, M., Kazemzadeh, A., Busso, C., Deng, Z., Lee, S., & Narayanan, S. (2004) · Interspeech 2004 · doi:10.21437/interspeech.2004-322 · Cited by 228
Class-level spectral features for emotion recognition
Bitouk, D., Verma, R., & Nenkova, A. (2010) · Speech Communication · doi:10.1016/j.specom.2010.02.010 · Cited by 219
Analysis of physiological signals for recognition of boredom, pain, and surprise emotions
Jang, E.-H., Park, B.-J., Park, M.-S., Kim, S.-H., & Sohn, J.-H. (2015) · Journal of Physiological Anthropology · doi:10.1186/s40101-015-0063-5 · Cited by 159
Seabrook, E. M., Kern, M. L., Fulcher, B. D., & Rickard, N. S. (2018) · Journal of Medical Internet Research · doi:10.2196/jmir.9267 · Cited by 144
Emotion recognition based on EEG features in movie clips with channel selection
Özerdem, M. S., & Polat, H. (2017) · Brain Informatics · doi:10.1007/s40708-017-0069-3 · Cited by 136
la Torre, F. D., Chu, W.-S., Xiong, X., Vicente, F., Ding, X., & Cohn, J. (2015) · International Conference on Automatic Face and Gesture Recognition · doi:10.1109/fg.2015.7163082 · Cited by 125
Sentiment analysis on tweets for social events
Zhou, X., Tao, X., Yong, J., & Yang, Z. (2013) · Proceedings of the 2013 IEEE 17th International Conference on Computer Supported Cooperative Work in Design (CSCWD) · doi:10.1109/cscwd.2013.6581022 · Cited by 124
Twitter Analysis: Studying US Weekly Trends in Work Stress and Emotion
Wang, W., Hernandez, I., Newman, D. A., He, J., & Bian, J. (2016) · Applied Psychology · doi:10.1111/apps.12065 · Cited by 120
The dynamics of health behavior sentiments on a large online social network
Salathé, M., Vu, D. Q., Khandelwal, S., & Hunter, D. R. (2013) · EPJ Data Science · doi:10.1140/epjds16 · Cited by 116
The EU-Emotion Stimulus Set: A validation study
O’Reilly, H., Pigat, D., Fridenson-Hayo, S., Berggren, S., Tal, S., Golan, O., Bölte, S., Baron‐Cohen, S., & Lundqvist, D. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0601-4 · Cited by 114
Park, J., Lee, D. S., Shablack, H., Verduyn, P., Deldin, P., Ybarra, O., Jonides, J., & Kross, E. (2016) · Journal of Affective Disorders · doi:10.1016/j.jad.2016.01.048 · Cited by 104
Emotion recognition from physiological signals
GOUIZI, K., BEREKSI REGUIG, F., & MAAOUI, C. (2011) · Journal of Medical Engineering & Technology · doi:10.3109/03091902.2011.601784 · Cited by 92
Khezri, M., Firoozabadi, M., & Sharafat, A. R. (2015) · Computer Methods and Programs in Biomedicine · doi:10.1016/j.cmpb.2015.07.006 · Cited by 89
Once More with Feeling: Supportive Responses to Social Sharing on Facebook
Burke, M., & Develin, M. (2016) · Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing · doi:10.1145/2818048.2835199 · Cited by 89 · cited 3×
“The aim of [22] was to report on the associations between depression severity and the variability (time-unstructured) and instability (time-structured) in emotion word expression on Facebook and Twitter across status updates.”
BackgroundInfluentialEmotion Recognition Based on Weighted Fusion Strategy of Multichannel Physiological Signals
Wei, W., Jia, Q., Feng, Y., & Chen, G. (2018) · Computational Intelligence and Neuroscience · doi:10.1155/2018/5296523 · Cited by 82
AutoEmotive: bringing empathy to the driving experience to manage stress
Hernandez, J., McDuff, D., Benavides, X., Amores, J., Maes, P., & Picard, R. (2014) · Proceedings of the 2014 companion publication on Designing interactive systems · doi:10.1145/2598784.2602780 · Cited by 79
Automatic emotion recognition based on body movement analysis: a survey
Zacharatos, H., Gatzoulis, C., & Chrysanthou, Y. L. (2014) · IEEE Computer Graphics and Applications · doi:10.1109/mcg.2014.106 · Cited by 77
Emotion recognition using Kinect motion capture data of human gaits
Li, S., Cui, L., Zhu, C., Li, B., Zhao, N., & Zhu, T. (2016) · PeerJ · doi:10.7717/peerj.2364 · Cited by 71
Techniques and applications of emotion recognition in speech
Lugović, S., Đunđer, I., & Horvat, M. (2016) · International Convention on Information and Communication Technology, Electronics and Microelectronics · doi:10.1109/mipro.2016.7522336 · Cited by 67
Kukolja, D., Popović, S., Horvat, M., Kovač, B., & Ćosić, K. (2014) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2014.05.006 · Cited by 66
A Hybrid Model for Automatic Emotion Recognition in Suicide Notes
Yang, H., Willis, A., Roeck, A. N. D., & Nuseibeh, B. (2012) · Biomedical Informatics Insights · doi:10.4137/bii.s8948 · Cited by 63
Schuller, B. W., Weninger, F. J., Zhang, Y., Ringeval, F., Batliner, A. M., Steidl, S., Eyben, F., Marchi, E., Vinciarelli, A., Scherer, K. R., et al. (2018) · Computer Speech & Language · doi:10.1016/j.csl.2018.02.004 · Cited by 56
Goshvarpour, A., Abbasi, A., & Goshvarpour, A. (2017) · Australasian Physical & Engineering Sciences in Medicine · doi:10.1007/s13246-017-0571-1 · Cited by 46
Fisher Kernels on Phase-Based Features for Speech Emotion Recognition
Deng, J., Xu, X., Zhang, Z., Frühholz, S., Grandjean, D., & Schuller, B. W. (2016) · Lecture notes in electrical engineering · doi:10.1007/978-981-10-2585-3_15 · Cited by 32
Kortelainen, J., Tiinanen, S., Huang, X., Li, X., Laukka, S., Pietikainen, M., & Seppanen, T. (2012) · 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society · doi:10.1109/embc.2012.6347175 · Cited by 30
Perception of emotional expressions in different representations using facial feature points
Afzal, S., Sezgin, T. M., Gao, Y., & Robinson, P. (2009) · 2009 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops · doi:10.1109/acii.2009.5349549 · Cited by 25
Arruti, A., Cearreta, I., Álvarez, A., Lazkano, E., & Sierra, B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0108975 · Cited by 21
The effect of emotion on keystroke: An experimental study using facial feedback hypothesis
Tsui, W.-H., Lee, P.-M., & Hsiao, T. (2013) · Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) · doi:10.1109/embc.2013.6610139 · Cited by 17
Detection and Analysis of Emotion from Speech Signals
Davletcharova, A., Sugathan, S., Abraham, B., & James, A. P. (2015) · Procedia Computer Science · doi:10.1016/j.procs.2015.08.032 · Cited by 16
Explore the Effects of Emoticons on Twitter Sentiment Analysis
Wegrzyn-Wolska, K., Bougueroua, L., Yu, H., & Zhong, J. (2016) · Computer Science & Information Technology ( CS & IT ) · doi:10.5121/csit.2016.61006 · Cited by 13
Self-citationRRSS - Rating Reviews Support System Purpose Built for Movies Recommendation
Dziczkowski, G., & Węgrzyn-Wolska, K. M. (2007) · Springer eBooks · doi:10.1007/978-3-540-72575-6_14 · Cited by 12
Self-citationMonitoring chronic disease at home using connected devices
Amira, T., Istrate, D., Az-eddine, B., Ngo, H. H., Said, G., & Węgrzyn-Wolska, K. M. (2018) · International Conference on System of Systems Engineering · doi:10.1109/sysose.2018.8428754 · Cited by 12
Self-citationAn Autonomous System Designed for Automatic Detection and Rating of Film Reviews
Dziczkowski, G., & Węgrzyn-Wolska, K. M. (2008) · IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology (WI-IAT) · doi:10.1109/wiiat.2008.262 · Cited by 10
Self-citationDziczkowski, G., & Węgrzyn-Wolska, K. M. (2008) · doi:10.5220/0001894402180223 · Cited by 9
Self-citationChaspari, T., Soldatos, C., & Maragos, P. (2015) · The World Journal of Biological Psychiatry · doi:10.3109/15622975.2015.1012228 · Cited by 8
Tyburek, K., Prokopowicz, P., Kotlarz, P., & Řepka, M. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-19324-3_44 · Cited by 6
Self-citationFuzzy System for the Classification of Sounds of Birds Based on the Audio Descriptors
Tyburek, K., Prokopowicz, P., & Kotlarz, P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-07176-3_61 · Cited by 4
Self-citationTensor Fusion Network for Multimodal Sentiment Analysis
Zadeh, A., Chen, M., Poria, S., Cambria, E., & Morency, L.-P. (2017) · arXiv · doi:10.48550/arxiv.1707.07250
Research works printed but not linked: 5
- Mood Ring Monitors Your State of Mind, Chicago Tribune, 8 October 1975, at C1: Ring Buyers Warm Up to Quartz Jewelry That Is Said to Reflect Their Emotions. The Wall Street Journal, 14 October 1975, at p. 16; and “A Ring Around the Mood Market”, The Washington Post, 24 November 1975, at B9
- Dziczkowski, G., Węgrzyn-Wolska, K.: Tool of the intelligence economic: recognition function of reviews critics. In: ICSOFT 2008 Proceedings. INSTICC Press (2008)
- Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. J. CS224N Proj. Rep., Stanford 1, 12 (2009)
- Steidl, S.: Automatic classification of emotion-related user states in spontaneous children’s speech. Ph.D. thesis, Erlangen (2009)
- Ekman, P., Huang, T.S., Sejnowski, T.J., et al.: Final report to NSF of the planning workshop on facial expression understanding, vol. 378. Human Interaction Laboratory, University of California, San Francisco (1993)
Cited by
Sorted by citations
Bayoudh, K., Knani, R., Hamdaoui, F., & Mtibaa, A. (2022) · The Visual Computer · doi:10.1007/s00371-021-02166-7 · Cited by 456
Facial emotion recognition using deep learning: review and insights
Mellouk, W., & Handouzi, W. (2020) · Procedia Computer Science · doi:10.1016/j.procs.2020.07.101 · Cited by 370
Multimodal Emotion Recognition using Deep Learning
Abdullah, S. M. S. A., Ameen, S. Y. A., M. Sadeeq, M. A., & Zeebaree, S. (2021) · Journal of Applied Science and Technology Trends · doi:10.38094/jastt20291 · Cited by 347
Saganowski, S., Perz, B., Polak, A. G., & Kazienko, P. (2022) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2022.3176135 · Cited by 236
A Review of Key Technologies for Emotion Analysis Using Multimodal Information
Zhu, X., Guo, C., Feng, H., Huang, Y., Feng, Y., Wang, X., & Wang, R. (2024) · Cognitive Computation · doi:10.1007/s12559-024-10287-z · Cited by 143
Metadata
- Authors
- Marechal, C., Mikołajewski, D., Tyburek, K., Prokopowicz, P., Bougueroua, L., Ancourt, C., & Węgrzyn-Wolska, K.
- Affiliations
- Kazimierz Wielki University in Bydgoszcz (PL)
- Engineering School of Information and Digital Technologies (FR)
- Université Paris Sciences et Lettres (FR)
- Corresponding author
- Węgrzyn-Wolska, Katarzyna
- ORCID
- Venue
- Lecture Notes in Computer Science
- Publisher
- Springer International Publishing
- Published
- 2019
- Volume
- 11400
- Pages
- 307-324
- ISSN
- 0302-9743 (ISSN-L)
- 1611-3349
- OpenAlex
- W2922915988
- Semantic Scholar
- 059ce4ee466aa3d85bd351970b7e4716d7c9cfd9
- Cited by
- 137
- References
- 63
- Citations by year
- 2026: 6 · 2025: 11 · 2024: 25 · 2023: 17 · 2022: 24 · 2021: 34 · 2020: 19 · 2019: 1
- FWCI
- 24.56
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Lecture notes in computer science (publishedVersion, cc-by)
- HAL (submittedVersion, other-oa)
- Funders
- Uniwersytet Kazimierza Wielkiego w Bydgoszczy
- Topics
- Physical Sciences › Computer Science › Artificial Intelligence › Emotion and Mood Recognition
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Multimodal Machine Learning Applications
- Keywords
- multimodal emotion recognition, automatic emotion recognition, emotion analysis, text emotion analysis, speech emotion analysis, image-based emotion recognition, video-based emotion recognition, physiological signals, emotion prediction, emotion representation
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-08, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-08 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
