The Human in Emotion Recognition on Social Media: Attitudes, Outcomes, Risks
The DOI matches its Crossref record; 23 of the 23 references it cites were found in a registry or an index.
Abstract
Emotion recognition algorithms recognize, infer, and harvest emotions using data sources such as social media behavior, streaming service use, voice, facial expressions, and biometrics in ways often opaque to the people providing these data. People's attitudes towards emotion recognition and the harms and outcomes they associate with it are important yet unknown. Focusing on social media, we interviewed 13 adult U.S. social media users to fill this gap. We find that people view emotions as insights to behavior, prone to manipulation, intimate, vulnerable, and complex. Many find emotion recognition invasive and scary, associating it with autonomy and control loss. We identify two categories of emotion recognition's risks: individual and societal. We discuss findings' implications for algorithmic accountability and argue for considering emotion data as sensitive. Using a Science and Technology Studies lens, we advocate that technology users should be considered as a relevant social group in emotion recognition advancements.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ellison, N. B., Steinfield, C., & Lampe, C. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00367.x · Cited by 9,939 · cited 1×
“People use social media and express their emotions on them for reasons including identity expression, support exchange, and finding community [6,7,43].”
BackgroundPinch, T. J., & Bijker, W. E. (1984) · Social Studies of Science · doi:10.1177/030631284014003004 · Cited by 3,890
Experimental evidence of massive-scale emotional contagion through social networks
Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1320040111 · Cited by 3,687 · cited 1×
“Studies and social media corporate experiments interacting with people’s private information (e.g. the “emotion contagion” study [72]) have received public and scholarly attention, and sometimes backlash (e.g., [12,38,55,64,85,86,97,113,130,131]).”
BackgroundThe effect of emotion on cue utilization and the organization of behavior
Easterbrook, J. A. (1959) · Psychological Review · doi:10.1037/h0047707 · Cited by 3,090
Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications
Calvo, R. A., & D'Mello, S. (2010) · IEEE Transactions on Affective Computing · doi:10.1109/t-affc.2010.1 · Cited by 1,893 · cited 1×
“Emotion researchers across disciplines view emotions from several theoretical perspectives [22].”
BackgroundBarrett, L. F. (2006) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2006.00003.x · Cited by 1,859 · cited 1×
“Some have critiqued the notion of only six emotions [101] and their universality [13,53], and argued”
BackgroundEmotional expressions reconsidered: Challenges to inferring emotion from human facial movements
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019) · Psychological Science in the Public Interest · doi:10.1177/1529100619832930 · Cited by 1,748
Russell, J. A. (1994) · Psychological Bulletin · doi:10.1037/0033-2909.115.1.102 · Cited by 1,640 · cited 1×
“Some have critiqued the notion of only six emotions [101] and their universality [13,53], and argued”
BackgroundFairness and Abstraction in Sociotechnical Systems
Selbst, A. D., Boyd, D., Friedler, S. A., Venkatasubramanian, S., & Vertesi, J. (2019) · Proceedings of the Conference on Fairness, Accountability, and Transparency · doi:10.1145/3287560.3287598 · Cited by 1,631 · cited 3×
“It is before the stabilization period, and certainly before the closure period, that we should critically decide what problems are important and what social groups should be included in decisions about developing, making sense of, and adopting emerging technologies [105].”
BackgroundMethodData science and its relationship to big data and data-driven decision making
Provost, F., & Fawcett, T. (2013) · Big Data · doi:10.1089/big.2013.1508 · Cited by 1,568 · cited 1×
“Studies and social media corporate experiments interacting with people’s private information (e.g. the “emotion contagion” study [72]) have received public and scholarly attention, and sometimes backlash (e.g., [12,38,55,64,85,86,97,113,130,131]).”
BackgroundEmotion Elicits the Social Sharing of Emotion: Theory and Empirical Review
Rimé, B. (2009) · Emotion Review · doi:10.1177/1754073908097189 · Cited by 1,546 · cited 1×
“, finding support and community, improved wellbeing [10,57,93]), but involve privacy calculations and complex decision-making processes [7,9,16,100].”
Background“But the data is already public”: on the ethics of research in Facebook
Zimmer, M. (2010) · Ethics and Information Technology · doi:10.1007/s10676-010-9227-5 · Cited by 900
Instagram photos reveal predictive markers of depression
Reece, A. G., & Danforth, C. M. (2017) · EPJ Data Science · doi:10.1140/epjds/s13688-017-0110-z · Cited by 519
Emotion, disclosure, & health.
(1995) · doi:10.1037/10182-000 · Cited by 470
Deep learning for robust feature generation in audiovisual emotion recognition
Kim, Y., Lee, H., & Provost, E. M. (2013) · 2013 IEEE International Conference on Acoustics, Speech and Signal Processing · doi:10.1109/icassp.2013.6638346 · Cited by 401 · cited 1×
“facial expressions, biometrics, and body language in ways often unknown to users [29,69,84,92].”
BackgroundEmotional AI: The Rise of Empathic Media
McStay, A. (2018) · doi:10.4135/9781526451293 · Cited by 385
reconstructing the past: a century of ideas about emotion in psychology
Gendron, M., & Feldman Barrett, L. (2009) · Emotion Review · doi:10.1177/1754073909338877 · Cited by 361 · cited 1×
“Some have critiqued the notion of only six emotions [101] and their universality [13,53], and argued”
BackgroundSocial media, big data, and mental health: current advances and ethical implications
Conway, M., & O’Connor, D. (2016) · Current Opinion in Psychology · doi:10.1016/j.copsyc.2016.01.004 · Cited by 231 · cited 1×
“Scholars have examined social media users’ attitudes toward using Twitter data to monitor depression for research, finding that people were more comfortable with aggregate level monitoring than individual assessments, and concerned about consent, permanence, and privacy [32,33,87].”
BackgroundModeling and Understanding Visual Attributes of Mental Health Disclosures in Social Media
Manikonda, L., & De Choudhury, M. (2017) · Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3025453.3025932 · Cited by 116 · cited 4×
“More recently, academics and technologists have used machine learning and facial recognition to detect emotions and mental health status based on more obscure signs (e.g. visual markers of depression on Instagram [79,98].”
BackgroundInfluentialAn impulse to exploit: the behavioral turn in data-driven marketing
Nadler, A., & McGuigan, L. (2018) · Critical Studies in Media Communication · doi:10.1080/15295036.2017.1387279 · Cited by 92 · cited 2×
“Additionally, harvesting emotional data online is part of a “behavioral turn” in digital commerce [89].”
BackgroundEngaging the ethics of data science in practice
Barocas, S., & Boyd, D. (2017) · Communications of the ACM · doi:10.1145/3144172 · Cited by 86 · cited 1×
“Studies and social media corporate experiments interacting with people’s private information (e.g. the “emotion contagion” study [72]) have received public and scholarly attention, and sometimes backlash (e.g., [12,38,55,64,85,86,97,113,130,131]).”
BackgroundInferring Mood Instability on Social Media by Leveraging Ecological Momentary Assessments
Saha, K., Chan, L., De Barbaro, K., Abowd, G. D., & De Choudhury, M. (2017) · Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies · doi:10.1145/3130960 · Cited by 77
The Illusion of Control: Placebo Effects of Control Settings
Vaccaro, K., Huang, D., Eslami, M., Sandvig, C., Hamilton, K., & Karahalios, K. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173590 · Cited by 65
Cited by
How does HCI Understand Human Agency and Autonomy?
Bennett, D., Metatla, O., Roudaut, A., & Mekler, E. D. (2023) · Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3544548.3580651 · Cited by 111
On manipulation by emotional AI: UK adults’ views and governance implications
Bakir, V., Laffer, A., McStay, A., Miranda, D., & Urquhart, L. (2024) · Frontiers in Sociology · doi:10.3389/fsoc.2024.1339834 · Cited by 34
Akter, T., Ahmed, T., Kapadia, A., & Swaminathan, M. (2022) · ACM Transactions on Accessible Computing · doi:10.1145/3506857 · Cited by 29
State-of-the-art Multimodal Emotion Recognition: A comprehensive survey and taxonomy
Yazici, A., Kucukyilmaz, T., Dokeroglu, T., Sharipbay, A., Lee, M.-H., & Tyler, B. (2026) · Intelligent Systems with Applications · doi:10.1016/j.iswa.2026.200642 · Cited by 11
Metadata
- Authors
- Andalibi, N., & Buss, J.
- Venue
- Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
- Publisher
- ACM
- Published
- 2020-04-21
- Pages
- 1-16
- OpenAlex
- W3031265594
- Cited by
- 116
- References
- 23
- FWCI
- 15.62
- Type
- conference paper
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.