Affective News: The Automated Coding of Sentiment in Political Texts

Young, Lori; Soroka, Stuart

doi:10.1080/10584609.2012.671234

Cited by 610Sentiment Analysis and Opinion MiningComputational and Text Analysis MethodsMedia Studies and Communication

Affective News: The Automated Coding of Sentiment in Political Texts

Young, Lori, Soroka, Stuart·2012·Political Communication

DOI 10.1080/10584609.2012.671234

Open this paper in CoChat — free account

Abstract

An increasing number of studies in political communication focus on the “sentiment” or “tone” of news content, political speeches, or advertisements. This growing interest in measuring sentiment coincides with a dramatic increase in the volume of digitized information. Computer automation has a great deal of potential in this new media environment. The objective here is to outline and validate a new automated measurement instrument for sentiment analysis in political texts. Our instrument uses a dictionary-based approach consisting of a simple word count of the frequency of keywords in a text from a predefined dictionary. The design of the freely available Lexicoder Sentiment Dictionary (LSD) is discussed in detail here. The dictionary is tested against a body of human-coded news content, and the resulting codes are also compared to results from nine existing content-analytic dictionaries. Analyses suggest that the LSD produces results that are more systematically related to human coding than are results based on the other available dictionaries. The LSD is thus a useful starting point for a revived discussion about dictionary construction and validation in sentiment analysis for political communication.

Integrity signals

Registry

Crossref · 10.1080/10584609.2012.671234

Cites

92 of 93 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524 · see citing works
Prospect theory: An analysis of decision under risk
Kahneman, D. J., & Tversky, A. (1988) · Cambridge University Press eBooks · doi:10.1017/cbo9780511609220.014 · Cited by 32,700 · see citing works
Untitled work
(1994) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,542 · see citing works
DESCARTES' ERROR: Emotion, Reason, and the Human Brain
Marg, E. (1995) · Optometry and Vision Science · doi:10.1002/j.1538-9235.1995.tb03988.x · Cited by 8,177 · see citing works
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026683 · Cited by 8,134 · see citing works
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024 · see citing works
A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge.
Landauer, T. K., & Dumais, S. T. (1997) · Psychological Review · doi:10.1037/0033-295x.104.2.211 · Cited by 6,282 · see citing works
Untitled work
(1996) · Cited by 3,692 · see citing works
Recognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · doi:10.3115/1220575.1220619 · Cited by 3,388 · see citing works
Psychological Aspects of Natural Language Use: Our Words, Our Selves
Pennebaker, J. W., Mehl, M. R., & Niederhoffer, K. G. (2003) · Annual Review of Psychology · doi:10.1146/annurev.psych.54.101601.145041 · Cited by 2,624 · see citing works
More Than Words: Quantifying Language to Measure Firms' Fundamentals
Tetlock, P. C., Saar‐Tsechansky, M., & Macskassy, S. A. (2008) · The Journal of Finance · doi:10.1111/j.1540-6261.2008.01362.x · Cited by 2,263 · see citing works
Linguistic Inquiry and Word Count (LIWC)
Chung, C. K., & Pennebaker, J. W. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch012 · Cited by 2,152 · see citing works
Mining the peanut gallery
Dave, K. S., Lawrence, S., & Pennock, D. M. (2003) · International World Wide Web Conference (WWW) · doi:10.1145/775152.775226 · Cited by 1,925 · see citing works
On the primacy of affect.
Zajonc, R. (1984) · American Psychologist · doi:10.1037/0003-066x.39.2.117 · Cited by 1,903 · see citing works
Affective Intelligence and Political Judgment
Marcus, G. E., Neuman, W. R., & MacKuen, M. (2000) · Cited by 1,686 · see citing works
What Drives Media Slant? Evidence From U.S. Daily Newspapers
Gentzkow, M. A., & Shapiro, J. M. (2010) · Econometrica · doi:10.3982/ecta7195 · Cited by 1,559 · see citing works
Measuring praise and criticism
Turney, P. D., & Littman, M. L. (2003) · ACM Transactions on Information Systems · doi:10.1145/944012.944013 · Cited by 1,515 · see citing works
Predicting the semantic orientation of adjectives
Hatzivassiloglou, V., & McKeown, K. R. (1997) · Annual Meeting of the Association for Computational Linguistics (ACL) · doi:10.3115/976909.979640 · Cited by 1,445 · see citing works
WordNet Affect: an Affective Extension of WordNet
Strapparava, C., & Valitutti, A. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/2yctm9q26ar4 · Cited by 1,387 · see citing works
Extracting Policy Positions from Political Texts Using Words as Data
LAVER, M., BENOIT, K., & GARRY, J. (2003) · American Political Science Review · doi:10.1017/s0003055403000698 · Cited by 1,377 · see citing works
Negative information weighs more heavily on the brain: The negativity bias in evaluative categorizations.
Ito, T. A., Larsen, J. T., Smith, N. K., & Cacioppo, J. T. (1998) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.75.4.887 · Cited by 1,340 · see citing works
The general inquirer: A computer approach to content analysis.
Smith, M. S., Stone, P. J., & Dunphy, D. C. (1966) · Medical Entomology and Zoology · Cited by 1,165 · see citing works
The General Inquirer: A Computer Approach to Content Analysis
Green, B. F., Stone, P. J., Dunphy, D. C., Smith, M. S., & Ogilvie, D. M. (1967) · American Educational Research Journal · doi:10.2307/1161774 · Cited by 1,013 · see citing works
The Effect of Economic Events on Votes for President
Fair, R. C. (1978) · The Review of Economics and Statistics · doi:10.2307/1924969 · Cited by 971 · see citing works
Contrasting Rational and Psychological Analyses of Political Choice
Quattrone, G. A., & Tversky, A. (1988) · American Political Science Review · doi:10.2307/1962487 · Cited by 883 · see citing works
Alchemies of the Mind
Elster, J. (1998) · Cambridge University Press eBooks · doi:10.1017/cbo9781139173308 · Cited by 878 · see citing works
Good News and Bad News: Asymmetric Responses to Economic Information
Soroka, S. N. (2006) · The Journal of Politics · doi:10.1111/j.1468-2508.2006.00413.x · Cited by 850 · see citing works
Self-citation
SENTIMENT CLASSIFICATION of MOVIE REVIEWS USING CONTEXTUAL VALENCE SHIFTERS
Kennedy, A., & Inkpen, D. Z. (2006) · Computational Intelligence · doi:10.1111/j.1467-8640.2006.00277.x · Cited by 755 · see citing works
Using WordNet to Measure Semantic Orientations of Adjectives
Kamps, J., Marx, M., Mokken, R. J., & de Rijke, M. (2004) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/222z3zcs9izx · Cited by 654 · see citing works
Computer simulation
Watson, G. (1988) · Cited by 614 · see citing works
Three Steps toward a Theory of Motivated Political Reasoning
Lodge, M. G., & Taber, C. S. (2000) · Cambridge University Press eBooks · doi:10.1017/cbo9780511805813.009 · Cited by 611 · see citing works
What Drives Media Slant? Evidence from U.S. Daily Newspapers
Gentzkow, M. A., & Shapiro, J. M. (2006) · National Bureau of Economic Research · doi:10.3386/w12707 · Cited by 579 · see citing works
THE DICTIONARY OF AFFECT IN LANGUAGE
Whissell, C. (1989) · Elsevier eBooks · doi:10.1016/b978-0-12-558704-4.50011-6 · Cited by 559 · see citing works
Learning Subjective Adjectives from Corpora
Wiebe, J. (2000) · Cited by 520 · see citing works
Framing Responsibility for Political Issues
Iyengar, S. (1996) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716296546001006 · Cited by 516 · see citing works
The sentimental citizen: emotion in democratic politics
(2003) · Choice Reviews Online · doi:10.5860/choice.40-3068 · Cited by 505 · see citing works
Get out the vote
Thomas, M., Pang, B., & Lee, L. (2006) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1610075.1610122 · Cited by 480 · see citing works
Elements of Reason: Cognition, Choice, and the Bounds of Rationality
Lupia, A., McCubbins, M. D., & Popkin, S. L. (2008) · Cited by 469 · see citing works
Extracting opinions, opinion holders, and topics expressed in online news media text
Kim, S.-M., & Hovy, E. H. (2006) · doi:10.3115/1654641.1654642 · Cited by 460 · see citing works
Voter Response to Short-Run Economic Conditions: the Asymmetric Effect of Prosperity and Recession
Bloom, H. S., & Price, H. D. (1975) · American Political Science Review · doi:10.2307/1955284 · Cited by 458 · see citing works
The Effects of Negative Political Advertisements: A Meta-Analytic Assessment
Lau, R. R., Sigelman, L., Heldman, C., & Babbitt, P. (1999) · American Political Science Review · doi:10.2307/2586117 · Cited by 432 · see citing works
Telling Stories About Global Climate Change
McComas, K. A., & Shanahan, J. E. (1999) · Communication Research · doi:10.1177/009365099026001003 · Cited by 418 · see citing works
A bootstrapping method for learning semantic lexicons using extraction pattern contexts
Thelen, M., & Riloff, E. (2002) · doi:10.3115/1118693.1118721 · Cited by 363 · see citing works
Unsupervised Learning of Semantic Orientation from a Hundred-Billion-Word CorpusPreprint
Turney, P., & Littman, M. L. (2002) · arXiv (Cornell University) · doi:10.4224/8914027 · Cited by 359 · see citing works
The effects of emotional arousal and valence on television viewers’ cognitive capacity and memory
Lang, A., Dhillon, K., & Dong, Q. (1995) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838159509364309 · Cited by 329 · see citing works
The Clockwork Muse: The Predictability of Artistic Change.
Clignet, R. P., & Martindale, C. (1991) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2073749 · Cited by 329 · see citing works
Mining WordNet for a Fuzzy Sentiment: Sentiment Tag Extraction from WordNet Glosses
Andreevskaia, A., & Bergler, S. (2006) · Cited by 317 · see citing works
Affect analysis of text using fuzzy semantic typing
Subašić, P., & Huettner, A. K. (2001) · IEEE Transactions on Fuzzy Systems · doi:10.1109/91.940962 · Cited by 293 · see citing works
News narratives and news framing: constructing political reality
(2005) · Choice Reviews Online · doi:10.5860/choice.42-4478 · Cited by 257 · see citing works
The Affect Effect
Marcus, G. E., Neuman, W. R., MacKuen, M., & Crigler, A. N. (2007) · University of Chicago Press eBooks · doi:10.7208/chicago/9780226574431.001.0001 · Cited by 222 · see citing works
Political Narratives and Political Reality
Shenhav, S. R. (2006) · International Political Science Review · doi:10.1177/0192512106064474 · Cited by 222 · see citing works
The general inquirer: A computer system for content analysis and retrieval based on the sentence as a unit of information
Stone, P. J., Bales, R. F., Namenwirth, J. Z., & Ogilvie, D. M. (2007) · Systems Research and Behavioral Science · doi:10.1002/bs.3830070412 · Cited by 220 · see citing works
A Bio-Informational Theory of Emotion: Motion and Image Size Effects on Viewers
Detenber, B. H., & Reeves, B. B. (1996) · Journal of Communication · doi:10.1111/j.1460-2466.1996.tb01489.x · Cited by 212 · see citing works
TV news images that induce anger, fear, and disgust: Effects on approach‐avoidance and memory
Newhagen, J. E. (1998) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838159809364448 · Cited by 206 · see citing works
Emotion-abstraction patterns in verbatim protocols: A new way of describing psychotherapeutic processes.
Mergenthaler, E. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.64.6.1306 · Cited by 205 · see citing works
Computer simulation of personality
Hedl, J. J. (1968) · Journal of School Psychology · doi:10.1016/0022-4405(68)90021-6 · Cited by 199 · see citing works
Emotion–abstraction patterns in verbatim protocols: A new way of describing psychotherapeutic processes.
Mergenthaler, E. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.6.1306 · Cited by 199 · see citing works
Attentional homogeneity in industries: the effect of discretion
Abrahamson, E., & Hambrick, D. C. (1997) · Journal of Organizational Behavior · doi:10.1002/(sici)1099-1379(199711)18:1+ · Cited by 195 · see citing works
Elite Economic Forecasts, Economic News, Mass Economic Judgments, and Presidential Approval
Nadeau, R., Niemi, R. G., & Amato, T. (1999) · The Journal of Politics · doi:10.2307/2647777 · Cited by 191 · see citing works
Computer simulation of "hot cognition"
Abelson, R. P. (1963) · Medical Entomology and Zoology · Cited by 190 · see citing works
Media, Terrorism, and Emotionality: Emotional Differences in Media Content and Public Reactions to the September 11th Terrorist Attacks
Cho, J., Boyle, M. P., Keum, H., Shevy, M. D., McLeod, D. M., Shah, D. V., & Pan, Z. (2003) · Journal of Broadcasting & Electronic Media · doi:10.1207/s15506878jobem4703_1 · Cited by 186 · see citing works
The Clockwork Muse: The Predictability of Artistic Change
Porter, L. M., & Martindale, C. (1992) · Journal of Aesthetics and Art Criticism · doi:10.2307/430964 · Cited by 168 · see citing works
Redeveloping Diction: Theoretical Considerations
Hart, R. P. (2001) · doi:10.5040/9798216186847.ch-003 · Cited by 153 · see citing works
Verbal Style and the Presidency: A Computer-Based Analysis
Tetlock, P. E., & Hart, R. B. (1985) · Political Psychology · doi:10.2307/3791035 · Cited by 144 · see citing works
Romantic Progression: The Psychology of Literary History
Lindauer, M. S., & Martindale, C. (1976) · The American Journal of Psychology · doi:10.2307/1421428 · Cited by 143 · see citing works
Verbal style and the presidency: A computer-based analysis. By Roderick P. Hart Orlando: Academic Press, 1984. Pp. xiv, 322.
Javkin, H. R. (1985) · Language · doi:10.2307/414438 · Cited by 139 · see citing works
Sentiment Classification of Movie and Product Reviews Using Contextual Valence Shifters
Kennedy, A., & Inkpen, D. Z. (2005) · Cited by 123 · see citing works
‘Passions and constraint’
Hall, C. (2002) · Philosophy & Social Criticism · doi:10.1177/019145370202800607 · Cited by 92 · see citing works
Understanding how bloggers feel
Leshed, G., & Kaye, J. (2006) · CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA) · doi:10.1145/1125451.1125646 · Cited by 89 · see citing works
ASR for emotional speech: Clarifying the issues and enhancing performance
Athanaselis, T., Bakamidis, S., Dologlou, I., Cowie, R., Douglas‐Cowie, E., & Cox, C. (2005) · Neural Networks · doi:10.1016/j.neunet.2005.03.008 · Cited by 88 · see citing works
Resonating minds: A school-independent theoretical conception and its empirical application to psychotherapeutic processes
Mergenthaler, E. (2008) · Psychotherapy Research · doi:10.1080/10503300701883741 · Cited by 84 · see citing works
The Nightly News Nightmare: Media Coverage of U.S. Presidential Elections, 1988-2008
Farnsworth, S. J., & Lichter, S. R. (2010) · Cited by 82 · see citing works
Political Language and Political Reality
Edelman, M. J. (1985) · PS Political Science & Politics · doi:10.1017/s1049096500021247 · Cited by 74 · see citing works
Force and Influence in Content Analysis: The Production of New Social Knowledge
Hogenraad, R. L., McKenzie, D. P., & Péladeau, N. (2003) · Quality & Quantity · doi:10.1023/a:1024401325472 · Cited by 67 · see citing works
Political keywords : using language that uses us
Hart, R. P. (2005) · Oxford University Press eBooks · Cited by 64 · see citing works
Automated classification of congressional legislation
Purpura, S., & Hillard, D. (2006) · Conference on Digital Government Research · doi:10.1145/1146598.1146660 · Cited by 62 · see citing works
Measuring Affect and Action in Inter National Reaction Models
Holsti, O. R., Brody, R. A., & North, R. C. (1964) · Journal of Peace Research · doi:10.1177/002234336400100303 · Cited by 61 · see citing works
Validating the Coverage of Lexical Resources for Affect Analysis and Automatically Classifying New Words along Semantic Axes
Grefenstette, G., Qu, Y., Evans, D. A., et al. (2006) · Springer eBooks · doi:10.1007/1-4020-4102-0_9 · Cited by 61 · see citing works
The Tone of Local Presidential News Coverage
Eshbaugh‐Soha, M. (2010) · Political Communication · doi:10.1080/10584600903502623 · Cited by 60 · see citing works
Verbal Style and the Presidency: A Computer-Based Analysis.
Stone, P. J., & Hart, R. P. (1986) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2070923 · Cited by 59 · see citing works
Network Tv News Framing of Good Vs. Bad Economic News under Democrat and Republican Presidents: A Lexical Analysis of Political Bias
Lowry, D. T. (2008) · Journalism & Mass Communication Quarterly · doi:10.1177/107769900808500301 · Cited by 56 · see citing works
The cognitive and affective components of political attitudes: Measuring the determinants of candidate evaluations
Ottati, V., Steenbergen, M. R., & Riggle, E. D. B. (1992) · Political Behavior · doi:10.1007/bf00992043 · Cited by 54 · see citing works
What the Words of War Can Tell Us About the Risk of War.
Hogenraad, R. L. (2005) · Peace and Conflict Journal of Peace Psychology · doi:10.1207/s15327949pac1102_2 · Cited by 53 · see citing works
Linguistic Markers of Psychological State through Media Interviews: John Kerry and John Edwards in 2004, Al Gore in 2000
Pennebaker, J. W., Slatcher, R. B., & Chung, C. K. (2005) · Analyses of Social Issues and Public Policy · doi:10.1111/j.1530-2415.2005.00065.x · Cited by 52 · see citing works
Passion and politics
Walzer, M. (2002) · Philosophy & Social Criticism · doi:10.1177/019145370202800602 · Cited by 51 · see citing works
An automated method of topic-coding legislative speech over time with application to the 105th-108th U.S. senate
Quinn, K. M., Monroe, B. L., Colaresi, M. P., Crespin, M. H., & Radev, D. (2006) · Cited by 47 · see citing works
Dimensional Reduction of Word-Frequency Data as a Substitute for Intersubjective Content Analysis
Simon, A. F., & Xenos, M. A. (2004) · Political Analysis · doi:10.1093/pan/mph004 · Cited by 46 · see citing works
Campaign News and Vote Intentions
Soroka, S., Bodet, M. A., Young, L., et al. (2009) · Journal of Elections Public Opinion and Parties · doi:10.1080/17457280903275030 · Cited by 43 · see citing works
Self-citation
Incorporating topic information into sentiment analysis models
Mullen, T., & Collier, N. (2004) · doi:10.3115/1219044.1219069 · Cited by 26 · see citing works
Determining the Semantic Orientation of Web-Based Corpora
Arno Scharl, A., Pollach, I., & Bauer, C. A. (2003) · Lecture notes in computer science · doi:10.1007/978-3-540-45080-1_116 · Cited by 26 · see citing works
Experiments on semantic categorisation of texts: analysis of positive and negative dimension
Bolasco, S. (2004) · IRIS Research product catalog (Sapienza University of Rome) · Cited by 18 · see citing works
Interactions and Perceptions in the Test Ban Negotiations
Hopmann, P. T., & King, T. A. (1976) · International Studies Quarterly · doi:10.2307/3014016 · Cited by 16 · see citing works
Towards a validated model for affective classification of texts
Généreux, M., & Evans, R. (2006) · doi:10.3115/1654641.1654649 · Cited by 8 · see citing works

Flagged first, then by citations

1 of the 94 references couldn’t be matched to a known paper.

Research works printed but not linked: 11

Bradley M. M., 1999, Affective Norms for English Words (ANEW): Stimuli, instruction manual and affective ratings
Grefenstette G., 2004, Exploring attitude and affect in text: Theories and applications, 93
Hart R. P., 2000, DICTION 5.0: The text analysis program
Huitt W., 2003, The affective system: Educational psychology interactive
Krueger J. S., 2005, The place of prediction in politics
Mishne G., 2005, Experiments with mood classification in blog posts
Murphy C., 2006, The rhetorical semantics of state ballot initiative arguments in California, 1980–2004
Nussbaum M., 2004, Hiding from humanity: Shame, disgust and the law
Roget P. M., 1911, Roget's thesaurus of English words and phrases (supplemented electronic version)
Tong R., 2001, Detecting and tracking opinions in online discussions
Whissell C., 1989, Emotion: Theory and research, 113

Sorted by citations

Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · see citing works
“Scholars have long recognized this, but the massive costs of analyzing even moderately sized collections of texts have hindered their use in political science research.”
Background
quanteda: An R package for the quantitative analysis of textual data
Benoit, K. R., Watanabe, K., Wang, H., Nulty, P., Obeng, A., Müller, S., & Matsuo, A. (2018) · The Journal of Open Source Software · doi:10.21105/joss.00774 · Cited by 1,341 · see citing works
“More general textmodel functions include efficient implementations of a multinomial Naive Bayes classifier designed specifically for textual data (Manning, Raghavan, & Schütze, 2008) and latent semantic analysis (Deerwester, Dumais, Furnas, Landauer, & Harshman, 1990). quanteda also works flexibly and efficiently with…”
Good debt or bad debt: Detecting semantic orientations in economic texts
Malo, P., Sinha, A., Korhonen, P., Wallenius, J., & Takala, P. (2014) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23062 · Cited by 513 · see citing works
“Most of the recent work has focused on the analysis of opinionated blog-writings and informal texts in social media (Young & Soroka, 2012; Mitrovic, Paltoglou, & Tadic, 2011; Bollen, Gonc¸alves, Ruan, & Mao, 2011; Bal-ahur, Kozareva, & Montoyo, 2009; Thelwall et al., 2012).”
Background
Measuring news sentiment
Shapiro, A. H., Sudhof, M., & Wilson, D. J. (2020) · Journal of Econometrics · doi:10.1016/j.jeconom.2020.07.053 · Cited by 455 · see citing works
“As mentioned above, we use this corpus both to construct, for a given emotion, both a lexical dictionary and a predictive model that can be used to measure the degree to which a piece of text exhibits that emotion. We first automatically construct lexicons that relate emotions to words and phrases that are indicative…”
Political Polarization on the Digital Sphere: A Cross-platform, Over-time Analysis of Interactional, Positional, and Affective Polarization on Social Media
Yarchi, M., Baden, C., & Kligler-Vilenchik, N. (2021) · Political Communication · doi:10.1080/10584609.2020.1785067 · Cited by 437 · see citing works
“While the text-based sentiment analysis is unable to distinguish between specific relational attitudes, it is widely established as a suitable means for gauging evaluations of a focal actor in political discourse (Young & Soroka, 2012).”
The Validity of Sentiment Analysis: Comparing Manual Annotation, Crowd-Coding, Dictionary Approaches, and Machine Learning Algorithms
van Atteveldt, W., van der Velden, M. A. C. G., & Boukes, M. (2021) · Communication Methods and Measures · doi:10.1080/19312458.2020.1869198 · Cited by 420 · see citing works
Consumer Demand for Cynical and Negative News Frames
Trussler, M., & Soroka, S. (2014) · The International Journal of Press/Politics · doi:10.1177/1940161214524832 · Cited by 334 · see citing works
Self-citation
Pierre Bourdieu: theorizing the digital
Ignatow, G., & Robinson, L. (2017) · Information Communication & Society · doi:10.1080/1369118x.2017.1301519 · Cited by 331 · see citing works
“For example, dictionaries have been used by political scientists to measure the tone of newspaper articles (Eshbaugh-Soha, 2010) and political texts such as speeches and advertisements (Young & Soroka, 2011).”
Method
Text Mining in Big Data Analytics
Hassani, H., Beneki, C. C., Unger, S., Mazinani, M. T., & Yeganegi, M. R. (2020) · Big Data and Cognitive Computing · doi:10.3390/bdcc4010001 · Cited by 345 · see citing works
“Then, counting the positive and negative terms will get a score showing how much a text opposes or approves a given subject [28]:”
Background
The Gatekeeping Function: Distributions of Information in Media and the Real World
Soroka, S. (2012) · The Journal of Politics · doi:10.1017/s002238161100171x · Cited by 278 · see citing works
Self-citation
It's (Change in) the (Future) Economy, Stupid: Economic Indicators, the Media, and Public Opinion
Soroka, S., Stecula, D. A., & Wlezien, C. B. (2014) · American Journal of Political Science · doi:10.1111/ajps.12145 · Cited by 252 · see citing works
Self-citation
More than Bags of Words: Sentiment Analysis with Word Embeddings
Rudkowsky, E., Haselmayer, M., Wastian, M., Jenny, M., Emrich, Š., & Sedlmair, M. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2018.1455817 · Cited by 222 · see citing works
“(Haselmayer & Jenny, 2017; Young & Soroka, 2012).”
BackgroundMethodInfluential
Taking Stock of the Toolkit
Boumans, J. W., & Trilling, D. (2015) · Digital Journalism · doi:10.1080/21670811.2015.1096598 · Cited by 212 · see citing works
“The level of sophistication of sentiment analyses varies (Young and Soroka 2012).”
Background
Sentiment analysis of political communication: combining a dictionary approach with crowdcoding
Haselmayer, M., & Jenny, M. (2016) · Quality & Quantity: International Journal of Methodology · doi:10.1007/s11135-016-0412-4 · Cited by 193 · see citing works
“Social scientists have accordingly stressed the need for custom-made dictionaries (Loughran and McDonald 2011; Young and Soroka 2012; Grimmer and Stewart 2013; Gonza´lez-Bailo´n and Paltoglou 2015; Soroka et al. 2015a).”
BackgroundResultInfluential
Multilingual Sentiment Analysis: A New Approach to Measuring Conflict in Legislative Speeches
Proksch, S., Lowe, W., Wäckerle, J., & Soroka, S. (2018) · Legislative Studies Quarterly · doi:10.1111/lsq.12218 · Cited by 181 · see citing works
Self-citation
Framing Climate Change: Economics, Ideology, and Uncertainty in American News Media Content From 1988 to 2014
Stecula, D. A., & Merkley, E. (2019) · Frontiers in Communication · doi:10.3389/fcomm.2019.00006 · Cited by 185 · see citing works
“These techniques are increasingly used by scholars in the social sciences to study news content and political discourse more broadly (Young and Soroka, 2012; Grimmer and Stewart, 2013; Lacy et al., 2015).”
BackgroundMethod
Understanding cultural differences and extreme attitudes in the 2021 OECD Trust Survey
Smid, S. (2023) · OECD working papers on public governance · doi:10.1787/ef25d883-en · Cited by 166 · see citing works
Automated Text Classification of News Articles: A Practical Guide
Barberá, P., Boydstun, A., Linn, S., McMahon, R., & Nagler, J. (2020) · Political Analysis · doi:10.1017/pan.2020.8 · Cited by 156 · see citing works
Deciding Who's Legitimate: News Media Framing of Immigrants and Refugees
Lawlor, A., & Tolley, E. (2017) · International journal of communication · Cited by 147 · see citing works
Social science in the era of big data
González‐Bailón, S. (2013) · Policy & Internet · doi:10.1002/1944-2866.poi328 · Cited by 143 · see citing works
“…(Converse, 1987), can increase the significance of online public discourse as a regulatory mechanism for policymaking—or at least as a barometer to measure the public mood, and the topics that are most salient in their minds (González-Bail ó n, Banchs, & Kaltenbrunner, 2012; Young & Soroka, 2012).”
ECONOMETRICS MEETS SENTIMENT: AN OVERVIEW OF METHODOLOGY AND APPLICATIONS
Algaba, A., Ardia, D., Bluteau, K., Borms, S., & Boudt, K. (2020) · Journal of Economic Surveys · doi:10.1111/joes.12370 · Cited by 135 · see citing works
What’s the Tone? Easy Doesn’t Do It: Analyzing Performance and Agreement Between Off-the-Shelf Sentiment Analysis Tools
Boukes, M., van de Velde, B., Araujo, T., & Vliegenthart, R. (2019) · Communication Methods and Measures · doi:10.1080/19312458.2019.1671966 · Cited by 128 · see citing works
“Expanding the list of words that Kleinnijenhuis, Schultz, Oegema, and Van Atteveldt (2013) used to analyze business news about financial institutions, Damstra and Boukes (2018) used a tailor-made list of 65 words particularly focusing on the sentiment within economic news.”
BackgroundMethodInfluential
Bad News or Mad News? Sentiment Scoring of Negativity, Fear, and Anger in News Content
Soroka, S., Young, L., & Balmas, M. (2015) · The ANNALS of the American Academy of Political and Social Science · doi:10.1177/0002716215569217 · Cited by 128 · see citing works
Self-citation
In Validations We Trust? The Impact of Imperfect Human Annotations as a Gold Standard on the Quality of Validation of Automated Content Analysis
Song, H., Tolochko, P., Eberl, J.-M., Eisele, O., Greussing, E., Heidenreich, T., Lind, F., Galyga, S., & Boomgaarden, H. G. (2020) · Political Communication · doi:10.1080/10584609.2020.1723752 · Cited by 123 · see citing works
“For instance, the results from existing, o ﬀ -the-shelf dictionaries may be validated against the manual coding of highly trained researchers (e.g., Muddiman, McGregor, & Stroud, 2018; Rooduijn & Pauwels, 2011; Young & Soroka, 2012).”
Latent Semantic Scaling: A Semisupervised Text Analysis Technique for New Domains and Languages
Watanabe, K. (2020) · Communication Methods and Measures · doi:10.1080/19312458.2020.1832976 · Cited by 124 · see citing works
“I collected older news articles from the New York Times to construct a corpus that overlaps or contains the sample used in Young and Soroka (2012).”
Method
Scientific networks on Twitter: Analyzing scientists’ interactions in the climate change debate
Walter, S., Lörcher, I., & Brüggemann, M. (2019) · Public Understanding of Science · doi:10.1177/0963662519844131 · Cited by 99 · see citing works
What’s in a post? How sentiment and issue salience affect users’ emotional reactions on Facebook
Eberl, J., Tolochko, P., Jost, P., Heidenreich, T., & Boomgaarden, H. G. (2020) · Journal of Information Technology & Politics · doi:10.1080/19331681.2019.1710318 · Cited by 125 · see citing works
Empirical Studies of Media Bias
Puglisi, R., & Snyder, J. M. (2015) · Handbook of media economics · doi:10.1016/b978-0-444-63685-0.00015-2 · Cited by 118 · see citing works
Simple politics for the people? Complexity in campaign messages and political knowledge
Bischof, D., & Senninger, R. (2017) · European Journal of Political Research · doi:10.1111/1475-6765.12235 · Cited by 113 · see citing works
“Thereby, the existing research largely neglects the style and tone politicians use when communicating with the masses (for notable exceptions, see Spirling 2015;Young & Soroka 2012).”
Background
The big two dictionaries: Capturing agency and communion in natural language
Pietraszkiewicz, A., Formanowicz, M., Sendén, M. G., Boyd, R. L., Sikström, S., & Sczesny, S. (2018) · European Journal of Social Psychology · doi:10.1002/ejsp.2561 · Cited by 109 · see citing works
Signals of Public Opinion in Online Communication
González‐Bailón, S., & Paltoglou, G. (2015) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716215569192 · Cited by 105 · see citing works
“Finally, the Lexicoder Sentiment Dictionary (Young and Soroka 2012) was developed for the automatic coding of political text drawn from news articles covering a range of policy issues. two of the five dictionaries (ANeW and labMt) do not apply stemming— whether this has implications for their…”
MethodInfluential
Validating a sentiment dictionary for German political language—a workbench note
Rauh, C. (2018) · Journal of Information Technology & Politics · doi:10.1080/19331681.2018.1485608 · Cited by 100 · see citing works
“A typical net sentiment score (cf. Young & Soroka, 2012: 215) is thus given by Sentiment ¼ # positive terms # negative terms # all terms (1) This measure, also used in the remainder of this note, is then interpreted as a relative gap between positively and negatively connoted language.”
BackgroundMethod
Turning Words Into Consumer Preferences: How Sentiment Analysis Is Framed in Research and the News Media
Puschmann, C., & Powell, A. B. (2018) · Social Media + Society · doi:10.1177/2056305118797724 · Cited by 99 · see citing works
“…that sentiment analysis came to prominence exclusively within computational linguistics and psychology, but a rich parallel history within the social sciences, particularly within political science exists (see Ceron, 2015; Ceron & Negri, 2016; Young & Soroka, 2012, for contemporary examples).”
Background
Creating and Comparing Dictionary, Word Embedding, and Transformer-Based Models to Measure Discrete Emotions in German Political Text
Widmann, T., & Wich, M. (2022) · Political Analysis · doi:10.1017/pan.2022.15 · Cited by 101 · see citing works
Large Language Models versus Natural Language Understanding and Generation
Karanikolas, N., Manga, E., Samaridi, N., Tousidou, E., & Vassilakopoulos, M. (2023) · Proceedings of the 27th Pan-Hellenic Conference on Progress in Computing and Informatics · doi:10.1145/3635059.3635104 · Cited by 102 · see citing works
Text as Policy: Measuring Policy Similarity through Bill Text Reuse
Linder, F., Desmarais, B. A., Burgess, M., & Giraudy, E. (2018) · Policy Studies Journal · doi:10.1111/psj.12257 · Cited by 96 · see citing works
“…and public policy that draw upon text corpora to operationalize variables of interest (see, e.g., Hart, Smith-Howell and Llewellyn, 1990; Van Atteveldt, Kleinnijenhuis and Ruigrok, 2008; Young and Soroka, 2012; Mikhaylov, Laver and Benoit, 2012; Mondou, Skogstad and Houle, 2014; Witting, 2015).”
Background
Undermining, defusing or defending European integration? Assessing public communication of European executives in times of EU politicisation
Rauh, C., Bes, B. J., & Schoonvelde, M. (2019) · European Journal of Political Research · doi:10.1111/1475-6765.12350 · Cited by 94 · see citing works
“This tool has been extensively validated against human coders, provides a total of 4,450 word stems with term-level sentiment weights and contains various pre-processing tools that, among other things, cover negation (Young & Soroka 2012).”
Method
Are Experts (News)Worthy? Balance, Conflict, and Mass Media Coverage of Expert Consensus
Merkley, E. (2020) · Political Communication · doi:10.1080/10584609.2020.1713269 · Cited by 94 · see citing works
“Supervised machine learning techniques are increasingly used by scholars to study large volumes of text, including news content (Grimmer & Stewart, 2013; Lacey et al., 2015; Young & Soroka, 2012).”
Playing to the Gallery: Emotive Rhetoric in Parliaments
OSNABRÜGGE, M., HOBOLT, S. B., & RODON, T. (2021) · American Political Science Review · doi:10.1017/s0003055421000356 · Cited by 118 · see citing works
Women on the Run: Gender, Media, and Political Campaigns in a Polarized Era
Hayes, D., & Lawless, J. L. (2016) · Cited by 93 · see citing works
Political leaders and the media. Can we measure political leadership images in newspapers using computer-assisted content analysis?
Aaldering, L., & Vliegenthart, R. (2015) · Quality & Quantity · doi:10.1007/s11135-015-0242-9 · Cited by 93 · see citing works
“…content analysis, which relies on human coders, for instance when populism (Rooduijn and Pauwels 2011), negative economic news coverage (Hol-landers and Vliegenthart 2011) or the tone of news report on political parties and candidates in election campaigns (Young and Soroka 2012) are measured.”
Contextualizing the Crisis: The Framing of Syrian Refugees in Canadian Print Media
Wallace, R. (2018) · Canadian Journal of Political Science · doi:10.1017/s0008423917001482 · Cited by 90 · see citing works
Measuring News Sentiment
Francisco, F. R. B. O. S., Shapiro, A. H., Sudhof, M., Wilson, D. J., & Francisco, F. R. B. O. S. (2017) · Federal Reserve Bank of San Francisco, Working Paper Series · doi:10.24148/wp2017-01 · Cited by 90 · see citing works
Sentiment is Not Stance: Target-Aware Opinion Classification for Political Text Analysis
Bestvater, S. E., & Monroe, B. L. (2022) · Political Analysis · doi:10.1017/pan.2022.10 · Cited by 88 · see citing works
“The original application of the method, presented in Young and Soroka (2012) was concerned with the sentiment of news articles, for example.”
BackgroundMethod
News-based sentiment and bitcoin volatility
Sapkota, N. (2022) · International Review of Financial Analysis · doi:10.1016/j.irfa.2022.102183 · Cited by 89 · see citing works
Computational Identification of Media Frames: Strengths, Weaknesses, and Opportunities
Nicholls, T., & Culpepper, P. D. (2020) · Political Communication · doi:10.1080/10584609.2020.1812777 · Cited by 86 · see citing works
Politicians unleashed? Political communication on Twitter and in parliament in Western Europe
Silva, B. C., & Proksch, S. (2021) · Political Science Research and Methods · doi:10.1017/psrm.2021.36 · Cited by 86 · see citing works
“We rely on automatic translations of the Lexicoder sentiment dictionary (Soroka, 2012; Young and Soroka, 2012; Proksch et al. , 2019).”
Robotic involvement in the service encounter: a value-centric experience framework and empirical validation
Wu, L., Fan, A., Yang, Y., & He, Z.-Y. (2021) · Journal of service management · doi:10.1108/josm-12-2020-0448 · Cited by 74 · see citing works
Ideology, Salience, and Complexity: Determinants of Policy Issue Incongruence between Voters and Parties
Walgrave, S., & Lefevere, J. (2013) · Journal of Elections Public Opinion and Parties · doi:10.1080/17457289.2013.810630 · Cited by 81 · see citing works
A Progressive Supervised-learning Approach to Generating Rich Civil Strife Data
Nardulli, P. F., Althaus, S. L., & Hayes, M. (2015) · Sociological Methodology · doi:10.1177/0081175015581378 · Cited by 78 · see citing works
Negativity and Positivity Biases in Economic News Coverage: Traditional Versus Social Media
Soroka, S., Daku, M., Hiaeshutter-Rice, D., Guggenheim, L., & Josh Pasek, B. (2017) · Communication Research · doi:10.1177/0093650217725870 · Cited by 72 · see citing works
“The LSD has already been described in some detail elsewhere (Young & Soroka, 2012).”
BackgroundMethodSelf-citation
Does newspaper coverage influence or reflect public perceptions of the economy?
Hopkins, D. J., Kim, E., & Kim, S. (2017) · Research & Politics · doi:10.1177/2053168017737900 · Cited by 75 · see citing works
“Soroka et al. (2015) were able to analyze about 30,000 economic news stories in the US, and Kayser and Peress (2015) collected over 2 million articles in 16 different countries (see also Young and Soroka, 2012).”
Local and National Accounts of Immigration Framing in a Cross-national Perspective
Lawlor, A. (2015) · Journal of Ethnic and Migration Studies · doi:10.1080/1369183x.2014.1001625 · Cited by 70 · see citing works
Trump’s and Clinton’s Style and Rhetoric during the 2016 Presidential Election
Savoy, J. (2017) · Journal of Quantitative Linguistics · doi:10.1080/09296174.2017.1349358 · Cited by 68 · see citing works
“Based also on several word lists, Young and Soroka (2012) describe how one can detect and measure sentiments appearing in political texts.”
Background
Twitter and elections: are tweets, predictive, reactive, or a form of buzz?
Murthy, D. (2015) · Information Communication & Society · doi:10.1080/1369118x.2015.1006659 · Cited by 67 · see citing works
“We implemented our own text pre-processing based on some of the core recommendations of the LSD model.”
BackgroundMethodInfluential
What kind of populism? Tone and targets in the Twitter discourse of French and American presidential candidates
Maurer, P., & Diehl, T. (2020) · European Journal of Communication · doi:10.1177/0267323120909288 · Cited by 66 · see citing works
“Next, sentiment was scored using the Lexicoder Sentiment Dictionary (LSD), a set of word lists that combine several existing dictionaries (Young and Soroka, 2012).”
BackgroundMethod
Media sentiment and international asset prices
Fraiberger, S. P., Lee, D., Puy, D., & Rancière, R. G. (2021) · Journal of International Economics · doi:10.1016/j.jinteco.2021.103526 · Cited by 61 · see citing works
The discursive construction of digitalization: a comparative analysis of national discourses on the digital future of work
Marenco, M., & Seidl, T. (2021) · European Political Science Review · doi:10.1017/s175577392100014x · Cited by 65 · see citing works
“We use the Lexicoder Sentiment Dictionary, which was specifically designed for political text (Young and Soroka, 2012), taking into account negators (positive negatives and negative positives) as well.”
Method
Critical findings of the sixth assessment report (AR6) of working Group I of the intergovernmental panel on climate change (IPCC) for global climate change policymaking a summary for policymakers (SPM) analysis
Asadnabizadeh, M. (2022) · International Journal of Climate Change Strategies and Management · doi:10.1108/ijccsm-04-2022-0049 · Cited by 67 · see citing works
Political migration discourses on social media: a comparative perspective on visibility and sentiment across political Facebook accounts in Europe
Heidenreich, T., Eberl, J., Lind, F., & Boomgaarden, H. G. (2019) · Journal of Ethnic and Migration Studies · doi:10.1080/1369183x.2019.1665990 · Cited by 66 · see citing works
“Unlike Lexicoder, SentiStrength was not validated on political texts but on MySpace comments.”
BackgroundMethodInfluential
Seeded Sequential LDA: A Semi-Supervised Algorithm for Topic-Specific Analysis of Sentences
Watanabe, K., & Baturo, A. (2023) · Social Science Computer Review · doi:10.1177/08944393231178605 · Cited by 66 · see citing works
“We classi ﬁ ed all the sentences in the corpus into the six topics using a Seeded Sequential LDA model ( K ¼ 8; γ ¼ 0 : 5; μ ¼ 0 : 3) and performed sentiment analysis on them, employing the Lexicoder Sentiment Dictionary, which is widely used in social sciences (Young & Soroka, 2012).”
Method
Text Analysis for Psychology: Methods, Principles, and PracticesPreprint
Kennedy, B., Ashokkumar, A., Boyd, R. L., & Dehghani, M. (2021) · doi:10.31234/osf.io/h2b8t · Cited by 57 · see citing works
Reducing Ageism: Changes in Students’ Attitudes after Participation in an Intergenerational Reverse Mentoring Program
Leedahl, S. N., Brasher, M. S., LoBuono, D. L., Wood, B. M., & Estus, E. L. (2020) · Sustainability · doi:10.3390/su12176870 · Cited by 62 · see citing works
Analysis of the style and the rhetoric of the 2016 US presidential primaries
Savoy, J. (2017) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqx007 · Cited by 60 · see citing works
“Also grounded on several dictionaries (or categories), Young and Soroka (2012) describe how one can detect and measure sentiments appearing in political texts.”
Background
Polarization of climate politics results from partisan sorting: Evidence from Finnish Twittersphere
Chen, T. H. Y., Salloum, A., Gronow, A., Ylä‐Anttila, T., & Kivelä, M. (2021) · Global Environmental Change · doi:10.1016/j.gloenvcha.2021.102348 · Cited by 65 · see citing works
Étude comparative des logiciels d’aide à l’analyse de données qualitatives : de l’approche automatique à l’approche manuelle
Roy, N., & Garon, R. (2013) · Recherches qualitatives · doi:10.7202/1084616ar · Cited by 58 · see citing works
quanteda: Quantitative Analysis of Textual Data
Benoit, K. R., Watanabe, K., Wang, H., et al. (2015) · doi:10.32614/cran.package.quanteda · Cited by 58 · see citing works
Party Competition and Emotive Rhetoric
Kosmidis, S., HOBOLT, S. B., Molloy, E., & Whitefield, S. (2018) · Comparative Political Studies · doi:10.1177/0010414018797942 · Cited by 57 · see citing works
“Note also that Young and Soroka (2012) examined the relationship between a variety of sentiment dictionaries.”
Background
Framing Immigration in the Canadian and British News Media
Lawlor, A. (2015) · Canadian Journal of Political Science · doi:10.1017/s0008423915000499 · Cited by 55 · see citing works
Theoretical Foundations for Digital Text Analysis
Ignatow, G. (2015) · Journal for the Theory of Social Behaviour · doi:10.1111/jtsb.12086 · Cited by 52 · see citing works
“Recent social science applications of dictionary methods include work by Young and Soroka (2011) and Eshbaugh-Soha (2010).”
Method
Going in circles? The influence of the electoral cycle on the party behaviour in parliament
Schwalbach, J. (2021) · European Political Science Review · doi:10.1017/s1755773921000291 · Cited by 51 · see citing works
“Especially with the increase of technical possibilities for automated text analysis, the analysis of sentiment as a measure of conflictuality has become an important tool in the social sciences (Young and Soroka, 2012; Liu, 2015).”
Method
Latino Portrayals in Local News Media: Underrepresentation, Negative Stereotypes, and Institutional Predictors of Coverage
Sui, M., & Paul, N. (2017) · Journal of Intercultural Communication Research · doi:10.1080/17475759.2017.1322124 · Cited by 51 · see citing works
“In particular, without an understanding of the context in which these terms are presented, the dictionary-based automated coding often fails to interpret semantic patterns (Young & Soroka, 2012).”
BackgroundMethodInfluential
Social Media as an Alternative to Surveys of Opinions About the Economy
Conrad, F. G., Gagnon-Bartsch, J. A., Ferg, R. A., Schober, M. F., Josh Pasek, B., & Hou, E. (2019) · Social Science Computer Review · doi:10.1177/0894439319875692 · Cited by 51 · see citing works
“Lexicoder (Young & Soroka, 2012) was developed for measuring tone in news content.”
Background
Extremists on the Left and Right Use Angry, Negative Language
Frimer, J. A., Brandt, M. J., Melton, Z. J., & Motyl, M. S. (2018) · Personality and Social Psychology Bulletin · doi:10.1177/0146167218809705 · Cited by 51 · see citing works
Emotion and Political Psychology
Gadarian, S. K., & Brader, T. A. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197541302.013.5 · Cited by 53 · see citing works
Media Coverage of Campaign Promises Throughout the Electoral Cycle
Müller, S. (2020) · Political Communication · doi:10.1080/10584609.2020.1744779 · Cited by 49 · see citing works
“I apply the Lexicoder Sentiment Dictionary (Young & Soroka 2012), a frequently used sentiment dictionary for political and economic news, to all sentences in the corpus.8 If the media did not exert a negativity bias, we would expect sentences on broken or fulfilled promises not to differ…”
Method
The Role of Minority Journalists, Candidates, and Audiences in Shaping Race-Related Campaign News Coverage
Sui, M., Paul, N., Shah, P. R., Spurlock, B., Chastant, B., & Dunaway, J. (2018) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699018762078 · Cited by 49 · see citing works
“…but not least, despite many advantages regarding the dictionary-based automatic coding technique, that is, high efficiency and reliability (e.g., Lewis, Zamith, & Hermida, 2013; Young & Soroka, 2012), one critical limitation is the precision in identifying “relevant” content (Lacy et al., 2015).”
BackgroundResult
Tracking the Coverage of Public Policy in Mass Media
Soroka, S., & Wlezien, C. B. (2018) · Policy Studies Journal · doi:10.1111/psj.12285 · Cited by 48 · see citing works
Self-citation
How to measure public demand for policies when there is no appropriate survey data?
Oehl, B., Schaffer, L. M., & Bernauer, T. (2016) · Journal of Public Policy · doi:10.1017/s0143814x16000155 · Cited by 48 · see citing works
“With rapidly improving technologies for machine-assisted data coding and “big data” approaches to retrieve information from the internet, the problem of reliability and the costs of implementing media content coding are likely to decrease substantially (see e.g. Young and Soroka 2012).”
Method
Social media research: The application of supervised machine learning in organizational communication research.
van Zoonen, W., & van der Meer, T. G. L. A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2016.05.028 · Cited by 46 · see citing works
“For example, in sentiment analysis of political texts, words such as ‘respect,’ ‘vindication,’ and ‘cheerfulness’ were identified as indicative for positive sentiment, whereas ‘insolence,’ ‘malevolence,’ and ‘painfulness’ indicated negative sentiment (Young & Soroka, 2012).”
Background
Evaluating the News: (Mis)Perceptions of Objectivity and Credibility
Kelly, D. (2018) · Political Behavior · doi:10.1007/s11109-018-9458-4 · Cited by 47 · see citing works
Social Media and the Changing Information Environment
Kraft, P. W., Krupnikov, Y., Milita, K., Ryan, J. B., & Soroka, S. (2020) · Public Opinion Quarterly · doi:10.1093/poq/nfaa015 · Cited by 46 · see citing works
Self-citation
Inside the Black Box of Dictionary Building for Text Analytics: A Design Science Approach
Deng, Q., Hine, M. J., Ji, S., et al. (2019) · Journal of international technology and information management · doi:10.58729/1941-6679.1376 · Cited by 44 · see citing works
“WordStat has been used extensively in dictionary building related research (see Bengston & Xu, 1995; Boritz et al., 2013; de-Miguel-Molina et al., 2016; Laver & Garry, 2000; Loughran & McDonald, 2011; Opoku et al., 2006; Wilson, 2006; Young & Soroka, 2012).”
BackgroundMethodInfluential
Clause Analysis: Using Syntactic Information to Automatically Extract Source, Subject, and Predicate from Texts with an Application to the 2008–2009 Gaza War
van Atteveldt, W., Sheafer, T., Shenhav, S. R., & Fogel-Dror, Y. (2017) · Political Analysis · doi:10.1017/pan.2016.12 · Cited by 42 · see citing works
“For example, dictionary based approaches have been used to measure political affect (e.g. Young and Soroka 2012) and framing (Kellstedt 2003; Ruigrok and Van Atteveldt 2007).”
Method
The Effects of Partisan Media in the Face of Global Pandemic: How News Shaped COVID-19 Vaccine Hesitancy
Motta, M., & Stecula, D. A. (2023) · Political Communication · doi:10.1080/10584609.2023.2187496 · Cited by 57 · see citing works
“In general, dictionary methods have been shown to accurately capture basic sentiment of text, and these dictionaries have been successfully validated by human coders (Young & Soroka, 2012).”
Method
Testing the Validity of Automatic Speech Recognition for Political Text Analysis
Proksch, S., Wratil, C., & Wäckerle, J. (2019) · Political Analysis · doi:10.1017/pan.2018.62 · Cited by 41 · see citing works
“…use of automatic transcriptions for the Wordfish scaling model (Slapin and Proksch 2008) as well as dictionary-based approaches to sentiment (e.g., Young and Soroka 2012; Daku, Soroka, and Young 2015; Proksch et al. 2018) as two prominent “bag-of-words” models using a corpus of “State of the…”
Method
A Dynamic Model of Speech for the Social Sciences
Knox, D., & Lucas, C. (2021) · American Political Science Review · doi:10.1017/s000305542000101x · Cited by 41 · see citing works
Whose News? Class-Biased Economic Reporting in the United States
Jacobs, A. M., Matthews, J. S., Hicks, T. J., & Merkley, E. (2021) · American Political Science Review · doi:10.1017/s0003055421000137 · Cited by 41 · see citing works
Logic is (Somewhat) Overrated: Image-Based Versus Concept-Based Rhetoric in Crowdfunding Narratives
Patel, P. C., Wolfe, M. T., & Manikas, A. S. (2020) · Entrepreneurship Theory and Practice · doi:10.1177/1042258720976370 · Cited by 41 · see citing works
“By employing rich theoretical perspectives regarding the persuasive efficacy of image- based versus conceptbased rhetoric (Cariola, 2014; Young & Soroka, 2012), we further extend our understanding of the association between narrative content and crowdfunding performance.”
Background
The Automatic Analysis of Emotion in Political Speech Based on Transcripts
Cochrane, C., Rheault, L., Godbout, J., Whyte, T., Wong, M. W.-C., & Borwein, S. (2021) · Political Communication · doi:10.1080/10584609.2021.1952497 · Cited by 40 · see citing works
“We test five widely used sentiment dictionaries in our analysis: Lexicoder 3.0 (Daku et al., 2015; Young & Soroka, 2012), Sentiwordnet 3.0 (Baccianella et al., 2010; Esuli & Sebastiani, 2006), the Hu-Liu Lexicon (Hu & Liu, 2004), Jockers-Rinker’s Lexicon (Jockers, 2015; Rinker et al., 2019), and…”
BackgroundMethodInfluential
Media portrayals of Muslims: a comparative sentiment analysis of American newspapers, 1996–2015
Bleich, E., & van der Veen, A. M. (2018) · Politics Groups and Identities · doi:10.1080/21565503.2018.1531770 · Cited by 40 · see citing works
“Recent years have seen rapidly growing interest among social scientists in the automated analysis of texts, with sentiment analysis prominent among the tools political scientists have used (see e.g., Grimmer and Stewart 2013; Hopkins and King 2010; Thomas, Pang, and Lee 2006; Young and Soroka 2012).”
Method
Automated content analysis across six languages
Windsor, L., Cupit, J. G., & Windsor, A. J. (2019) · PLoS ONE · doi:10.1371/journal.pone.0224425 · Cited by 39 · see citing works
“753) with the Lexicoder Sentiment Dictionary used in social science research, compared with eight other automated sentiment analysis programs [32].”
Method
Economic News Through the Magnifying Glass
van Dalen, A., de Vreese, C. H., & Albæk, E. (2015) · Journalism Studies · doi:10.1080/1461670x.2015.1089183 · Cited by 38 · see citing works
“As a validity check we compared the results of the automated tone coding with hand-coded economic news articles, following Young and Soroka (2012).”
MethodResultInfluential
What social media activities reveal about election results? The use of Facebook during the 2015 general election campaign in Croatia
Babac, M. B., & Podobnik, V. (2018) · Information Technology and People · doi:10.1108/itp-08-2016-0200 · Cited by 38 · see citing works
“The examples of content attributes that would acquire better understanding of human behavioural patterns in politics are age (Vitak et al., 2011), gender (Bagić Babac and Podobnik, 2016), or different sentiments (Young and Soroka, 2012; Grčić et al., 2017) regarding political participation.”
Background
The economic effects of U.S. presidential tax communication: Evidence from a correlated topic model
Dybowski, T. P., & Adämmer, P. (2018) · European Journal of Political Economy · doi:10.1016/j.ejpoleco.2018.05.001 · Cited by 37 · see citing works
The Temporal Focus of Campaign Communication
Müller, S. (2021) · The Journal of Politics · doi:10.1086/715165 · Cited by 37 · see citing works
“Second, results stay the same when using different dictionaries (Proksch et al. 2019; Rauh 2018; Young and Soroka 2012), an alternative aggregation of sentiment, or a continuous measure of prospective rhetoric.”
BackgroundResult
Political Communication
Young, D. G., & Miller, J. M. (2023) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780197541302.013.15 · Cited by 39 · see citing works
Tone Matters: Effects of Exposure to Positive and Negative Tone of Television News Stories on Anti-Immigrant Attitudes and Carry-Over Effects to Uninvolved Immigrant Groups
Jacobs, L., & van der Linden, M. (2016) · International Journal of Public Opinion Research · doi:10.1093/ijpor/edw036 · Cited by 36 · see citing works
Words Matter: How WTO Rulings Handle Controversy
Busch, M., & Pelc, K. J. (2019) · International Studies Quarterly · doi:10.1093/isq/sqz025 · Cited by 36 · see citing works
A Model of Attentiveness to Outlying News
Lamberson, P. J., & Soroka, S. (2018) · Journal of Communication · doi:10.1093/joc/jqy040 · Cited by 35 · see citing works
Self-citation

ИСПОЛЬЗОВAНИЕ ПОТЕНЦИAЛA СОЦИAЛЬНЫХ ПAРТНЕРОВ В ПОДГОТОВКЕ БУДУЩИХ ПЕДAГОГОВ
Атабекова, Б. Б. (2024) · Habaršy. Pedagogika ġylymdary seriâsy · doi:10.51889/2959-5762.2023.80.4.019
Susquehanna Chorale Spring Concert "Roots and Wings"
Hawkins, D. (2017)
Optimized sentiment analysis tool: A sentiment analysis tool to study cognitive inclinations
Sharma, N., Indu, S., Sharma, C., et al. (2016) · International Conference on Computing for Sustainable Global Development · Cited by 1 · see citing works
A Comparative Study of Different Classification Techniques for Sentiment Analysis
Ghosh, S., Hazra, A., & Raj, A. (2020) · International Journal of Synthetic Emotions · doi:10.4018/ijse.20200101.oa · Cited by 12 · see citing works
Sentiment analysis: a challenge
Gupta, E., Kumar, A. P., & Kumar, M. (2018) · International Journal of Engineering & Technology · doi:10.14419/ijet.v7i2.27.16288 · Cited by 2 · see citing works

Authors: Young, L., & Soroka, S.
Affiliations: University of Pennsylvania (US)
McGill University (CA)
Corresponding author: Soroka, Stuart
ORCID: Soroka, Stuart
Venue: Political Communication
Publisher: Informa UK Limited
Published: 2012-04
Volume: 29
Issue: 2
Pages: 205-231
ISSN: 1058-4609 (ISSN-L)
1091-7675
DOI: 10.1080/10584609.2012.671234
OpenAlex: W2072514260
Semantic Scholar: fa26c8c77f34e2f500af22f8dbb643fb7654b585
Cited by: 609
References: 94
Citations by year: 2026: 35 · 2025: 46 · 2024: 55 · 2023: 60 · 2022: 53 · 2021: 82 · 2020: 68 · 2019: 56 · 2018: 42 · 2017: 32 · 2016: 19 · 2015: 20 · 2014: 12 · 2013: 26 · 2012: 2
FWCI: 19.21
Type: journal article
Language: en
Open access: closed
Locations: Political Communication (publishedVersion)
Topics: Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
Social Sciences › General Social Sciences › Computational and Text Analysis Methods
Social Sciences › Communication › Media Studies and Communication
Keywords: sentiment analysis, lexicon-based sentiment analysis, political news, computational text analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Affective News: The Automated Coding of Sentiment in Political Texts

Abstract

Integrity signals

Claims the paper makes

Cites

Cited by

Related

Record

About these checks