Affective News: The Automated Coding of Sentiment in Political Texts

“Scholars have long recognized this, but the massive costs of analyzing even moderately sized collections of texts have hindered their use in political science research.”

“More general textmodel functions include efficient implementations of a multinomial Naive Bayes classifier designed specifically for textual data (Manning, Raghavan, & Schütze, 2008) and latent semantic analysis (Deerwester, Dumais, Furnas, Landauer, & Harshman, 1990). quanteda also works flexibly and efficiently with…”

“Most of the recent work has focused on the analysis of opinionated blog-writings and informal texts in social media (Young & Soroka, 2012; Mitrovic, Paltoglou, & Tadic, 2011; Bollen, Gonc¸alves, Ruan, & Mao, 2011; Bal-ahur, Kozareva, & Montoyo, 2009; Thelwall et al., 2012).”

“As mentioned above, we use this corpus both to construct, for a given emotion, both a lexical dictionary and a predictive model that can be used to measure the degree to which a piece of text exhibits that emotion. We first automatically construct lexicons that relate emotions to words and phrases that are indicative…”

“While the text-based sentiment analysis is unable to distinguish between specific relational attitudes, it is widely established as a suitable means for gauging evaluations of a focal actor in political discourse (Young & Soroka, 2012).”

“For example, dictionaries have been used by political scientists to measure the tone of newspaper articles (Eshbaugh-Soha, 2010) and political texts such as speeches and advertisements (Young & Soroka, 2011).”

“Then, counting the positive and negative terms will get a score showing how much a text opposes or approves a given subject [28]:”

“(Haselmayer & Jenny, 2017; Young & Soroka, 2012).”

“The level of sophistication of sentiment analyses varies (Young and Soroka 2012).”

“Social scientists have accordingly stressed the need for custom-made dictionaries (Loughran and McDonald 2011; Young and Soroka 2012; Grimmer and Stewart 2013; Gonza´lez-Bailo´n and Paltoglou 2015; Soroka et al. 2015a).”

“These techniques are increasingly used by scholars in the social sciences to study news content and political discourse more broadly (Young and Soroka, 2012; Grimmer and Stewart, 2013; Lacy et al., 2015).”

“…(Converse, 1987), can increase the significance of online public discourse as a regulatory mechanism for policymaking—or at least as a barometer to measure the public mood, and the topics that are most salient in their minds (González-Bail ó n, Banchs, & Kaltenbrunner, 2012; Young & Soroka, 2012).”

“Expanding the list of words that Kleinnijenhuis, Schultz, Oegema, and Van Atteveldt (2013) used to analyze business news about financial institutions, Damstra and Boukes (2018) used a tailor-made list of 65 words particularly focusing on the sentiment within economic news.”

“For instance, the results from existing, o ff -the-shelf dictionaries may be validated against the manual coding of highly trained researchers (e.g., Muddiman, McGregor, & Stroud, 2018; Rooduijn & Pauwels, 2011; Young & Soroka, 2012).”

“I collected older news articles from the New York Times to construct a corpus that overlaps or contains the sample used in Young and Soroka (2012).”

“Thereby, the existing research largely neglects the style and tone politicians use when communicating with the masses (for notable exceptions, see Spirling 2015;Young & Soroka 2012).”

“Finally, the Lexicoder Sentiment Dictionary (Young and Soroka 2012) was developed for the automatic coding of political text drawn from news articles covering a range of policy issues. two of the five dictionaries (ANeW and labMt) do not apply stemming— whether this has implications for their…”

“A typical net sentiment score (cf. Young & Soroka, 2012: 215) is thus given by Sentiment ¼ # positive terms # negative terms # all terms (1) This measure, also used in the remainder of this note, is then interpreted as a relative gap between positively and negatively connoted language.”

“…that sentiment analysis came to prominence exclusively within computational linguistics and psychology, but a rich parallel history within the social sciences, particularly within political science exists (see Ceron, 2015; Ceron & Negri, 2016; Young & Soroka, 2012, for contemporary examples).”

“…and public policy that draw upon text corpora to operationalize variables of interest (see, e.g., Hart, Smith-Howell and Llewellyn, 1990; Van Atteveldt, Kleinnijenhuis and Ruigrok, 2008; Young and Soroka, 2012; Mikhaylov, Laver and Benoit, 2012; Mondou, Skogstad and Houle, 2014; Witting, 2015).”

“This tool has been extensively validated against human coders, provides a total of 4,450 word stems with term-level sentiment weights and contains various pre-processing tools that, among other things, cover negation (Young & Soroka 2012).”

“Supervised machine learning techniques are increasingly used by scholars to study large volumes of text, including news content (Grimmer & Stewart, 2013; Lacey et al., 2015; Young & Soroka, 2012).”

“…content analysis, which relies on human coders, for instance when populism (Rooduijn and Pauwels 2011), negative economic news coverage (Hol-landers and Vliegenthart 2011) or the tone of news report on political parties and candidates in election campaigns (Young and Soroka 2012) are measured.”

“The original application of the method, presented in Young and Soroka (2012) was concerned with the sentiment of news articles, for example.”

“We rely on automatic translations of the Lexicoder sentiment dictionary (Soroka, 2012; Young and Soroka, 2012; Proksch et al. , 2019).”

“The LSD has already been described in some detail elsewhere (Young & Soroka, 2012).”

“Soroka et al. (2015) were able to analyze about 30,000 economic news stories in the US, and Kayser and Peress (2015) collected over 2 million articles in 16 different countries (see also Young and Soroka, 2012).”

“Based also on several word lists, Young and Soroka (2012) describe how one can detect and measure sentiments appearing in political texts.”

“We implemented our own text pre-processing based on some of the core recommendations of the LSD model.”

“Next, sentiment was scored using the Lexicoder Sentiment Dictionary (LSD), a set of word lists that combine several existing dictionaries (Young and Soroka, 2012).”

“We use the Lexicoder Sentiment Dictionary, which was specifically designed for political text (Young and Soroka, 2012), taking into account negators (positive negatives and negative positives) as well.”

“Unlike Lexicoder, SentiStrength was not validated on political texts but on MySpace comments.”

“We classi fi ed all the sentences in the corpus into the six topics using a Seeded Sequential LDA model ( K ¼ 8; γ ¼ 0 : 5; μ ¼ 0 : 3) and performed sentiment analysis on them, employing the Lexicoder Sentiment Dictionary, which is widely used in social sciences (Young & Soroka, 2012).”

“Also grounded on several dictionaries (or categories), Young and Soroka (2012) describe how one can detect and measure sentiments appearing in political texts.”

“Note also that Young and Soroka (2012) examined the relationship between a variety of sentiment dictionaries.”

“Recent social science applications of dictionary methods include work by Young and Soroka (2011) and Eshbaugh-Soha (2010).”

“Especially with the increase of technical possibilities for automated text analysis, the analysis of sentiment as a measure of conflictuality has become an important tool in the social sciences (Young and Soroka, 2012; Liu, 2015).”

“In particular, without an understanding of the context in which these terms are presented, the dictionary-based automated coding often fails to interpret semantic patterns (Young & Soroka, 2012).”

“Lexicoder (Young & Soroka, 2012) was developed for measuring tone in news content.”

“I apply the Lexicoder Sentiment Dictionary (Young & Soroka 2012), a frequently used sentiment dictionary for political and economic news, to all sentences in the corpus.8 If the media did not exert a negativity bias, we would expect sentences on broken or fulfilled promises not to differ…”

“…but not least, despite many advantages regarding the dictionary-based automatic coding technique, that is, high efficiency and reliability (e.g., Lewis, Zamith, & Hermida, 2013; Young & Soroka, 2012), one critical limitation is the precision in identifying “relevant” content (Lacy et al., 2015).”

“With rapidly improving technologies for machine-assisted data coding and “big data” approaches to retrieve information from the internet, the problem of reliability and the costs of implementing media content coding are likely to decrease substantially (see e.g. Young and Soroka 2012).”

“For example, in sentiment analysis of political texts, words such as ‘respect,’ ‘vindication,’ and ‘cheerfulness’ were identified as indicative for positive sentiment, whereas ‘insolence,’ ‘malevolence,’ and ‘painfulness’ indicated negative sentiment (Young & Soroka, 2012).”

“WordStat has been used extensively in dictionary building related research (see Bengston & Xu, 1995; Boritz et al., 2013; de-Miguel-Molina et al., 2016; Laver & Garry, 2000; Loughran & McDonald, 2011; Opoku et al., 2006; Wilson, 2006; Young & Soroka, 2012).”

“For example, dictionary based approaches have been used to measure political affect (e.g. Young and Soroka 2012) and framing (Kellstedt 2003; Ruigrok and Van Atteveldt 2007).”

“In general, dictionary methods have been shown to accurately capture basic sentiment of text, and these dictionaries have been successfully validated by human coders (Young & Soroka, 2012).”

“…use of automatic transcriptions for the Wordfish scaling model (Slapin and Proksch 2008) as well as dictionary-based approaches to sentiment (e.g., Young and Soroka 2012; Daku, Soroka, and Young 2015; Proksch et al. 2018) as two prominent “bag-of-words” models using a corpus of “State of the…”

“By employing rich theoretical perspectives regarding the persuasive efficacy of image- based versus conceptbased rhetoric (Cariola, 2014; Young & Soroka, 2012), we further extend our understanding of the association between narrative content and crowdfunding performance.”

“We test five widely used sentiment dictionaries in our analysis: Lexicoder 3.0 (Daku et al., 2015; Young & Soroka, 2012), Sentiwordnet 3.0 (Baccianella et al., 2010; Esuli & Sebastiani, 2006), the Hu-Liu Lexicon (Hu & Liu, 2004), Jockers-Rinker’s Lexicon (Jockers, 2015; Rinker et al., 2019), and…”

“Recent years have seen rapidly growing interest among social scientists in the automated analysis of texts, with sentiment analysis prominent among the tools political scientists have used (see e.g., Grimmer and Stewart 2013; Hopkins and King 2010; Thomas, Pang, and Lee 2006; Young and Soroka 2012).”

“753) with the Lexicoder Sentiment Dictionary used in social science research, compared with eight other automated sentiment analysis programs [32].”

“As a validity check we compared the results of the automated tone coding with hand-coded economic news articles, following Young and Soroka (2012).”

“The examples of content attributes that would acquire better understanding of human behavioural patterns in politics are age (Vitak et al., 2011), gender (Bagić Babac and Podobnik, 2016), or different sentiments (Young and Soroka, 2012; Grčić et al., 2017) regarding political participation.”

“Second, results stay the same when using different dictionaries (Proksch et al. 2019; Rauh 2018; Young and Soroka 2012), an alternative aggregation of sentiment, or a continuous measure of prospective rhetoric.”