The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; none of the 21 sources its claims cite is checked for support yet.
Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict
Open accessAbstract
Entries in the burgeoning “text-as-data” movement are often accompanied by lists or visualizations of how word (or other lexical feature) usage differs across some pair or set of documents. These are intended either to establish some target semantic concept (like the content of partisan frames) to estimate word-specific measures that feed forward into another analysis (like locating parties in ideological space) or both. We discuss a variety of techniques for selecting words that capture partisan, or other, differences in political speech and for evaluating the relative importance of those words. We introduce and emphasize several new approaches based on Bayesian shrinkage and regularization. We illustrate the relative utility of these approaches with analyses of partisan, gender, and distributive speech in the U.S. Senate.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Monroe critiques common lexical feature-selection and evaluation methods for text-as-data and proposes model-based Bayesian shrinkage/regularization approaches, illustrated on U.S. Senate speech. The recommended methods reduce overemphasis on very frequent or very rare words and yield more semantically valid partisan word lists.
Key findings
- Nonmodel-based measures fail to account for sampling variation: difference of proportions is dominated by high-frequency words (top Democratic list includes “to,” “my,” “and”; top Republican list includes “the,” “of,” “not”), while log-odds-ratios are dominated by obscure low-frequency words (e.g., “bankruptci,” “Snowe,” “infant,” “Chines”). The paper notes the odds of a Republican using “babi” are 5.4 times a Democrat’s, but the odds of using “April” are 7.4 times a Democrat’s.
- Model-based z-scores of log-odds-ratios improve semantic validity: abortion speech features Republican framing around baby/child and partial-birth procedure, and Democratic framing around woman/women and her/their right to choose.
- An informative Dirichlet prior (prior sample of 500 words per party per day) replaces function words in the Republican top 20 with “aliv,” “infant,” “brutal,” “brain,” and “necessari.” A Laplace prior shrinks most contrasts to exactly zero, leaving prominent Republican words “kill,” “babi,” “procedur,” “child,” “mother,” “abort” and Democratic words “women,” “woman,” “decis,” “famili,” “friend,” “right.”
- Applications show taxes were more partisan than abortion; media odds of using Democratic over Republican tax frames were roughly 20% higher for The New York Times than The Washington Times and roughly 63% higher for NPR than Fox News. Defense polarization fell near zero after 9/11 and rose through and after the invasion of Iraq.
Methods
- Design: methodological critique and proposal, with a running example of U.S. Senate speeches on abortion in the 106th Congress (1999–2000), using topics from Quinn et al. (2006) for Senate speech 1997–2004.
- Data: Congressional Speech Corpus from the public-domain Congressional Record, truncated at December 31, 2004, parsed into tagged XML, filtered for capitalization/punctuation, and stemmed with the Porter Snowball II stemmer.
- Analysis: compares classification, nonmodel-based measures (difference of frequencies/proportions, stop-word removal, odds/log-odds, tf.idf, WordScores), then multinomial/Dirichlet model-based log-odds-ratio estimates, z-scores, informative Dirichlet shrinkage, and Laplace-prior L1 regularization; extends to taxes, gender within Democrats, dynamics, polarization, and state-level dimensionality.
Limitations
- Stated: the Laplace prior’s nondifferentiability increases computational complexity even for point estimates and makes dynamic extension difficult, so it is less ideal for feature evaluation and feed-forward applications; the Dirichlet approach still leaves unclear where to draw the line for feature selection and may overfit if estimates feed forward.
- Stated: the media framing analysis is a “toy example,” and the applications are intended to illustrate possibilities rather than provide full substantive analyses.
- Evident: empirical demonstrations are limited to U.S. Senate speech from 1997–2004 and selected topics; no validation outside this corpus is reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 21 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, in our running example of speech on abortion, we find that the most common words (the, of, and, is, to, a) are considered Republican.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.1 Difference of frequencies
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the case of two ''documents,'' as is the case here, this is nearly identical to the difference of proportions measure.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results for our running example are nearly identical, qualitatively and quantitatively with those from raw difference of proportions, shown in Figure 1.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that regardless of the zero treatment, the most extreme words are obscure ones.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is obvious in Figure 1, the sampling variation in difference of proportions is greatest in high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 3, the extreme W kw are received by the words spoken by only one party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is shown in Figure 5, this has a desirable effect on the function-word problem noted above.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The best example here is Senate, which is deemed Democratic by difference of proportions, but is, in this context, simply a high-frequency word with high sampling variability.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But the lack of overall semantic validity is clear in the overemphasis on high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most partisan words are the words spoken the most by one party, while spoken not once by the other (bankruptci, infant).15”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- It may state two separate claims as one. 35
- Probably not a substantial claim. 42
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This has the same problems, with the dominant words still the same list of obscurities.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most Democratic words are then bankruptci, Snow½e, ratifi, confidenti, and church, and the most Republican words are infant, admit, Chines, industri, and 40.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although function words no longer dominate the lists as with several previous techniques, some still appear to be too prominent in their implied partisanship, among them the, of, be, not, and my.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In this example, they correlate at over 10.998.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The top Democratic word list is dominated by to and includes my, and, and for; the top Republican word list is dominated by the and includes of, not, be, that, you, it, and a.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Figure 4 shows the feature weightings based on the fðD 2 RÞ kw for our running example.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we instead throw out the words with zero counts in one party, the most Democratic words are treati, discrim, abroad, domest, and privacy, and the most Republican words are perfect, subsid, percent, overrid, and cell.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“One example is given by Slapin and Proksch (2008), made more striking because they provide a plot that clearly demonstrates the heteroskedasticity.Their Figure 2 looks like Figure 2 turned on its side.The resulting lists of their Table 1 contain many obscurities.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
What the sources say
- Not checked yetcites Slapin and Proksch (2008)
Strength:boosted· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The weights are correlated (at 10.997 in this case)”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our data, the odds of a Republican using a variant of babi are 5.4 times those of a Democratic when talking about abortion, which seems informative.But, the odds of a Republican using the word April are 7.4 times those of a Democrat when talking about abortion, which seems less so.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.4
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sticking to the familiar metric of odds ratios, the odds of The New York Times using these Democratic over Republican frames are roughly 20% higher than those of the Washington Times.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Chong
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, we find several instances where Republicans ''own'' an issue or frame while in opposition to Clinton are successful in obtaining a policy change during the Bush honeymoon period of early 2001 or in the rally period after 9/11, only to find Democrats ''owning'' the issue when the policy becomes unpopular.”
- Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain
“Dramatic examples include debt when discussing the budget, oil when discussing energy policy, and Iraq when discussing defense policy.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
- Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“During the Bush era, Democrats consistently filibustered his judicial nominees, and Republicans discovered a deeply held belief in presidential deference and up-or-down votes.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Iraq supplies both of these, and in turn, we see relative nonpartisanship pre-9/11 shift to a Republican talking point post-9/11 only to see events on the ground and the cost of the war increase Democratic leverage on the issue.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is also clear variation in the intensity of preferences, with great intensity on multiple issues from the Alaska delegation, but limited intensity from the Alabama delegation.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, the Republican top 20 list has shuffled, with the, you, not, of, and be being replaced by the considerably more evocative aliv, infant, brutal, brain, and necessari.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The prior used in this example has almost no effect on estimates in topics with much more speech and strong partisan differences (like defense, post-Iraq), and overwhelms estimates-as it should-in topics with very little speech or no partisan differences (like symbolic constituent tributes).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Defense is particularly interesting, reaching near zero polarization in the aftermath of 9/11 and dramatically increasing through and after the invasion of Iraq.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For our particular substantive and theoretical interests, pooling all speech together has several undesirable consequences and topic modeling is a crucial prior step.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our proposed solution relies on a model-based approach to avoid inefficiency and shrinkage and regularization to avoid both infinite estimates and overfitting the sample data.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“This was the approach used by Gentzkow and Shapiro (2006).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
What the sources say
- Not checked yetcites Gentzkow and Shapiro (2006)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to the conjugacy, the full Bayesian estimate using the Dirichlet prior is also analytically available in analogous form:”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.3 Estimation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A related problem is that, although there is variation, every word in the vocabulary receives nonzero weight, which could lead to overfitting if these estimates are used in some subsequent model.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“tf.idf is recommended in standard textbooks (Jurafsky and Martin (2000, 651-4) (Manning and Schu ¨tze 1999, 541-4) and is widely used in document search and information retrieval tasks.14”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
What the sources say
- Not checked yetcites Jurafsky and Martin (2000, 651-4)
- Not checked yetcites Manning and Schu ¨tze 1999, 541-4
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eliminating function words not only eliminates words like her inappropriately but it also elevates high-frequency non-stop words inappropriately.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Perhaps the most prominent text-as-data approach in political science is the WordScores procedure (Laver et al. 2003), which embeds a feature evaluation technique in its algorithm.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
What the sources say
- Not checked yetcites Laver et al. 2003
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With the prior, this is a relabeling of the smoothed log-odds-ratio discussed before.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These do not seem like words that most fundamentally define the partisan division on abortion.Words with plausible partisan content on abortion (infant, church) are overwhelmed by oddities that require quite a bit more investigation to interpret (Chines, bankruptci).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“This mistake is also common in many of the emergent text-as-data discussions in political science.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
What the sources say
- Not checked yetcites Slapin and Proksch (2008)
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Working naively, it is unclear what we are to do with words that are spoken by only one party and therefore have infinite odds ratios.If we let the infinite values have infinite weight, the partisan word list consists of only those spoken by a single party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Without the prior, this is of course simply the observed log-odds-ratio.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Republican word choice reflects framing the debate from the point of view of the baby/child and emphasizing the details of the partial birth abortion procedure.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we wish to proceed directly to ML estimation, the lack of covariates results in an immediately available analytical solution for the MLE of b ðiÞ kw .”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.1 The likelihood
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The logic of tf.idf is that the words containing the greatest information about a particular document are the words that appear many times in that document, but in relatively few others.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is, use of any particular Dirichlet prior defined by a affects the posterior exactly as if we had observed in the data an additional a w -1 instances of word w.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.2 Prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problem is again the failure to account for sampling variability.With log-oddsratios, the sampling variation goes down with increased frequency, as is clear in Figure 2. So, this measure will be inappropriately dominated by obscure words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, WordScores demonstrates the same failure to account for sampling variation and the same overweighting of high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Generally, this simply removes the most problematic features without resolving the issue.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is no implied directionality (e.g., Democratic vs. Republican), however, and the v 2 value itself does not have any obvious interpretation as an evaluation weight.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our example, a stop list would eliminate a word like her, which almost definitely has political content in the context of abortion, 9 and a word like their, which might (e.g., women and their doctors).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There are several approaches, but the notion in common is to put a strong conservative prior on the model.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5 Shrinkage and Regularization
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problems we have seen to this point are typical of many contemporary data-rich problems, emerging not just from computational linguistics but from other similarly scaled or ''ill-posed'' machine and statistical learning problems in areas like image processing and gene expression.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5 Shrinkage and Regularization
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variance is based on the absolute frequency of a word in all, or both, documents of interest, and its implied absolute frequency in the associated priors.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, the fact that these estimates could be recovered from appropriate standard Bayesian generalized linear models means the approach is easily extended beyond the setting of a strict partitioning of the document space into discrete groups.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Rollcall-based work suggests that a single ideological dimension accounts for virtually all Congressional divisions (Poole 2005).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
What the sources say
- Not checked yetcites Poole 2005
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As with the above, the effect of any particular smoothing parameter will be affected by the relative volume of speech and the polarization of word use.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“This is important because of a general gender gap across the two parties both in core values (Rokeach 1973) and in framing (Lakoff 2004).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
What the sources say
- Not checked yetcites Rokeach 1973
- Not checked yetcites Lakoff 2004
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variation in language use offers an alternative measure, one that can be differentiated across political topics and at a finer dynamic grain.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is well known that this corresponds to a uniform prior for log(c).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the theoretical issue of majority rule, where you stand depends on where you sit.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can also extend our analysis across time to analyze dynamics across partisan (or extrapartisan) word use.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Republicans were pushing for elimination of the death tax, lowering of capital gains taxes, elimination of the marriage penalty, and flatter brackets; Democrats advocated for a tax credit for prescription drug coverage, for college, and against lowering the estate tax or giving breaks to the wealthiest.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Chong
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Not only can we look at the relative differences in language use between men and women, as we did with party, but we can also do so while controlling for party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A higher smoothing parameter would result in even fewer nonzero features; a smaller one would result in more.19”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The nondifferentiability at the mode does considerably increase the difficulty and computational overhead associated with estimation.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“One of the more prominent is the concern over whether women legislators behave differently than male legislators and whether they offer a fundamentally different form of representation by speaking ''in a different voice'' (Gilligan 1993;Williams 1998;Diaz 2005).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
What the sources say
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, this low computational overhead makes it relatively straightforward to develop more flexible variants, such as a dynamic version we discuss below.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The primary measure of polarization at the elite level has come from rollcall analysis, following Poole and Rosenthal (1997).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
What the sources say
- Not checked yetcites Poole and Rosenthal (1997)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach still suffers as a technique for feature selection: it is unclear where one draws the line between words that qualitatively matter and words that qualitatively do not.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to observe shifts in the tectonic plates of partisan conflict, we need both salience (the parties and presumably the public must care) and some exogenous shock or change in the status quo (to motivate change).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The analytic solution of the Dirichlet model makes dynamic extensions straightforward.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“A large-scale example clearly matching the criteria for a ''realignment'' or an ''issue evolution'' is the reversal of the parties on race and civil rights issues over the course of the 20th century; a small-scale example, which may or may not meet the criteria, is the polarization over time of the parties on the issue of abortion.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
What the sources say
- Not checked yetcites Carmines and Stimson 1990
- Not checked yetcites Poole and Rosenthal 1997
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Maximization of the posterior densities in equations 28 and 29 is complicated by the discontinuities in the partial derivatives that are introduced by the Laplace prior.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Appendix-Estimation of the Laplace Model
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These problems lead researchers to select and weight feature/word lists that have low semantic validity or vary wildly depending on arbitrary decisions.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 5 Discussion and Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
- Rests on an assumption: More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.
None of the sources cited for this claim have been checked yet.
In the paper
“Using results from Cawley and Talbot (2006), it is easy to show how the value of b ðiÞ k that maximizes pðb ðiÞ k jy ðiÞ k Þ } Z p À y ðiÞ k jb ðiÞ k Þpðb ðiÞ k jcÞp J ðcÞdcðA1Þ can be calculated via a modified version of the iterative approach of Shevade and Keerthi (2003) in which, at each iteration, a working version, c, of c is defined to be:”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Marginalizing over c with a Jeffreys Prior
What the sources say
- Not checked yetcites Cawley and Talbot (2006)
- Not checked yetcites Shevade and Keerthi (2003)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that when a 5 0, the results from this procedure are equivalent to those from the setup where c is given a Jeffreys prior and then integrated out of the posterior.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Marginalizing over c with an Exponential Prior
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is suggestive that there are measurable distributive (and therefore multidimensional) preference differences across Senators (which are unlikely to be present in rollcall votes once partisan coalitions have been built around particular bills).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
The paper relies on this without support.
Different speech patterns across groups can flow from both partisan and nonpartisan sources, including idiosyncrasies of dialect and style.
“Additionally, we know a priori that different speech patterns across groups can flow from both partisan and nonpartisan sources, including idiosyncrasies of dialect and style.”
The authors state this as an assumption.
The approximations in Equations 17 and 19 assume y(i)_kw >> a(i)_kw, y_kw >> a_kw and ignore covariance terms that will typically be close to 0.
“Where the approximations in Equations 17 and 19 assume y ðiÞ kw ) a ðiÞ kw ; y kw ) a kw and ignore covariance terms that will typically be close to 0”
The authors state this as an assumption.
Equations 18 and 20 additionally assume that n(i)_k >> y(i)_kw and n_k >> y_kw.
“while Equations 18 and 20 additionally assume that n ðiÞ k ) y ðiÞ kw and n k ) y kw .”
The paper relies on this without support.
Sports speeches congratulating a home state team have no partisan structure.
“For example, our Senate analysis finds a large number of ''sports'' speeches congratulating a home state team.We would not expect these to have partisan structure.”
The paper relies on this without support.
Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.
“Democrats are both more likely than Republicans to address ''women's issues,'' and also have more women members serving in the Senate than do the Republicans.”
The paper relies on this without support.
The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.
“We conducted Lexis-Nexis searches of two newspapers, The New York Times and The Washington Times, and two broadcast news sources, National Public Radio and Fox News, widely thought to be, relatively speaking, at different partisan extremes.”
The paper relies on this without support.
A genuine partisan realignment or issue evolution generally occurs only over fairly long time frames.
“This generally occurred (or been agreed to have occurred) only over fairly long time frames.”
The paper relies on this without support.
Speech data offers leverage on analyzing partisan conflict.
“Speech offers interesting potential leverage on the problem.”
The paper relies on this without support.
Prior efforts to systematically analyze political conflict have focused on easily quantified behaviors that obscure substance.
“Efforts to systematically analyze political conflict, especially in the American context, have previously focused on behaviors that are easily quantified but obscure substance (e.g., votes).”
The paper relies on this without support.
Language use in political speech is a valid indicator of political conflict and preferences.
“We have seen that static snapshots can illuminate the content of a particular partisan conflict.”
The paper relies on this without support.
More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.
“More recent attempts to investigating political dynamics that summarize and organize text have fallen into other traps, including inefficiency (e.g., classification techniques), and overfitting idiosyncratic group differences in low-or high-frequency words (e.g., tf.idf or difference of proportions).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Monroe, Colaresi, and Quinn (2008) describe a variety of methods that measure how well words separate already identified categories of interest”
From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Monroe, Colaresi, and Quinn (2008)
The cited source says what the claim attributes to it.
In the citing paper
“Although there have been efforts to understand differential word rate of use within topics pertaining to these terms (e.g., Monroe, Colaresi, and Quinn 2008), there has been relatively little work on whether the same words appear in different contexts.”
From Embedding Regression: Models for Context-Specific Description and Inference (2023) · cites this paper as Monroe, Colaresi, and Quinn 2008
In the citing paper
“For example, Monroe et al. explored a variety of techniques that compare two frequencies of words – one number for each of two groups [44]. In particular, they explored frequencies across democratic versus republican speeches and settled on a Bayesian model with regularization and shrinkage based on priors of word use.”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [44]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Nature of Statistical Learning Theory
Vapnik, V. (2000) · Statistics for Engineering and Information Science · doi:10.1007/978-1-4757-3264-1 · Cited by 46,387
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,346
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
(1973) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,144
Flagged first, then by citations
Research works printed but not linked: 9
- Diaz, 2005, Representing women? Female legislators in West European parliaments
- Carmines, 1990, Issue evolution
- The counts of estate, prescription, wage, wealthiest, death, bracket, flat, penalty, respectively, are (2078, 620, 1159, 208, 1280, 164, 431, 743) for The New York Times, (1032, 376, 692, 84, 731, 189, 399, 720) for The Washington Times, (172, 214, 224, 68, 209, 47, 68, 174) for National Public Radio, and (167, 362, 238, 54, 450, 55, 197, 281) for Fox News.
- Monroe, 2004, Society for Political Methodology
- Mucciaroni, 2006, Deliberative choice: Debating public policy in congress
- Riker, 1986, The art of political manipulation
- Emily, 2004, From boys to men
- Lakoff, 2004, Don't think of an elephant! Know your values and frame the debate
- Monroe, 2006, United States congressional speech corpus
Cited by
Sorted by citations
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297
“…than a count (Pang, Lee, and Vaithyanathan 2002; Hopkins and King 2010); (2) including some commonly used stopwords such as gendered pronouns (Monroe, Colaresi, and Quinn 2008); (3) a subset of features (either by automated feature selection or a lower dimensional projection) (Hofmann 1999);…”
BackgroundInfluentialPersonality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700
GoEmotions: A Dataset of Fine-Grained Emotions
Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.372 · Cited by 873
Computational Grounded Theory: A Methodological Framework
Nelson, L. K. (2017) · Sociological Methods & Research · doi:10.1177/0049124117729703 · Cited by 794
“More complicated lexical-based techniques include word scores that aim to identify important or distinctive words (Monroe et al. 2008), and relational semantic network techniques, such as mapping networks of words that occur near one another (Lee and Martin 2015).”
BackgroundHow to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671
Self-citationScaling Policy Preferences from Coded Political Texts
Lowe, W., Benoit, K. R., Mikhaylov, S., & Laver, M. (2011) · Legislative Studies Quarterly · doi:10.1111/j.1939-9162.2010.00006.x · Cited by 654
Automated Text Analysis for Consumer Research
Humphreys, A., & Wang, R. J.-H. (2018) · Journal of Consumer Research · doi:10.1093/jcr/ucx104 · Cited by 636
Mapping the Ideological Marketplace
Bonica, A. (2014) · American Journal of Political Science · doi:10.1111/ajps.12062 · Cited by 680
Computer-Assisted Text Analysis for Comparative Politics
Lucas, C., Nielsen, R. A., Roberts, M. E., Stewart, B. M., Storer, A., & Tingley, D. (2015) · Political Analysis · doi:10.1093/pan/mpu019 · Cited by 366
Denny, M. J., & Spirling, A. (2017) · Political Analysis · doi:10.1017/pan.2017.44 · Cited by 486
“This may be because the output of those models does not easily lend itself to answering substantive questions (see, e.g. Monroe, Colaresi and Quinn, 2008) or perhaps because the assumptions underlying those techniques are both consequential, and non-trivial to fully understand (see, e.g. Lowe,…”
BackgroundPark, J., Wood, I. B., Jing, E., Nematzadeh, A., Ghosh, S., Conover, M., & Ahn, Y. (2019) · Nature Communications · doi:10.1038/s41467-019-11380-w · Cited by 393
“To identify over-represented industries or geographical regions in a cluster, we employ the log-odds ratio informative Dirichlet prior method [Monroe et al., 2008].”
MethodIdeology and Interests in the Political Marketplace
Bonica, A. (2011) · American Journal of Political Science · doi:10.1111/ajps.12014 · Cited by 385
Problems in Current Text Simplification Research: New Data Can Help
Xu, W. H., Callison-Burch, C., & Napoles, C. (2015) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00139 · Cited by 369
“We choose the Monroe et al. (2008) method because many function words and punctuations are very frequent and play important roles in text simplification.”
MethodInfluentialStyle transfer through back-translation
Prabhumoye, S., Tsvetkov, Y., Salakhutdinov, R., & Black, A. W. (2018) · Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p18-1080 · Cited by 348
“Style lexicons are extracted using the log-odds ratio informative Dirichlet prior (Monroe et al., 2008), a method that identifies words that are statistically overrepresented in each of the categories.”
MethodHu, Y., Boyd-Graber, J. L., Satinoff, B., & Smith, A. (2013) · Machine Learning · doi:10.1007/s10994-013-5413-0 · Cited by 347
“This suggests that taking advantage of sparsity could also be used in other probabilistic models such as context-free grammars (Johnson et al, 2007; Johnson, 2010) and feature-based models (Monroe et al, 2008; Wu et al, 2010).”
MethodMachine Learning for Social Science: An Agnostic Approach
Grimmer, J., Roberts, M. E., & Stewart, B. (2021) · Annual Review of Political Science · doi:10.1146/annurev-polisci-053119-015921 · Cited by 362
Beyond Binary Labels: Political Ideology Prediction of Twitter Users
Preoțiuc-Pietro, D., Liu, Y., Hopkins, D. J., & Ungar, L. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1068 · Cited by 310
“First, researchers aimed to identify and quantify orientation of words (Monroe et al., 2008), hashtags (Weber et al., 2013) or documents (Iyyer et al., 2014), 1Data is available at http://www.preotiuc.ro or to detect bias (Yano et al., 2010) or impartiality (Zafar et al., 2016) at a document level.”
BackgroundInstitutional Foundations of Legislative Speech
Proksch, S., & Slapin, J. B. (2012) · American Journal of Political Science · doi:10.1111/j.1540-5907.2011.00565.x · Cited by 283
Social Biases in NLP Models as Barriers for Persons with Disabilities
Hutchinson, B., Prabhakaran, V., Denton, E., Webster, K., Zhong, Y., & Denuyl, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.487 · Cited by 274
Towards Emotional Support Dialog Systems
Liu, S., Zheng, C., DeMasi, O., Sabour, S., Yu, L., Yu, Z., Jiang, Y., & Huang, M. (2021) · Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2021.acl-long.269 · Cited by 286
“Each feature is ranked by the rounded z-scored log odds ratios (Monroe et al., 2008) in the parentheses.”
MethodBoumans, J. W., & Trilling, D. (2015) · Digital Journalism · doi:10.1080/21670811.2015.1096598 · Cited by 248
“Above all, however, these lexicons have to be tested and validated (see, e.g., Burscher et al. 2014; Monroe, Colaresi, and Quinn 2008).”
BackgroundSentiment analysis of political communication: combining a dictionary approach with crowdcoding
Haselmayer, M., & Jenny, M. (2016) · Quality & Quantity · doi:10.1007/s11135-016-0412-4 · Cited by 231
“…analysis of textual data has manifold applications in the social sciences, among them the study of polarization, public opinion or media tone (e.g., Monroe et al. 2008; Van Atteveldt et al. 2008; Hopkins and King 2010; Soroka 2012; Young and Soroka 2012; Burscher et al. 2015; Gonza´lez-Bailo´n and…”
BackgroundMethodInfluentialExamination of Stigmatizing Language in the Electronic Health Record
Himmelstein, G., Bates, D. W., & Zhou, L. (2022) · JAMA Network Open · doi:10.1001/jamanetworkopen.2021.44967 · Cited by 210
Bond, R. M., & Messing, S. (2015) · American Political Science Review · doi:10.1017/s0003055414000525 · Cited by 211
Latent Hatred: A Benchmark for Understanding Implicit Hate Speech
ElSherief, M., Ziems, C., Muchlinski, D., Anupindi, V., Seybolt, J., Choudhury, M. D., & Yang, D. (2021) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2021.emnlp-main.29 · Cited by 201
“Then we computed the log odds ratio with informative Dirichlet prior (Monroe et al., 2008) for each noun, hashtag, and adjective to identify the top 25 words per ideology.”
MethodWho do we invent for? Patents by women focus more on women’s health, but few women get to invent
Koning, R. M., Samila, S., & Ferguson, J. (2021) · Science · doi:10.1126/science.aba6990 · Cited by 184
Rationalizing Democracy: The Perceptual Bias and (Un)Democratic Behavior
Krishnarajan, S. (2022) · American Political Science Review · doi:10.1017/s0003055422000806 · Cited by 192
Word Embeddings: What Works, What Doesn’t, and How to Tell the Difference for Applied Research
Rodriguez, P. L., & Spirling, A. (2022) · The Journal of Politics · doi:10.1086/715162 · Cited by 254
Data-Driven Content Analysis of Social Media
Schwartz, H. A., & Ungar, L. (2015) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716215569197 · Cited by 193
“…scale. through status updates, tweets, and other online personal discourse, people freely post their daily activities, feelings, and thoughts. researchers have begun leveraging these data for a wide range of applications including monitoring influenza and other health outbreaks (ginsberg et al.”
BackgroundAnalyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings
Demszky, D., Garg, N., Voigt, R., Zou, J., Shapiro, J., Gentzkow, M., & Jurafsky, D. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1304 · Cited by 164
“To compare the partisanship of tokens at the event- and topic-level, we also z-score the log odds ratios (Monroe et al., 2008) within events and topics; the most partisan tokens are reported in Appendix F.”
BackgroundInfluentialDiscursive Power in Contemporary Media Systems: A Comparative Framework
Jungherr, A. M., Posegga, O., & An, J. (2019) · The International Journal of Press/Politics · doi:10.1177/1940161219841543 · Cited by 159
“There are a number of promising approaches, such as model-based keyword selection (Monroe et al. 2008) or Supervised Hierarchical Latent Dirichlet allocation (SHLDA) (Nguyen 2015).”
MethodAutomated Text Classification of News Articles: A Practical Guide
Barberá, P., Boydstun, A., Linn, S., McMahon, R., & Nagler, J. (2020) · Political Analysis · doi:10.1017/pan.2020.8 · Cited by 156
Model-based Clustering and Typologies in the Social Sciences
Ahlquist, J. S., & Breunig, C. (2011) · Political Analysis · doi:10.1093/pan/mpr039 · Cited by 158
“Mixture models and computational tools are already being developed to address this exact problem (Hillard, Purpura, and Wilkerson 2008; Monroe, Colaresi, and Quinn 2008; Quinn et al. 2010; Grimmer 2010).”
MethodTang, S. X., Kriz, R., Cho, S., Park, S. J., Harowitz, J., Gur, R. E., Bhati, M. T., Wolf, D. H., Sedoc, J., & Liberman, M. (2021) · Schizophrenia · doi:10.1038/s41537-021-00154-3 · Cited by 133
DLATK: Differential Language Analysis ToolKit
Schwartz, H. A., Giorgi, S., Sap, M., Crutchley, P., Ungar, L., & Eichstaedt, J. (2017) · Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/d17-2010 · Cited by 130
Defining the Enemy: How Donald Trump Frames the News Media
Meeks, L. M. (2019) · Journalism & Mass Communication Quarterly · doi:10.1177/1077699019857676 · Cited by 130
“Previous research has shown that people retweet as an expression of agreement, endorsement, or trust in the content or author (Metaxas & TwitterTrails Research Team, 2017), or to publicly agree with someone and validate others’ thoughts (boyd et al., 2010).”
BackgroundResultInfluentialA Framework for the Computational Linguistic Analysis of Dehumanization
Mendelsohn, J., Tsvetkov, Y., & Jurafsky, D. (2020) · Frontiers in Artificial Intelligence · doi:10.3389/frai.2020.00055 · Cited by 148
“Scholars have also developed computational methods to identify lexical cues of partisan political speech, political slant in mass media, and polarization in social media (Monroe et al., 2008; Gentzkow and Shapiro, 2010; Demszky et al., 2019).”
MethodScattertext: a Browser-Based Tool for Visualizing how Corpora Differ
Kessler, J. S. (2017) · Annual Meeting of the Association for Computational Linguistics: System Demonstrations (ACL Demo) · doi:10.18653/v1/p17-4015 · Cited by 138
“The reader is directed to Monroe et al. (2008) (subsequently referred to as MCQ) for an overview of model-based term scoring algorithms.”
BackgroundGeneral purpose computer-assisted clustering and conceptualization
Grimmer, J., & King, G. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1018067108 · Cited by 179
Field, A., Kliger, D., Wintner, S., Pan, J., Jurafsky, D., & Tsvetkov, Y. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1393 · Cited by 120
“Then, we use log odds with a Dirichlet prior (Jurafsky et al., 2014; Monroe et al., 2008) to identify words that are overrepresented or underrepresented from M + t to M + t +1 and from M − t to M − t +1 .”
MethodHow Getting the Facts Right Can Fuel Partisan‐Motivated Reasoning
Bisgaard, M. (2019) · American Journal of Political Science · doi:10.1111/ajps.12432 · Cited by 147
Lucy, L., Demszky, D., Bromley, P., & Jurafsky, D. (2020) · AERA Open · doi:10.1177/2332858420940312 · Cited by 119
“To compare the descriptors (adjectives or verbs) of two different groups of people A and B , we calculate the weighted log-odds-ratio with informative Dirichlet prior of the words associated with them, as described in Section 3.5.1 of Monroe et al. (2008).”
MethodInfluentialMarked Personas: Using Natural Language Prompts to Measure Stereotypes in Language Models
Cheng, M., Durmus, E., & Jurafsky, D. (2023) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/2023.acl-long.84 · Cited by 228
Demographic dialectal variation in social media: a case study of African-American English
Blodgett, S. L., Green, L., & O'Connor, B. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1120 · Cited by 291
“This model has broadly similar goals as nonlatent, log-linear generative models of text that condition on document-level covariates (Monroe et al., 2008; Eisenstein et al., 2011a; Taddy, 2013).”
MethodLook who’s talking: Parliamentary debate in the European Union
Slapin, J. B., & Proksch, S. (2010) · European Union Politics · doi:10.1177/1465116510369266 · Cited by 105
“parties (Laver and Benoit, 2002; Monroe and Maeda, 2004; Proksch and Slapin, forthcoming), lexical differences between and rhetorical tools used by politicians (Monroe et al., 2008; Klebanov et al., 2008), or the dynamics of coalition government using the length of debates (Martin and Vanberg, 2008).”
BackgroundThe strength of long-range ties in population-scale social networks
Park, P. S., Blumenstock, J. E., & Macy, M. W. (2018) · Science · doi:10.1126/science.aau9735 · Cited by 104
Text as Policy: Measuring Policy Similarity through Bill Text Reuse
Linder, F., Desmarais, B. A., Burgess, M., & Giraudy, E. (2018) · Policy Studies Journal · doi:10.1111/psj.12257 · Cited by 103
Proceedings of the 15th Python in Science Conference
(2016) · Proceedings of the Python in Science Conferences · doi:10.25080/majora-629e541a-014 · Cited by 94
Click and mortar: Organizations on the web
Powell, W. W., Horvath, A., & Brandtner, C. (2016) · Research in Organizational Behavior · doi:10.1016/j.riob.2016.07.001 · Cited by 90
A Neural Model for User Geolocation and Lexical Dialectology
Rahimi, A., Cohn, T., & Baldwin, T. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2033 · Cited by 90
“The main idea is to find lexical variables that are disproportionately distributed in different locations either via model-based or statistical methods (Monroe et al., 2008).”
BackgroundKnowledge Enhanced Masked Language Model for Stance Detection
Kawintiranon, K., & Singh, L. (2021) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/2021.naacl-main.376 · Cited by 93
“While TF-IDF is the preferred method for identifying important words in a corpus, we are interested in identifying important words for distinguishing stance, not just words that are important within the corpus.”
BackgroundMethodInfluentialClassifying the Political Leaning of News Articles and Users from User Votes
Zhou, D. X., Resnick, P., & Mei, Q. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14108 · Cited by 84
“One line of work used word frequencies, Bayesian statistical models, and topic models (Laver et al 2003, Martin and Vanberg 2008, Lin 2006, Lin et al 2008, Monroe et al 2008, Slaping and Proksch 2008).”
MethodNatural Sentences as Valid Units for Coded Political Texts
Däubler, T., Benoit, K. R., Mikhaylov, S., & Laver, M. (2012) · British Journal of Political Science · doi:10.1017/s0007123412000105 · Cited by 83
ConvoKit: A Toolkit for the Analysis of Conversations
Chang, J. P., Chiam, C., Fu, L., Wang, A., Zhang, J., & Danescu-Niculescu-Mizil, C. (2020) · Annual Meeting of the Special Interest Group on Discourse and Dialogue · doi:10.18653/v1/2020.sigdial-1.8 · Cited by 83
Validating Estimates of Latent Traits from Textual Data Using Human Judgment as a Benchmark
Lowe, W., & Benoit, K. R. (2013) · Political Analysis · doi:10.1093/pan/mpt002 · Cited by 81
“This is similar to the results of Monroe, Quinn, and Colaresi (2008), who found that uncorrected partisan association measures for female pronouns (of the kind that the Poisson scaling model uses) indicated that they were Democrat words. three MSc students in political science from Trinity College…”
ResultHollibaugh, G. E. (2018) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muy045 · Cited by 77
Gendered knowledge in fields and academic careers
Kim, L., Smith, D. S., Hofstra, B., & McFarland, D. A. (2021) · Research Policy · doi:10.1016/j.respol.2021.104411 · Cited by 76
A Gatekeeper among Gatekeepers
Welbers, K., van Atteveldt, W., Kleinnijenhuis, J., & Ruigrok, N. (2016) · Journalism Studies · doi:10.1080/1461670x.2016.1190663 · Cited by 76
No! Formal Theory, Causal Inference, and Big Data Are Not Contradictory Trends in Political Science
Monroe, B. L., Pan, J., Roberts, M. E., Sen, M., & Sinclair, B. (2014) · PS Political Science & Politics · doi:10.1017/s1049096514001760 · Cited by 75
Self-citationEmotions, Oral Arguments, and Supreme Court Decision Making
Black, R. C., Treul, S. A., Johnson, T. R., & Goldman, J. (2011) · The Journal of Politics · doi:10.1017/s002238161100003x · Cited by 75
Embedding Regression: Models for Context-Specific Description and Inference
RODRIGUEZ, P. L., SPIRLING, A., & STEWART, B. M. (2023) · American Political Science Review · doi:10.1017/s0003055422001228 · Cited by 70
“Although there have been efforts to understand differential word rate of use within topics pertaining to these terms (e.g., Monroe, Colaresi, and Quinn 2008), there has been relatively little work on whether the same words appear in different contexts.”
BackgroundRaman, N., Cao, M., Tsvetkov, Y., Kästner, C., & Vasilescu, B. (2020) · Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering: New Ideas and Emerging Results · doi:10.1145/3377816.3381732 · Cited by 70
“To alleviate this risk, we identified, using log odds with Dirichlet prior [29], words that are significantly overrepresented in software engineering language compared to general English, and replaced those words with a neutral filler word, so that the sentence structure would not be modified.”
MethodJensen, J., Kaplan, E. D., Naidu, S., & Wilse-Samson, L. H. (2012) · Brookings Papers on Economic Activity · doi:10.1353/eca.2012.0017 · Cited by 70
PsyQA: A Chinese Dataset for Generating Long Counseling Text for Mental Health Support
Sun, H., Lin, Z., Zheng, C., Liu, S., & Huang, M. (2021) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2021.findings-acl.130 · Cited by 78
“Without the clutter of unimportant words”
Chuang, J., Manning, C. D., & Heer, J. (2012) · ACM Transactions on Computer-Human Interaction · doi:10.1145/2362364.2362367 · Cited by 69
Sentiment and position-taking analysis of parliamentary debates: a systematic literature review
Abercrombie, G., & Batista-Navarro, R. (2019) · Journal of Computational Social Science · doi:10.1007/s42001-019-00060-w · Cited by 83
“• Textual: including word (all studies) or character [32] n-grams, custom dictionary keyword features [12], words from particular grammatical categories [28, 32, 35, 47, 48, 52, 72, 75], presence of questions [12], word embeddings [9, 21, 28, 30, 38, 48, 61, 62], sentence [64] embeddings, and parse trees [6, 28, 30].”
BackgroundGendered Conversation in a Social Game-Streaming Platform
Nakandala, S., Ciampaglia, G. L., Su, N. M., & Ahn, Y. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14885 · Cited by 67
Framing in Social Media: How the US Congress Uses Twitter Hashtags to Frame Political IssuesPreprint
Hemphill, L., Culotta, A., & Heston, M. (2013) · SSRN Electronic Journal · doi:10.2139/ssrn.2317335 · Cited by 56
Detecting Biased Statements in Wikipedia
Hube, C., & Fetahu, B. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3184558.3191640 · Cited by 65
“In our work, we show that similar to [13], words that introduce bias or violate NPOV are dependent on the context in which they appear and furthermore the topic at hand.”
BackgroundWhat happened? The Spread of Fake News Publisher Content During the 2016 U.S. Presidential Election
Budak, C. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308558.3313721 · Cited by 71
“We adopt the log-odds-ratio technique with informative Dirichlet priors [48] to compute the fake-ness measure for each word for each candidate.”
MethodAudience-Modulated Variation in Online Social Media
Pavalanathan, U., & Eisenstein, J. (2015) · American Speech · doi:10.1215/00031283-3130324 · Cited by 61
Conversational Flow in Oxford-style Debates
Zhang, J., Kumar, R., Ravi, S., & Danescu-Niculescu-Mizil, C. (2016) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n16-1017 · Cited by 61
“To find these words, we use the method introduced by Monroe et al. (2008) in the context of U.S. Senate speeches.”
BackgroundMethodD'Orazio, V. J., Landis, S. T., Palmer, G., & Schrodt, P. A. (2014) · Political Analysis · doi:10.1093/pan/mpt030 · Cited by 60
“…been standard approaches in natural language processing literature for some time (Luhn 1958; Rijsbergen 1979; Dasgupta et al. 2007) and are found in political science applications (Hopkins and King 2010; Spirling 2012), although some have made arguments against these practices (Monroe et al. 2008).”
BackgroundMethodNelson, L. K. (2021) · American Journal of Sociology · doi:10.1086/714915 · Cited by 65
More Voices Than Ever? Quantifying Media Bias in Networks
Lin, Y., Bagrow, J. P., & Lazer, D. M. J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14128 · Cited by 59
Deconfounded Lexicon Induction for Interpretable Social Science
Pryzant, R., Shen, K., Jurafsky, D., & Wagner, S. (2018) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n18-1146 · Cited by 60
“, 2016), log-odds (Szumilas, 2010; Monroe et al., 2008), mutual information (Berg, 2004), or matching (Tan et al.”
BackgroundModeling Perspective using Adaptor Grammars
Hardisty, E. A., & Resnik, P. (2010) · Cited by 58
Lexical and Hierarchical Topic Regression
Nguyen, V.-A., Boyd-Graber, J. L., & Resnik, P. (2013) · Cited by 58
Shedding (a Thousand Points of) Light on Biased Language
Yano, T., Resnik, T., & Smith, N. A. (2018) · doi:10.1184/r1/6473687 · Cited by 55
Digitization of the Canadian Parliamentary Debates
Beelen, K., Thijm, T. A., Cochrane, C., Halvemaan, K., Hirst, G., Kimmins, M., Lijbrink, S., Marx, M., Naderi, N., Rheault, L., et al. (2017) · Canadian Journal of Political Science · doi:10.1017/s0008423916001165 · Cited by 55
“…examples ranging from the detection of ideology and partisan polarization in speech to structural topic modelling (for example, Diermeier et al., 2012; Grimmer and King, 2011; Monroe et al., 2008; Proksch and Slapin, 2010; Roberts et al., 2014; see Grimmer and Stewart, 2013 for a recent summary).”
BackgroundSomething’s Brewing! Early Prediction of Controversy-causing Posts from Discussion Features
Hessel, J., & Lee, L. (2019) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.18653/v1/n19-1166 · Cited by 56
“We utilize Monroe et al.’s (2008) algorithm for comparing language usage in two bodies of text; the method places a Dirichlet prior over n-grams (n=1,2,3) and estimates Z-scores on the difference in rate-usage between controversial and non-controversial posts.”
MethodElevated threat levels and decreased expectations: How democracy handles terrorist threats
Bonilla, T., & Grimmer, J. (2013) · Poetics · doi:10.1016/j.poetic.2013.06.003 · Cited by 54
Bag of What? Simple Noun Phrase Extraction for Text Analysis
Handler, A., Denny, M. J., Wallach, H., & O'Connor, B. (2016) · Workshop on Natural Language Processing and Computational Social Science (NLP+CSS) · doi:10.18653/v1/w16-5615 · Cited by 54
“Many political scientists have studied the relationship between language usage and party affiliation (Laver et al., 2003; Monroe et al., 2008; Slapin and Proksch, 2008; Quinn et al., 2010; Grimmer and Stewart, 2013).”
BackgroundMethodInfluentialSánchez-Franco, M., & Rey-Tienda, S. (2023) · Management Decision · doi:10.1108/md-06-2023-0966 · Cited by 43
RtGender: A Corpus for Studying Differential Responses to Gender
Voigt, R., Jurgens, D., Prabhakaran, V., Jurafsky, D., & Tsvetkov, Y. (2018) · Language Resources and Evaluation Conference (LREC) · doi:10.63317/3di79eiki43x · Cited by 52
Proceedings of the Third Workshop on Abusive Language Online
(2019) · doi:10.18653/v1/w19-35 · Cited by 52
Causality and History: Modes of Causal Investigation in Historical Social Sciences
Ermakoff, I. (2019) · Annual Review of Sociology · doi:10.1146/annurev-soc-073117-041140 · Cited by 56
“…(Bourdieu 1988, Sapiro 2014, Slez & Martin 2007), individuals holding equivalent positions in a network structure (Bearman 1993, Gould 1996, Hillmann 2008b, Padgett & Ansell 1993), or textual elements (McLean 2007, p. 124; Monroe et al. 2008; Rule et al. 2015; Schonhardt-Bailey 2006, ch. 7–8).”
BackgroundShen, Q., & Rosé, C. P. (2019) · Workshop on Abusive Language Online (ALW) · doi:10.18653/v1/w19-3507 · Cited by 50
“…within each topic t for left-leaning and right-leaning users, for each word w , we use the z-score of the log-odds ratio with a Dirichlet prior (Monroe et al., 2008) as a salience score, δ where n c ( t ) is the number of words in corpus c , y w is the count of word w in corpus c ( t ) , l ( t…”
MethodFrameAxis: characterizing microframe bias and intensity with word embedding
Kwak, H., An, J., Jing, E., & Ahn, Y. (2021) · PeerJ Computer Science · doi:10.7717/peerj-cs.644 · Cited by 59
“To answer this question, we compare the log odds ratio with informative Dirichlet prior (Monroe et al., 2017).”
Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise
Wang, R., Ribeiro, A., Robinson, C., Loeb, S., & Demszky, D. (2024) · doi:10.21203/rs.3.rs-5363154/v1 · Cited by 53
AugESC: Dialogue Augmentation with Large Language Models for Emotional Support Conversation
Zheng, C., Sabour, S., Wen, J., Zhang, Z., & Huang, M. (2023) · Findings of the Association for Computational Linguistics: ACL (Findings of ACL) · doi:10.18653/v1/2023.findings-acl.99 · Cited by 84
“We calculated the log odds ratio, informative Dirichlet prior (Monroe et al., 2008) of all words for each dataset contrasting to the other.”
MethodTextual Predictors of Bill Survival in Congressional Committees
Yano, T., Smith, N. A., & Wilkerson, J. (2018) · doi:10.1184/r1/6473729 · Cited by 48
Unsupervised Discovery of Implicit Gender Bias
Field, A., & Tsvetkov, Y. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.44 · Cited by 58
Sharing Means Renting?: An Entire-Marketplace Analysis of AirbnbPreprint
Ke, Q. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2902840 · Cited by 47
Rahimi, A., Baldwin, T., & Cohn, T. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1016 · Cited by 46
“The two predominant approaches to this problem are model-based (Eisenstein et al., 2010; Ahmed et al., 2013; Eisenstein, 2015) and through the use of statistical metrics (Monroe et al., 2008; Cook et al., 2014).”
MethodBilateral Tensions, the Trade War, and US–China Trade Relations
Zeng, K., Wells, R., Gu, J., & Wilkins, A. (2022) · Business and Politics · doi:10.1017/bap.2022.8 · Cited by 47
From Optimism to Concern: Unveiling Sentiments and Perceptions Surrounding ChatGPT on Twitter
Demirel, S., Kahraman-Gokalp, E., & Gündüz, U. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2392964 · Cited by 46
“Besides, to find potential differences in word usage in tweets from various continents, we utilized complex calculations known as weighted log odds (Monroe et al., 2008).”
Ten Social Dimensions of Conversations and Relationships
Choi, M., Aiello, L. M., Varga, K. Z., & Quercia, D. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366423.3380224 · Cited by 49
“To reduce the sparsity of the ngram space, we considered only those that occur 10 times or more in the training set and we filtered them using log-odd Dirichlet priors to further narrow the set to those n-grams that are highly discriminative [85].”
MethodWeakly Supervised Learning of Nuanced Frames for Analyzing Polarization in News Media
Roy, S., & Goldwasser, D. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.620 · Cited by 49
Ke, Q. (2017) · ACM Web Science Conference (WebSci) · doi:10.1145/3091478.3091504 · Cited by 81
“5 or less star listings (corpora j)? To answer this, we employ a widely used method called “log-odds-ratio, informative Dirichlet prior” [38], which has been used to analyze”
Metadata
- Authors
- Monroe, B. L., Colaresi, M. P., & Quinn, K. M.
- Affiliations
- Pennsylvania State University (US)
- Michigan State University (US)
- Harvard University (US)
- Corresponding author
- Monroe, Burt L.
- Colaresi, Michael P.
- Quinn, Kevin M.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2008
- Volume
- 16
- Issue
- 4
- Pages
- 372-403
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2038411619
- Semantic Scholar
- ebc37575aa4e4afeb1dc94a18022cc0ebeb2fe09
- Cited by
- 542
- References
- 54
- Citations by year
- 2026: 44 · 2025: 41 · 2024: 51 · 2023: 52 · 2022: 28 · 2021: 69 · 2020: 38 · 2019: 38 · 2018: 32 · 2017: 30 · 2016: 29 · 2015: 16 · 2014: 16 · 2013: 20 · 2012: 13
- FWCI
- 42.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- public-domain
- www.cambridge.org/core/terms (unspecified, 3291 d)
- Locations
- Political Analysis (publishedVersion, public-domain)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
- Keywords
- political conflict, Bayesian shrinkage, regularization, text-as-data, distributive politics, United States Senate, ideological space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
