The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; none of the 21 sources its claims cite is checked for support yet.
Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict
Open accessAbstract
Entries in the burgeoning “text-as-data” movement are often accompanied by lists or visualizations of how word (or other lexical feature) usage differs across some pair or set of documents. These are intended either to establish some target semantic concept (like the content of partisan frames) to estimate word-specific measures that feed forward into another analysis (like locating parties in ideological space) or both. We discuss a variety of techniques for selecting words that capture partisan, or other, differences in political speech and for evaluating the relative importance of those words. We introduce and emphasize several new approaches based on Bayesian shrinkage and regularization. We illustrate the relative utility of these approaches with analyses of partisan, gender, and distributive speech in the U.S. Senate.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Monroe critiques common lexical feature-selection and evaluation methods for text-as-data and proposes model-based Bayesian shrinkage/regularization approaches, illustrated on U.S. Senate speech. The recommended methods reduce overemphasis on very frequent or very rare words and yield more semantically valid partisan word lists.
Key findings
- Nonmodel-based measures fail to account for sampling variation: difference of proportions is dominated by high-frequency words (top Democratic list includes “to,” “my,” “and”; top Republican list includes “the,” “of,” “not”), while log-odds-ratios are dominated by obscure low-frequency words (e.g., “bankruptci,” “Snowe,” “infant,” “Chines”). The paper notes the odds of a Republican using “babi” are 5.4 times a Democrat’s, but the odds of using “April” are 7.4 times a Democrat’s.
- Model-based z-scores of log-odds-ratios improve semantic validity: abortion speech features Republican framing around baby/child and partial-birth procedure, and Democratic framing around woman/women and her/their right to choose.
- An informative Dirichlet prior (prior sample of 500 words per party per day) replaces function words in the Republican top 20 with “aliv,” “infant,” “brutal,” “brain,” and “necessari.” A Laplace prior shrinks most contrasts to exactly zero, leaving prominent Republican words “kill,” “babi,” “procedur,” “child,” “mother,” “abort” and Democratic words “women,” “woman,” “decis,” “famili,” “friend,” “right.”
- Applications show taxes were more partisan than abortion; media odds of using Democratic over Republican tax frames were roughly 20% higher for The New York Times than The Washington Times and roughly 63% higher for NPR than Fox News. Defense polarization fell near zero after 9/11 and rose through and after the invasion of Iraq.
Methods
- Design: methodological critique and proposal, with a running example of U.S. Senate speeches on abortion in the 106th Congress (1999–2000), using topics from Quinn et al. (2006) for Senate speech 1997–2004.
- Data: Congressional Speech Corpus from the public-domain Congressional Record, truncated at December 31, 2004, parsed into tagged XML, filtered for capitalization/punctuation, and stemmed with the Porter Snowball II stemmer.
- Analysis: compares classification, nonmodel-based measures (difference of frequencies/proportions, stop-word removal, odds/log-odds, tf.idf, WordScores), then multinomial/Dirichlet model-based log-odds-ratio estimates, z-scores, informative Dirichlet shrinkage, and Laplace-prior L1 regularization; extends to taxes, gender within Democrats, dynamics, polarization, and state-level dimensionality.
Limitations
- Stated: the Laplace prior’s nondifferentiability increases computational complexity even for point estimates and makes dynamic extension difficult, so it is less ideal for feature evaluation and feed-forward applications; the Dirichlet approach still leaves unclear where to draw the line for feature selection and may overfit if estimates feed forward.
- Stated: the media framing analysis is a “toy example,” and the applications are intended to illustrate possibilities rather than provide full substantive analyses.
- Evident: empirical demonstrations are limited to U.S. Senate speech from 1997–2004 and selected topics; no validation outside this corpus is reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 21 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, in our running example of speech on abortion, we find that the most common words (the, of, and, is, to, a) are considered Republican.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.1 Difference of frequencies
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the case of two ''documents,'' as is the case here, this is nearly identical to the difference of proportions measure.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results for our running example are nearly identical, qualitatively and quantitatively with those from raw difference of proportions, shown in Figure 1.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that regardless of the zero treatment, the most extreme words are obscure ones.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is obvious in Figure 1, the sampling variation in difference of proportions is greatest in high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 3, the extreme W kw are received by the words spoken by only one party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is shown in Figure 5, this has a desirable effect on the function-word problem noted above.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The best example here is Senate, which is deemed Democratic by difference of proportions, but is, in this context, simply a high-frequency word with high sampling variability.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But the lack of overall semantic validity is clear in the overemphasis on high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most partisan words are the words spoken the most by one party, while spoken not once by the other (bankruptci, infant).15”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- It may state two separate claims as one. 35
- Probably not a substantial claim. 42
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This has the same problems, with the dominant words still the same list of obscurities.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most Democratic words are then bankruptci, Snow½e, ratifi, confidenti, and church, and the most Republican words are infant, admit, Chines, industri, and 40.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although function words no longer dominate the lists as with several previous techniques, some still appear to be too prominent in their implied partisanship, among them the, of, be, not, and my.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In this example, they correlate at over 10.998.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The top Democratic word list is dominated by to and includes my, and, and for; the top Republican word list is dominated by the and includes of, not, be, that, you, it, and a.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Figure 4 shows the feature weightings based on the fðD 2 RÞ kw for our running example.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we instead throw out the words with zero counts in one party, the most Democratic words are treati, discrim, abroad, domest, and privacy, and the most Republican words are perfect, subsid, percent, overrid, and cell.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“One example is given by Slapin and Proksch (2008), made more striking because they provide a plot that clearly demonstrates the heteroskedasticity.Their Figure 2 looks like Figure 2 turned on its side.The resulting lists of their Table 1 contain many obscurities.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
What the sources say
- Not checked yetcites Slapin and Proksch (2008)
Strength:boosted· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The weights are correlated (at 10.997 in this case)”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our data, the odds of a Republican using a variant of babi are 5.4 times those of a Democratic when talking about abortion, which seems informative.But, the odds of a Republican using the word April are 7.4 times those of a Democrat when talking about abortion, which seems less so.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.4
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sticking to the familiar metric of odds ratios, the odds of The New York Times using these Democratic over Republican frames are roughly 20% higher than those of the Washington Times.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Chong
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, we find several instances where Republicans ''own'' an issue or frame while in opposition to Clinton are successful in obtaining a policy change during the Bush honeymoon period of early 2001 or in the rally period after 9/11, only to find Democrats ''owning'' the issue when the policy becomes unpopular.”
- Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain
“Dramatic examples include debt when discussing the budget, oil when discussing energy policy, and Iraq when discussing defense policy.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
- Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“During the Bush era, Democrats consistently filibustered his judicial nominees, and Republicans discovered a deeply held belief in presidential deference and up-or-down votes.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Iraq supplies both of these, and in turn, we see relative nonpartisanship pre-9/11 shift to a Republican talking point post-9/11 only to see events on the ground and the cost of the war increase Democratic leverage on the issue.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is also clear variation in the intensity of preferences, with great intensity on multiple issues from the Alaska delegation, but limited intensity from the Alabama delegation.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, the Republican top 20 list has shuffled, with the, you, not, of, and be being replaced by the considerably more evocative aliv, infant, brutal, brain, and necessari.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The prior used in this example has almost no effect on estimates in topics with much more speech and strong partisan differences (like defense, post-Iraq), and overwhelms estimates-as it should-in topics with very little speech or no partisan differences (like symbolic constituent tributes).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Defense is particularly interesting, reaching near zero polarization in the aftermath of 9/11 and dramatically increasing through and after the invasion of Iraq.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For our particular substantive and theoretical interests, pooling all speech together has several undesirable consequences and topic modeling is a crucial prior step.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our proposed solution relies on a model-based approach to avoid inefficiency and shrinkage and regularization to avoid both infinite estimates and overfitting the sample data.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“This was the approach used by Gentzkow and Shapiro (2006).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
What the sources say
- Not checked yetcites Gentzkow and Shapiro (2006)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to the conjugacy, the full Bayesian estimate using the Dirichlet prior is also analytically available in analogous form:”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.3 Estimation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A related problem is that, although there is variation, every word in the vocabulary receives nonzero weight, which could lead to overfitting if these estimates are used in some subsequent model.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“tf.idf is recommended in standard textbooks (Jurafsky and Martin (2000, 651-4) (Manning and Schu ¨tze 1999, 541-4) and is widely used in document search and information retrieval tasks.14”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
What the sources say
- Not checked yetcites Jurafsky and Martin (2000, 651-4)
- Not checked yetcites Manning and Schu ¨tze 1999, 541-4
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eliminating function words not only eliminates words like her inappropriately but it also elevates high-frequency non-stop words inappropriately.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Perhaps the most prominent text-as-data approach in political science is the WordScores procedure (Laver et al. 2003), which embeds a feature evaluation technique in its algorithm.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
What the sources say
- Not checked yetcites Laver et al. 2003
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With the prior, this is a relabeling of the smoothed log-odds-ratio discussed before.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These do not seem like words that most fundamentally define the partisan division on abortion.Words with plausible partisan content on abortion (infant, church) are overwhelmed by oddities that require quite a bit more investigation to interpret (Chines, bankruptci).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“This mistake is also common in many of the emergent text-as-data discussions in political science.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
What the sources say
- Not checked yetcites Slapin and Proksch (2008)
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Working naively, it is unclear what we are to do with words that are spoken by only one party and therefore have infinite odds ratios.If we let the infinite values have infinite weight, the partisan word list consists of only those spoken by a single party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Without the prior, this is of course simply the observed log-odds-ratio.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Republican word choice reflects framing the debate from the point of view of the baby/child and emphasizing the details of the partial birth abortion procedure.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we wish to proceed directly to ML estimation, the lack of covariates results in an immediately available analytical solution for the MLE of b ðiÞ kw .”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.1 The likelihood
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The logic of tf.idf is that the words containing the greatest information about a particular document are the words that appear many times in that document, but in relatively few others.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is, use of any particular Dirichlet prior defined by a affects the posterior exactly as if we had observed in the data an additional a w -1 instances of word w.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.2 Prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problem is again the failure to account for sampling variability.With log-oddsratios, the sampling variation goes down with increased frequency, as is clear in Figure 2. So, this measure will be inappropriately dominated by obscure words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, WordScores demonstrates the same failure to account for sampling variation and the same overweighting of high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Generally, this simply removes the most problematic features without resolving the issue.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is no implied directionality (e.g., Democratic vs. Republican), however, and the v 2 value itself does not have any obvious interpretation as an evaluation weight.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our example, a stop list would eliminate a word like her, which almost definitely has political content in the context of abortion, 9 and a word like their, which might (e.g., women and their doctors).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There are several approaches, but the notion in common is to put a strong conservative prior on the model.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5 Shrinkage and Regularization
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problems we have seen to this point are typical of many contemporary data-rich problems, emerging not just from computational linguistics but from other similarly scaled or ''ill-posed'' machine and statistical learning problems in areas like image processing and gene expression.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5 Shrinkage and Regularization
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variance is based on the absolute frequency of a word in all, or both, documents of interest, and its implied absolute frequency in the associated priors.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, the fact that these estimates could be recovered from appropriate standard Bayesian generalized linear models means the approach is easily extended beyond the setting of a strict partitioning of the document space into discrete groups.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Rollcall-based work suggests that a single ideological dimension accounts for virtually all Congressional divisions (Poole 2005).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
What the sources say
- Not checked yetcites Poole 2005
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As with the above, the effect of any particular smoothing parameter will be affected by the relative volume of speech and the polarization of word use.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“This is important because of a general gender gap across the two parties both in core values (Rokeach 1973) and in framing (Lakoff 2004).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
What the sources say
- Not checked yetcites Rokeach 1973
- Not checked yetcites Lakoff 2004
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variation in language use offers an alternative measure, one that can be differentiated across political topics and at a finer dynamic grain.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is well known that this corresponds to a uniform prior for log(c).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the theoretical issue of majority rule, where you stand depends on where you sit.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can also extend our analysis across time to analyze dynamics across partisan (or extrapartisan) word use.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Republicans were pushing for elimination of the death tax, lowering of capital gains taxes, elimination of the marriage penalty, and flatter brackets; Democrats advocated for a tax credit for prescription drug coverage, for college, and against lowering the estate tax or giving breaks to the wealthiest.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Chong
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Not only can we look at the relative differences in language use between men and women, as we did with party, but we can also do so while controlling for party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A higher smoothing parameter would result in even fewer nonzero features; a smaller one would result in more.19”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The nondifferentiability at the mode does considerably increase the difficulty and computational overhead associated with estimation.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“One of the more prominent is the concern over whether women legislators behave differently than male legislators and whether they offer a fundamentally different form of representation by speaking ''in a different voice'' (Gilligan 1993;Williams 1998;Diaz 2005).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
What the sources say
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, this low computational overhead makes it relatively straightforward to develop more flexible variants, such as a dynamic version we discuss below.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The primary measure of polarization at the elite level has come from rollcall analysis, following Poole and Rosenthal (1997).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
What the sources say
- Not checked yetcites Poole and Rosenthal (1997)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach still suffers as a technique for feature selection: it is unclear where one draws the line between words that qualitatively matter and words that qualitatively do not.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to observe shifts in the tectonic plates of partisan conflict, we need both salience (the parties and presumably the public must care) and some exogenous shock or change in the status quo (to motivate change).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The analytic solution of the Dirichlet model makes dynamic extensions straightforward.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“A large-scale example clearly matching the criteria for a ''realignment'' or an ''issue evolution'' is the reversal of the parties on race and civil rights issues over the course of the 20th century; a small-scale example, which may or may not meet the criteria, is the polarization over time of the parties on the issue of abortion.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
What the sources say
- Not checked yetcites Carmines and Stimson 1990
- Not checked yetcites Poole and Rosenthal 1997
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Maximization of the posterior densities in equations 28 and 29 is complicated by the discontinuities in the partial derivatives that are introduced by the Laplace prior.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Appendix-Estimation of the Laplace Model
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These problems lead researchers to select and weight feature/word lists that have low semantic validity or vary wildly depending on arbitrary decisions.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 5 Discussion and Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
- Rests on an assumption: More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.
None of the sources cited for this claim have been checked yet.
In the paper
“Using results from Cawley and Talbot (2006), it is easy to show how the value of b ðiÞ k that maximizes pðb ðiÞ k jy ðiÞ k Þ } Z p À y ðiÞ k jb ðiÞ k Þpðb ðiÞ k jcÞp J ðcÞdcðA1Þ can be calculated via a modified version of the iterative approach of Shevade and Keerthi (2003) in which, at each iteration, a working version, c, of c is defined to be:”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Marginalizing over c with a Jeffreys Prior
What the sources say
- Not checked yetcites Cawley and Talbot (2006)
- Not checked yetcites Shevade and Keerthi (2003)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that when a 5 0, the results from this procedure are equivalent to those from the setup where c is given a Jeffreys prior and then integrated out of the posterior.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Marginalizing over c with an Exponential Prior
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is suggestive that there are measurable distributive (and therefore multidimensional) preference differences across Senators (which are unlikely to be present in rollcall votes once partisan coalitions have been built around particular bills).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
The paper relies on this without support.
Different speech patterns across groups can flow from both partisan and nonpartisan sources, including idiosyncrasies of dialect and style.
“Additionally, we know a priori that different speech patterns across groups can flow from both partisan and nonpartisan sources, including idiosyncrasies of dialect and style.”
The authors state this as an assumption.
The approximations in Equations 17 and 19 assume y(i)_kw >> a(i)_kw, y_kw >> a_kw and ignore covariance terms that will typically be close to 0.
“Where the approximations in Equations 17 and 19 assume y ðiÞ kw ) a ðiÞ kw ; y kw ) a kw and ignore covariance terms that will typically be close to 0”
The authors state this as an assumption.
Equations 18 and 20 additionally assume that n(i)_k >> y(i)_kw and n_k >> y_kw.
“while Equations 18 and 20 additionally assume that n ðiÞ k ) y ðiÞ kw and n k ) y kw .”
The paper relies on this without support.
Sports speeches congratulating a home state team have no partisan structure.
“For example, our Senate analysis finds a large number of ''sports'' speeches congratulating a home state team.We would not expect these to have partisan structure.”
The paper relies on this without support.
Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.
“Democrats are both more likely than Republicans to address ''women's issues,'' and also have more women members serving in the Senate than do the Republicans.”
The paper relies on this without support.
The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.
“We conducted Lexis-Nexis searches of two newspapers, The New York Times and The Washington Times, and two broadcast news sources, National Public Radio and Fox News, widely thought to be, relatively speaking, at different partisan extremes.”
The paper relies on this without support.
A genuine partisan realignment or issue evolution generally occurs only over fairly long time frames.
“This generally occurred (or been agreed to have occurred) only over fairly long time frames.”
The paper relies on this without support.
Speech data offers leverage on analyzing partisan conflict.
“Speech offers interesting potential leverage on the problem.”
The paper relies on this without support.
Prior efforts to systematically analyze political conflict have focused on easily quantified behaviors that obscure substance.
“Efforts to systematically analyze political conflict, especially in the American context, have previously focused on behaviors that are easily quantified but obscure substance (e.g., votes).”
The paper relies on this without support.
Language use in political speech is a valid indicator of political conflict and preferences.
“We have seen that static snapshots can illuminate the content of a particular partisan conflict.”
The paper relies on this without support.
More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.
“More recent attempts to investigating political dynamics that summarize and organize text have fallen into other traps, including inefficiency (e.g., classification techniques), and overfitting idiosyncratic group differences in low-or high-frequency words (e.g., tf.idf or difference of proportions).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Monroe, Colaresi, and Quinn (2008) describe a variety of methods that measure how well words separate already identified categories of interest”
From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Monroe, Colaresi, and Quinn (2008)
The cited source says what the claim attributes to it.
In the citing paper
“Although there have been efforts to understand differential word rate of use within topics pertaining to these terms (e.g., Monroe, Colaresi, and Quinn 2008), there has been relatively little work on whether the same words appear in different contexts.”
From Embedding Regression: Models for Context-Specific Description and Inference (2023) · cites this paper as Monroe, Colaresi, and Quinn 2008
In the citing paper
“For example, Monroe et al. explored a variety of techniques that compare two frequencies of words – one number for each of two groups [44]. In particular, they explored frequencies across democratic versus republican speeches and settled on a Bayesian model with regularization and shrinkage based on priors of word use.”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [44]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Nature of Statistical Learning Theory
Vapnik, V. (2000) · Statistics for Engineering and Information Science · doi:10.1007/978-1-4757-3264-1 · Cited by 46,387
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,346
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
(1973) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,144
Flagged first, then by citations
Research works printed but not linked: 9
- Diaz, 2005, Representing women? Female legislators in West European parliaments
- Carmines, 1990, Issue evolution
- The counts of estate, prescription, wage, wealthiest, death, bracket, flat, penalty, respectively, are (2078, 620, 1159, 208, 1280, 164, 431, 743) for The New York Times, (1032, 376, 692, 84, 731, 189, 399, 720) for The Washington Times, (172, 214, 224, 68, 209, 47, 68, 174) for National Public Radio, and (167, 362, 238, 54, 450, 55, 197, 281) for Fox News.
- Monroe, 2004, Society for Political Methodology
- Mucciaroni, 2006, Deliberative choice: Debating public policy in congress
- Riker, 1986, The art of political manipulation
- Emily, 2004, From boys to men
- Lakoff, 2004, Don't think of an elephant! Know your values and frame the debate
- Monroe, 2006, United States congressional speech corpus
Cited by
Sorted by citations
Items 201–300 of 541
Measuring trust: a text analysis approach to compare, contrast, and select trust questionnaires
Alsaid, A., Li, M., Chiou, E. K., & Lee, J. D. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1192020 · Cited by 23
Lee, C., Gligorić, K., Kalluri, P., Harrington, M., Durmus, E., Sanchez, K. L., San, N., Tse, D., Zhao, X., Hamedani, M. G., et al. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2322764121 · Cited by 26
Theming for terror: Organizational adornment in terrorist propaganda
Kinney, A. B., Davis, A. P., & Zhang, Y. (2018) · Poetics · doi:10.1016/j.poetic.2018.05.001 · Cited by 17
Causal Explanation Analysis on Social Media
Son, Y., Bayas, N., & Schwartz, H. A. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1372 · Cited by 17
Measuring Online Debaters’ Persuasive Skill from Text over Time
Luu, K., Tan, C., & Smith, N. A. (2019) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00281 · Cited by 17
The Civic Mission of MOOCs: Engagement across Political Differences in Online Forums
Yeomans, M., Stewart, B., Mavon, K., Kindel, A., Tingley, D. H., & Reich, J. (2017) · International Journal of Artificial Intelligence in Education · doi:10.1007/s40593-017-0161-0 · Cited by 17
“Specifically, we calculated two commonly used statistics that capture strength of association - variance-weighted long-odds ratio and mutual information - to evaluate the relationship between feature frequency and ideology in each class (Monroe et al. 2008).”
Legislative Institutions and Coalition Government
Martin, S., Saalfeld, T., Strøm, K. W., Martin, L. W., & Vanberg, G. (2014) · Oxford University Press eBooks · doi:10.1093/oxfordhb/9780199653010.013.0014 · Cited by 17
FrameAxis: Characterizing Framing Bias and Intensity with Word Embedding.
Kwak, H., An, J., & Ahn, Y. (2020) · IUScholarWorks Open (Indiana University) · Cited by 16
Controlled Analyses of Social Biases in Wikipedia Bios
Field, A., Park, C. Y., Lin, K., & Tsvetkov, Y. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512134 · Cited by 22
“words that are most associated with biography pages about cisgender men and women calculated using log-odds with a Dirichlet prior [32].”
BackgroundMethodInfluentialBreaking Down the Invisible Wall of Informal Fallacies in Online Discussions
Sahai, S. Y., Balalau, O., & Horincar, R. (2021) · doi:10.18653/v1/2021.acl-long.53 · Cited by 16
“We compute statistically overrepresented tokens in each propa-ganda technique in the training set using log-odds ratio with Dirichlet prior (Monroe et al., 2008).”
MethodTowards Emotion-Aware Agents For Negotiation Dialogues
Chawla, K., Clever, R., Ramirez, J., Lucas, G., & Gratch, J. S. (2021) · International Conference on Affective Computing and Intelligent Interaction and workshops · doi:10.1109/acii52823.2021.9597427 · Cited by 16
Gender-Distinguishing Features in Film Dialogue
Schofield, A., & Mehr, L. (2016) · Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL) · doi:10.18653/v1/w16-0204 · Cited by 20
Representativeness of Abortion Legislation Debate on Twitter: A Case Study in Argentina and Chile
Graells-Garrido, E., Baeza-Yates, R., & Lalmas, M. (2020) · The Web Conference · doi:10.1145/3366424.3383561 · Cited by 19
“To characterize the words most associated with each demographic group (country, gender, age cohort), Table 4 shows the most discriminating terms in both, tweets and biographies, weighted using log-odds ratio with uninformative Dirichlet prior [33].”
MethodMedia, markets and institutional change: evidence from the Protestant ReformationPreprint
Dittmar, J. E., & Seabold, S. (2015) · RePEc: Research Papers in Economics · Cited by 16
The decade-long growth of government-authored news media in China under Xi Jinping
Waight, H., Yuan, Y., Roberts, M. E., & Stewart, B. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2408260122 · Cited by 19
Shihadeh, J., Ackerman, M., Troske, A., Lawson, N. M., & Gonzalez, E. (2022) · IEEE Global Humanitarian Technology Conference (GHTC) · doi:10.1109/ghtc55712.2022.9910995 · Cited by 18
“The log-odds-ratio is calculated based on prior probabilities of words and word frequency counts using the informative Dirichlet prior described in Section 3.5.1 of Monroe et al. [34].”
MethodInfluentialChawla, K., Clever, R., Ramirez, J., Lucas, G., & Gratch, J. S. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2023.3238007 · Cited by 15
“We next compute the log-odds ratio, informative Dirichlet prior [47] of all the tokens for each emotion dimension relative to all other dimensions.”
MethodAnalyzing the direct role of governmental organizations in artificial intelligence innovation
Park, J. (2023) · The Journal of Technology Transfer · doi:10.1007/s10961-023-10048-4 · Cited by 16
“…abstracts invented by government organizations compared to those by non-government organizations, we used log-odds ratios with informative Dirichlet priors, which have been used in previous studies to identify over-representative words in text data (Monroe et al., 2008; Nakandala et al., 2017).”
Clark, J. H., Osborn, T., Winburn, J., & Wright, G. (2009) · State Politics & Policy Quarterly · doi:10.1177/153244000900900305 · Cited by 15
Wang, E., Luceri, L., Pierri, F., & Ferrara, E. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22197 · Cited by 15
Concise comparative summaries (CCS) of large text corpora with a human experiment
Jia, J., Miratrix, L. W., Yu, B., Gawalt, B., Ghaoui, L. E., Barnesmoore, L., & Clavier, S. M. (2014) · The Annals of Applied Statistics · doi:10.1214/13-aoas698 · Cited by 15
Grimmer, J., & King, G. (2009) · Cited by 14
National character stereotypes mirror language use: A study of Canadian and American tweets
Snefjella, B., Schmidtke, D., & Kuperman, V. (2018) · PLoS ONE · doi:10.1371/journal.pone.0206188 · Cited by 16
Predicting Partisan Responsiveness: A Probabilistic Text Mining Time-Series Approach
Buštíková, L., Siroky, D. S., Alashri, S., & Alzahrani, S. (2019) · Political Analysis · doi:10.1017/pan.2019.18 · Cited by 18
Why are A‐list bloggers continuously popular?
Ko, H. (2012) · Online Information Review · doi:10.1108/14684521211241422 · Cited by 15
“TF refers to a measure of how often a word is found in a collection of documents (Monroe et al., 2008).”
BackgroundIt Is Not Only About Grievances: Emotional Dynamics in Social Media During the Brazilian Protests
Costa, J. M. D. R., Rotabi, R., Murnane, E. L., & Choudhury, T. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v9i1.14667 · Cited by 14
Joseph, K., Swire‐Thompson, B., Masuga, H., Baum, M., & Lazer, D. M. J. (2019) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v13i01.3230 · Cited by 13
“As Monroe, Colaresi, and Quinn (2008) has argued, this decision is important especially when signal from a given data point is limited (e.g. there are few retweets or replies for a given tweet).”
BackgroundVoices from the far right: a text analysis of Swedish parliamentary debatesPreprint
Magnusson, M., Öhrvall, R., Barrling, K., & Mimno, D. (2018) · doi:10.31235/osf.io/jdsqc · Cited by 14
Küpfer, A. (2024) · Political Analysis · doi:10.1017/pan.2024.7 · Cited by 13
Zuromski, K. L., Low, D. M., Jones, N. C., Kuzma, R., Kessler, D. T., Zhou, L., Kastman, E. K., Epstein, J., Madden, C., Ghosh, S., et al. (2024) · Psychological Medicine · doi:10.1017/s0033291724001557 · Cited by 14
Dodds, P. S., Minot, J. R., Arnold, M., Alshaabi, T., Adams, J. L., Reagan, A. J., & Danforth, C. M. (2021) · PLoS ONE · doi:10.1371/journal.pone.0260592 · Cited by 19
“For data, we use Twitter as a vast, noisy, and distributed news and opinion aggregation service [19–23], Beyond the centrality of Twitter to Trump’s communications [24–29], a key benefit of using Twitter as “text as data” [30–34] is that popularity of story is encoded and recorded through social amplification by retwe…”
MethodAnalysing community reaction to refugees through text analysis of social media data
Kelling, C., & Monroe, B. L. (2022) · Journal of Ethnic and Migration Studies · doi:10.1080/1369183x.2022.2100551 · Cited by 14
“We characterise this discourse by applying computational text analysis techniques – the Structural Topic Model (Roberts, Stewart, and Tingley 2014) and ‘Fightin’ Words’ (Monroe, Colaresi, and Quinn 2008) – to a novel dataset comprised of comments on refugee-related local news articles posted on Facebook in Roanoke, Vi…”
MethodInfluentialSelf-citationFriendships, Rivalries, and Trysts: Characterizing Relations between Ideas in Texts
Tan, C., Card, D., & Smith, N. A. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-1072 · Cited by 14
“We identify a list of distinguishing keywords for each corpus by comparing its word frequencies to the background frequencies found in other corpora using the informative Dirichlet prior model in Monroe et al. (2008).”
MethodLi, P., Schloss, B., & Follmer, D. J. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0931-5 · Cited by 16
Semantic HypergraphsPreprint
Menezes, T., & Roth, C. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.10784 · Cited by 24
“Similarly, political positions in documents may be inferred from Wordscores [20] – a popular method in political science that also relies on the summation of pre-computed scores for individual words, and has more refined elaborations, e.g. with Bayesian regularization [24].”
BackgroundText Semantics Capture Political and Economic NarrativesPreprint
Ash, E., Gauthier, G., & Widmer, P. (2021) · SSRN Electronic Journal · doi:10.3929/ethz-b-000504025 · Cited by 15
“Overall, the narratives associated with divisive entities illuminate the key divergences in political priorities between Democrats and Republicans.”
MethodInfluentialReassessing the Effects of Emotions on Turnout
Phillips, J. B., & Plutzer, E. (2023) · The Journal of Politics · doi:10.1086/723816 · Cited by 12
A Framework to Understand Attitudes towards Immigration through Twitter
Freire-Vidal, Y., Graells-Garrido, E., & Rowe, F. (2021) · Applied Sciences · doi:10.3390/app11209689 · Cited by 13
“We weighted each term using log-odds ratios with an Uninformative Dirichlet Prior [73].”
MethodTamilEmo: Fine-grained Emotion Detection Dataset for Tamil
Vasantharajan, C., Priyadharshini, R., Kumaresan, P. K., Ponnusamy, R., Thangasamy, S., Benhur, S., Thenmozhi, D., Sivanraju, K., Anbukkarasi, S., & Chakravarthi, B. R. (2023) · Communications in computer and information science · doi:10.1007/978-3-031-33231-9_3 · Cited by 13
Introduction to the Virtual Issue: Recent Innovations in Text Analysis for Social Science
Roberts, M. E. (2016) · Political Analysis · doi:10.1017/s1047198700014418 · Cited by 12
Schmidtke, D., & Kuperman, V. (2016) · Corpus Linguistics and Linguistic Theory · doi:10.1515/cllt-2015-0047 · Cited by 12
“To achieve this, we applied the log-odds ratio informative Dirichlet prior (LORIDP) method as used by Monroe et al. (2008) (see also Jurafsky et al. 2014).”
MethodWang, Y., & Diakopoulos, N. A. (2021) · ACM Transactions on Social Computing · doi:10.1145/3484245 · Cited by 11
“To rule out the possibility of topical bias in the Pick labels, we ran lexical comparisons following Monroe et. al’s [43] fightin’ words result on a random sample of 1000 Picks and 1000 non-Picks.”
MethodJiang, J., Thomason, J., Barbieri, F., & Ferrara, E. (2023) · ACM Web Science Conference (WebSci) · doi:10.1145/3578503.3583596 · Cited by 11
“From Figure 3, we see that the top emojis used in Check-In tweets, as measured by weighted log odds (Monroe, Colaresi, and Quinn 2008) are noticeably more positive than those of Random tweets; the former contains happy faces and hearts, while the latter consist of an array of sad, contempt, or…”
BackgroundMethodInfluential“Fifty Shades of Bias”: Normative Ratings of Gender Bias in GPT Generated English Text
Hada, R., Seth, A., Diddee, H., & Bali, K. (2023) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2023.emnlp-main.115 · Cited by 12
Exploring equity perception of electric vehicles from a social media perspective
Ruan, T., & Lv, Q. (2024) · Transportation Research Interdisciplinary Perspectives · doi:10.1016/j.trip.2024.101103 · Cited by 12
Wei, Z., Xie, Y., Xiao, D., Zhang, S., Hui, P., & Zhou, M. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589334.3645334 · Cited by 11
Dinh, L., & Walczak, S. M. (2025) · Information Processing & Management · doi:10.1016/j.ipm.2025.104061 · Cited by 11
Coalition oversight and blame avoidance in Greece
Tzelgov, E. (2015) · European Political Science Review · doi:10.1017/s1755773915000284 · Cited by 11
“As suggested by Monroe et al. (2008) and Lowe et al. (2011), in order to compare across parties, I first calculate the odds of using blame avoidance for each party and institutional role.”
Grimmer, J. (2016) · Computational Social Science · doi:10.1017/cbo9781316257340.010 · Cited by 11
Sakamoto, T., & Takikawa, H. (2017) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2017.8258285 · Cited by 11
“The data, which contains more than 180 million words in total and spans a period of more than two decades, far surpasses in its coverage the legislative speech records that have so far been analyzed by other researchers employing topic models [6, 12].”
BackgroundCross-Topic Rumor Detection using Topic-Mixtures
Ren, X., Jiang, J., Khoo, L. M. S., & Chieu, H. L. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.131 · Cited by 11
“Here the topic-specific words are identified based on log-odds ratio with Dirichlet prior (Monroe et al., 2008), and we regard these topic-specific words as possible spurious patterns.”
BackgroundGiving shape to large digital libraries through exploratory data analysis
Organisciak, P., Schmidt, B. M., & Downie, J. S. (2021) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.24547 · Cited by 11
Help! Need Advice on Identifying Advice
Govindarajan, V. S., Chen, B. T., Warholic, R., Erk, K., & Li, J. J. (2020) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/2020.emnlp-main.427 · Cited by 15
“To counteract the tendency of log-odds scores to highlight infrequent lemmas (Monroe et al., 2017), we filter out lemmas that occurred less than 20 times in the train and validation set of our corpus.”
MethodMeasuring Spatial Subdivisions in Urban Mobility with Mobile Phone Data
Graells-Garrido, E., Meta, I., Serra-Burriel, F., Reyes, P., & Cucchietti, F. (2020) · International World Wide Web Conference (WWW) · doi:10.1145/3366424.3384370 · Cited by 10
“This method is known as Log-Odds ratio with Uninformative Dirichlet Prior [22].”
MethodBonica, A. (2016) · RSF The Russell Sage Foundation Journal of the Social Sciences · doi:10.7758/rsf.2016.2.7.02 · Cited by 10
Measuring Societal Biases from Text Corpora with Smoothed First-Order Co-occurrence
Rekabsaz, N., West, R., Henderson, J., & Hanbury, A. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18083 · Cited by 12
“First-order co-occurrence of words has been widely used to calculate societal phenomena particularly through counting and weighting words (Monroe, Colaresi, and Quinn 2008; Kirchler 1992; Rekabsaz et al. 2017).”
MethodQiu, H. S., Vasilescu, B., Kästner, C., Egelman, C., Jaspan, C. N. C., & Murphy-Hill, E. R. (2022) · IEEE/ACM International Conference on Software Engineering: Software Engineering in Society (ICSE-SEIS) · doi:10.1109/icse-seis55304.2022.9793879 · Cited by 34
Miyazaki, K., Uchiba, T., Tanaka, K., An, J., Kwak, H., & Sasahara, K. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22176 · Cited by 12
Kaufman, A. R. (2019) · Presidential Studies Quarterly · doi:10.1111/psq.12629 · Cited by 10
Pick a Fight or Bite your Tongue: Investigation of Gender Differences in Idiomatic Language Usage
Rabinovich, E., Gonen, H., & Stevenson, S. (2020) · International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.454 · Cited by 15
What Sounds “Right” to Me? Experiential Factors in the Perception of Political Ideology
Shen, Q., & Rosé, C. P. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.152 · Cited by 10
“Understanding political discussion on these platforms often requires one to identify the ideologies behind texts, as understanding the viewpoints reflected in a text can provide insight into the partisanship of beliefs (Monroe et al., 2008) or the persuasive strategies used by different ideological groups (Tsur et al.”
BackgroundMethodInfluentialHow do the kids speak? Improving educational use of text mining with child-directed language models
Organisciak, P., Newman, M., Eby, D., Acar, S., & Dumas, D. (2023) · Information and Learning Sciences · doi:10.1108/ils-06-2022-0082 · Cited by 14
Measuring error for adjacent policy position estimates: Dealing with uncertainty using CMP data
Meyer, T. M., & Jenny, M. (2012) · Electoral Studies · doi:10.1016/j.electstud.2012.08.003 · Cited by 10
“5 Computer-based content analyses, which eliminate the human coder, are gaining more and more fans (Diermeier et al., 2012; Laver et al., 2003; Laver and Garry, 2000; Monroe et al., 2008; Slapin and Proksch, 2008).”
BackgroundA Computational Approach to Identifying Cultural Keywords Across Languages
Lim, Z. W., Stuart, H., Deyne, S. D., Regier, T., Vylomova, E., Cohn, T., & Kemp, C. (2024) · Cognitive Science · doi:10.1111/cogs.13402 · Cited by 9
Smith, M. A. L., & Haenschen, K. (2024) · Mass Communication & Society · doi:10.1080/15205436.2024.2347317 · Cited by 9
“Previous research has employed similar techniques: Monroe et al. (2008) evaluated the weights of features to inductively determine the words most representative of Republican and Democratic Congressional speeches on the abortion.”
MiTextExplorer: Linked brushing and mutual information for exploratory text data analysis
O'Connor, B. (2014) · doi:10.3115/v1/w14-3101 · Cited by 9
“For term-covariate analysis, Monroe et al. (2008) contains a review of many different methods, from both political science as well as computer science; they also propose a hierarchical prior method, and to rank by statistical significance via the asymptotic This might be called a “half-pointwise”…”
MethodLaw, Fact, and the Threat of Reversal From Above
Smith, J. L. (2013) · American Politics Research · doi:10.1177/1532673x13488997 · Cited by 9
Non-fear-Based Road Safety Campaign as a Community Service: Contexts from Social Media
Das, S., Dutta, A. K., Mudgal, A., & Datta, S. (2019) · Communications in computer and information science · doi:10.1007/978-3-030-37484-6_5 · Cited by 9
Exploiting Summarization Data to Help Text Simplification
Sun, R., Yang, Z., & Wan, X. (2023) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2023.eacl-main.3 · Cited by 10
Foreword: Understanding Constitutional Reasoning
Dyèvre, A., & Jakab, A. (2013) · German Law Journal · doi:10.1017/s2071832200002133 · Cited by 15
From the Stage to the Audience: Propaganda on Reddit
Balalau, O., & Horincar, R. (2021) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/2021.eacl-main.309 · Cited by 9
“represented words are computed using log-odds ratio with Dirichlet prior (Monroe et al., 2008), and we present the results in Table 4.”
MethodA Poisson convolution model for characterizing topical content with word frequency and exclusivityPreprint
Airoldi, E. M., & Bischof, J. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.4631 · Cited by 9
“In this approach we borrow ideas from the statistical literature, in which models of differential word usage have been leveraged for analyzing writing styles in a supervised setting (Mosteller and Wallace, 1984; Airoldi et al., 2005, 2006, 2007; Monroe et al., 2008), and combine them with ideas from the machine learni…”
BackgroundMethodImproved Data Collection from Online Sources Using Query Expansion and Active LearningPreprint
Linder, F. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.3026393 · Cited by 9
Greene, Z., Cerón, A., Schumacher, G., & Fazekas, Z. (2016) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/ghxj8 · Cited by 9
“This path-breaking research comes with clear prescriptions for the efficient and valid use of automated text techniques in various settings (e.g. Laver et al. 2003; Lowe 2008; Monroe et al. 2008; Proksch and Slapin 2009; Grimmer and Stewart 2013; Lucas et al. 2015).”
MethodAbramitzky, R., Boustan, L. P., Catron, P., Connor, D. S., & Voigt, R. (2021) · doi:10.31235/osf.io/429jp · Cited by 9
“Figure 2 presents two word clouds that provide insight on the potential topics and words that led coders to classify immigrants as refugees or non-refugees, calculated using Monroe et al. (2008)’s log-odds ratio method.”
MethodIncorporating target-aware knowledge into prompt-tuning for few-shot stance detection
Wang, S., Sun, F., Wang, X., & Pan, L. (2024) · Information Processing & Management · doi:10.1016/j.ipm.2024.103815 · Cited by 8
Conducting marine genetic research for whom? Mapping knowledge flows from science to patents
Dunshirn, P., & Zhivkoplias, E. K. (2024) · npj Ocean Sustainability · doi:10.1038/s44183-024-00088-0 · Cited by 8
Allotaxonometry and rank-turbulence divergence: a universal instrument for comparing complex systems
Dodds, P., Minot, J. R., Arnold, M. V., Alshaabi, T., Adams, J. L., Dewhurst, D., Gray, T., Frank, M. R., Reagan, A. J., & Danforth, C. (2020) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00400-x · Cited by 47
“As we describe below, our histograms fully present the meaningful differences between two size-rank distributions, allowing for divergence measures to be overlaid in easily interpretable ways (c.f., [47, 48]).”
BackgroundToward the Automatic Extraction of Policy Networks Using Web Links and Documents
Moschopoulos, T., Iosif, E., Demetropoulou, L., Potamianos, A., & Narayanan, S. S. (2012) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2012.159 · Cited by 8
“Identifying actors, links, and boundaries, i.e., analyzing a policy network’s structure, requires refined techniques and extensive and time-consuming manual collection of data through interviews and questionnaires.”
BackgroundIdentification and Characterization of Newsworthy Verbs in World News
Nye, B., & Nenkova, A. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1166 · Cited by 8
Relational Summarization for Corpus Analysis
Handler, A., & O'Connor, B. T. (2018) · North American Chapter of the Association for Computational Linguistics · doi:10.18653/v1/n18-1159 · Cited by 10
“Examples of such interfaces include: commandline software for examining noteworthy terms or phrases (Squirrell, 2017; Robinson, 2016; Monroe et al., 2008), point-and-click browsers which display named entities and their interconnections on a network diagram (Wright et al.”
BackgroundMedical students in Russia evaluate the training during the COVID-19 pandemic: a student survey
Mukharyamova, L., Алексеевна, З. А., Zhidjaevskij, A., Galimova, L., & Kuznetsov, M. (2021) · BMC Medical Education · doi:10.1186/s12909-021-02997-x · Cited by 8
“For overrepresentation analysis, the weighted logarithmic odds were estimated as described by Monroe et al. (2008) 9 .”
MethodPasquini, G., Ferguson, G., Bouklas, I., Vu, H., Zamani, M., Zhaoyang, R., Harrington, K., Roque, N. A., Mogle, J. A., Schwartz, H. A., et al. (2022) · PLoS ONE · doi:10.1371/journal.pone.0264280 · Cited by 7
Modeling Political Activism around Gun Debate via Social Media
Mejova, Y., An, J., Morales, G. D. F., & Kwak, H. (2022) · ACM Transactions on Social Computing · doi:10.1145/3532102 · Cited by 8
“To compute the over-representation, we use log-odds ratios with informative Dirichlet priors [91] computed by using background word frequency on the entire dataset.”
MethodUnsupervised Approach for Identifying Users’ Political Orientations
Meguebli, Y., Kacimi, M., Doan, B.-L., & Popineau, F. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-06028-6_49 · Cited by 8
“Another line of work used word frequencies, Bayesian statistical models, and topic models [8] [11] [9] [12] [16].”
MethodJin, H., Sakai, T., & Yatani, K. (2014) · International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI) · doi:10.1145/2628363.2628373 · Cited by 8
“[20] examined word classification methods with political content to uncover what word was used primarily by Democrats or Republicans in the United States of America.”
BackgroundHumphreys, A. (2018) · Springer eBooks · doi:10.1007/978-3-319-05542-8_26-1 · Cited by 8
Towards Emotional Support Dialog SystemsPreprint
Liu, S., Zheng, C., DeMasi, O., Sabour, S., Li, Y., Yu, Z., Jiang, Y., & Huang, M. (2021) · arXiv (Cornell University) · doi:10.48550/arxiv.2106.01144 · Cited by 8
“Each feature is ranked by the rounded z-scored log odds ratios (Monroe et al., 2008) in the parentheses.”
MethodDerton, A., Guevara, M., Chen, S., Moningi, S., Kozono, D. E., Liu, D., Miller, T. A., Savova, G. K., Mak, R. H., & Bitterman, D. S. (2023) · JCO Clinical Cancer Informatics · doi:10.1200/cci.22.00196 · Cited by 8
Park, J., Yang, J., Tolbert, A. N., & Bunsold, K. (2024) · Journal of Information Science · doi:10.1177/01655515241238405 · Cited by 10
Sauder, M., Shi, Y., & Lynn, F. B. (2024) · RSF The Russell Sage Foundation Journal of the Social Sciences · doi:10.7758/rsf.2024.10.5.04 · Cited by 7
“Dictionary method is a commonly used approach in text analysis (see, for example, Boni-kowski and Gidron 2016; Monroe, Colaresi, and Quinn 2017).”
Data quality and privacy concerns in digital trace data
Engel, U., & Dahlhaus, L. (2021) · Routledge eBooks · doi:10.4324/9781003024583-23 · Cited by 8
Luo, Y., Gligorić, K., & Jurafsky, D. (2024) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v18i1.31367 · Cited by 7
“1; also Monroe, Colaresi, and Quinn (2008)).”
Legislative Process in International Organizations
Conrad, C. R., & Monroe, N. W. (2020) · International Studies Review · doi:10.1093/isr/viaa060 · Cited by 7
COVID-19 Coverage By Cable and Broadcast Networks
Budak, C., Muddiman, A., Kim, Y., Murray, C., & Stroud, N. J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18118 · Cited by 7
“This approach commonly outperforms simpler methods such as computing differences in frequencies, ratio of frequencies, or the log odds ratio (Monroe, Colaresi, and Quinn 2008; Jurafsky et al. 2014).”
MethodSexism in the Judiciary: The Importance of Bias Definition in NLP and In Our Courts
Gillis, N. B. (2021) · Workshop on Gender Bias in Natural Language Processing (GeBNLP) · doi:10.18653/v1/2021.gebnlp-1.6 · Cited by 8
Gender Stereotypes in Film Language: A Corpus-Assisted Analysis
Busso, L., & Vignozzi, G. (2017) · Accademia University Press eBooks · doi:10.4000/books.aaccademia.2367 · Cited by 7
“In recent times, corpus linguistics and computational linguistics have shown interest in analysing differences in language between genders (Argamom et al, 2003, Baker 2006, Herring & Paolillo 2006, McEnery 2006, Monroe et al. 2008, amongst others).”
BackgroundSchoots, J., Rohr, B., Prajda, K., & Padgett, J. F. (2020) · Poetics · doi:10.1016/j.poetic.2019.101386 · Cited by 7
Metadata
- Authors
- Monroe, B. L., Colaresi, M. P., & Quinn, K. M.
- Affiliations
- Pennsylvania State University (US)
- Michigan State University (US)
- Harvard University (US)
- Corresponding author
- Monroe, Burt L.
- Colaresi, Michael P.
- Quinn, Kevin M.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2008
- Volume
- 16
- Issue
- 4
- Pages
- 372-403
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2038411619
- Semantic Scholar
- ebc37575aa4e4afeb1dc94a18022cc0ebeb2fe09
- Cited by
- 542
- References
- 54
- Citations by year
- 2026: 44 · 2025: 41 · 2024: 51 · 2023: 52 · 2022: 28 · 2021: 69 · 2020: 38 · 2019: 38 · 2018: 32 · 2017: 30 · 2016: 29 · 2015: 16 · 2014: 16 · 2013: 20 · 2012: 13
- FWCI
- 42.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- public-domain
- www.cambridge.org/core/terms (unspecified, 3291 d)
- Locations
- Political Analysis (publishedVersion, public-domain)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
- Keywords
- political conflict, Bayesian shrinkage, regularization, text-as-data, distributive politics, United States Senate, ideological space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
