The DOI matches its Crossref record; 54 of the 54 references it cites were found in a registry or an index; none of the 21 sources its claims cite is checked for support yet.
Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict
Open accessAbstract
Entries in the burgeoning “text-as-data” movement are often accompanied by lists or visualizations of how word (or other lexical feature) usage differs across some pair or set of documents. These are intended either to establish some target semantic concept (like the content of partisan frames) to estimate word-specific measures that feed forward into another analysis (like locating parties in ideological space) or both. We discuss a variety of techniques for selecting words that capture partisan, or other, differences in political speech and for evaluating the relative importance of those words. We introduce and emphasize several new approaches based on Bayesian shrinkage and regularization. We illustrate the relative utility of these approaches with analyses of partisan, gender, and distributive speech in the U.S. Senate.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
Monroe critiques common lexical feature-selection and evaluation methods for text-as-data and proposes model-based Bayesian shrinkage/regularization approaches, illustrated on U.S. Senate speech. The recommended methods reduce overemphasis on very frequent or very rare words and yield more semantically valid partisan word lists.
Key findings
- Nonmodel-based measures fail to account for sampling variation: difference of proportions is dominated by high-frequency words (top Democratic list includes “to,” “my,” “and”; top Republican list includes “the,” “of,” “not”), while log-odds-ratios are dominated by obscure low-frequency words (e.g., “bankruptci,” “Snowe,” “infant,” “Chines”). The paper notes the odds of a Republican using “babi” are 5.4 times a Democrat’s, but the odds of using “April” are 7.4 times a Democrat’s.
- Model-based z-scores of log-odds-ratios improve semantic validity: abortion speech features Republican framing around baby/child and partial-birth procedure, and Democratic framing around woman/women and her/their right to choose.
- An informative Dirichlet prior (prior sample of 500 words per party per day) replaces function words in the Republican top 20 with “aliv,” “infant,” “brutal,” “brain,” and “necessari.” A Laplace prior shrinks most contrasts to exactly zero, leaving prominent Republican words “kill,” “babi,” “procedur,” “child,” “mother,” “abort” and Democratic words “women,” “woman,” “decis,” “famili,” “friend,” “right.”
- Applications show taxes were more partisan than abortion; media odds of using Democratic over Republican tax frames were roughly 20% higher for The New York Times than The Washington Times and roughly 63% higher for NPR than Fox News. Defense polarization fell near zero after 9/11 and rose through and after the invasion of Iraq.
Methods
- Design: methodological critique and proposal, with a running example of U.S. Senate speeches on abortion in the 106th Congress (1999–2000), using topics from Quinn et al. (2006) for Senate speech 1997–2004.
- Data: Congressional Speech Corpus from the public-domain Congressional Record, truncated at December 31, 2004, parsed into tagged XML, filtered for capitalization/punctuation, and stemmed with the Porter Snowball II stemmer.
- Analysis: compares classification, nonmodel-based measures (difference of frequencies/proportions, stop-word removal, odds/log-odds, tf.idf, WordScores), then multinomial/Dirichlet model-based log-odds-ratio estimates, z-scores, informative Dirichlet shrinkage, and Laplace-prior L1 regularization; extends to taxes, gender within Democrats, dynamics, polarization, and state-level dimensionality.
Limitations
- Stated: the Laplace prior’s nondifferentiability increases computational complexity even for point estimates and makes dynamic extension difficult, so it is less ideal for feature evaluation and feed-forward applications; the Dirichlet approach still leaves unclear where to draw the line for feature selection and may overfit if estimates feed forward.
- Stated: the media framing analysis is a “toy example,” and the applications are intended to illustrate possibilities rather than provide full substantive analyses.
- Evident: empirical demonstrations are limited to U.S. Senate speech from 1997–2004 and selected topics; no validation outside this corpus is reported.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
0 of 21 cited sources checked · 0 supported · 0 flagged
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, in our running example of speech on abortion, we find that the most common words (the, of, and, is, to, a) are considered Republican.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.1 Difference of frequencies
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In the case of two ''documents,'' as is the case here, this is nearly identical to the difference of proportions measure.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The results for our running example are nearly identical, qualitatively and quantitatively with those from raw difference of proportions, shown in Figure 1.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Note that regardless of the zero treatment, the most extreme words are obscure ones.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is obvious in Figure 1, the sampling variation in difference of proportions is greatest in high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Figure 3, the extreme W kw are received by the words spoken by only one party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As is shown in Figure 5, this has a desirable effect on the function-word problem noted above.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The best example here is Senate, which is deemed Democratic by difference of proportions, but is, in this context, simply a high-frequency word with high sampling variability.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesA result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“But the lack of overall semantic validity is clear in the overemphasis on high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesThe paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most partisan words are the words spoken the most by one party, while spoken not once by the other (bankruptci, infant).15”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passes
- It may state two separate claims as one. 35
- Probably not a substantial claim. 42
- The quote only partly states this claim. 2
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“This has the same problems, with the dominant words still the same list of obscurities.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The most Democratic words are then bankruptci, Snow½e, ratifi, confidenti, and church, and the most Republican words are infant, admit, Chines, industri, and 40.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Although function words no longer dominate the lists as with several previous techniques, some still appear to be too prominent in their implied partisanship, among them the, of, be, not, and my.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:hedged· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In this example, they correlate at over 10.998.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The top Democratic word list is dominated by to and includes my, and, and for; the top Republican word list is dominated by the and includes of, not, be, that, you, it, and a.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.2 Difference of proportions
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Figure 4 shows the feature weightings based on the fðD 2 RÞ kw for our running example.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we instead throw out the words with zero counts in one party, the most Democratic words are treati, discrim, abroad, domest, and privacy, and the most Republican words are perfect, subsid, percent, overrid, and cell.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“One example is given by Slapin and Proksch (2008), made more striking because they provide a plot that clearly demonstrates the heteroskedasticity.Their Figure 2 looks like Figure 2 turned on its side.The resulting lists of their Table 1 contain many obscurities.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
What the sources say
- Not checked yetcites Slapin and Proksch (2008)
Strength:boosted· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The weights are correlated (at 10.997 in this case)”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Tests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesThe quote only partly states this claim.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our data, the odds of a Republican using a variant of babi are 5.4 times those of a Democratic when talking about abortion, which seems informative.But, the odds of a Republican using the word April are 7.4 times those of a Democrat when talking about abortion, which seems less so.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.4
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Sticking to the familiar metric of odds ratios, the odds of The New York Times using these Democratic over Republican frames are roughly 20% higher than those of the Washington Times.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Chong
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, we find several instances where Republicans ''own'' an issue or frame while in opposition to Clinton are successful in obtaining a policy change during the Bush honeymoon period of early 2001 or in the rally period after 9/11, only to find Democrats ''owning'' the issue when the policy becomes unpopular.”
- Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain
“Dramatic examples include debt when discussing the budget, oil when discussing energy policy, and Iraq when discussing defense policy.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
- Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“During the Bush era, Democrats consistently filibustered his judicial nominees, and Republicans discovered a deeply held belief in presidential deference and up-or-down votes.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one. The quote only partly states this claim.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Iraq supplies both of these, and in turn, we see relative nonpartisanship pre-9/11 shift to a Republican talking point post-9/11 only to see events on the ground and the cost of the war increase Democratic leverage on the issue.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is also clear variation in the intensity of preferences, with great intensity on multiple issues from the Alaska delegation, but limited intensity from the Alabama delegation.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For example, the Republican top 20 list has shuffled, with the, you, not, of, and be being replaced by the considerably more evocative aliv, infant, brutal, brain, and necessari.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Tests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The prior used in this example has almost no effect on estimates in topics with much more speech and strong partisan differences (like defense, post-Iraq), and overwhelms estimates-as it should-in topics with very little speech or no partisan differences (like symbolic constituent tributes).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Defense is particularly interesting, reaching near zero polarization in the aftermath of 9/11 and dramatically increasing through and after the invasion of Iraq.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For our particular substantive and theoretical interests, pooling all speech together has several undesirable consequences and topic modeling is a crucial prior step.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our proposed solution relies on a model-based approach to avoid inefficiency and shrinkage and regularization to avoid both infinite estimates and overfitting the sample data.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 1 Introduction
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim. It may state two separate claims as one.
In the paper
“This was the approach used by Gentzkow and Shapiro (2006).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
What the sources say
- Not checked yetcites Gentzkow and Shapiro (2006)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Due to the conjugacy, the full Bayesian estimate using the Dirichlet prior is also analytically available in analogous form:”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.3 Estimation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A related problem is that, although there is variation, every word in the vocabulary receives nonzero weight, which could lead to overfitting if these estimates are used in some subsequent model.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“tf.idf is recommended in standard textbooks (Jurafsky and Martin (2000, 651-4) (Manning and Schu ¨tze 1999, 541-4) and is widely used in document search and information retrieval tasks.14”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
What the sources say
- Not checked yetcites Jurafsky and Martin (2000, 651-4)
- Not checked yetcites Manning and Schu ¨tze 1999, 541-4
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Eliminating function words not only eliminates words like her inappropriately but it also elevates high-frequency non-stop words inappropriately.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“Perhaps the most prominent text-as-data approach in political science is the WordScores procedure (Laver et al. 2003), which embeds a feature evaluation technique in its algorithm.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
What the sources say
- Not checked yetcites Laver et al. 2003
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“With the prior, this is a relabeling of the smoothed log-odds-ratio discussed before.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These do not seem like words that most fundamentally define the partisan division on abortion.Words with plausible partisan content on abortion (infant, church) are overwhelmed by oddities that require quite a bit more investigation to interpret (Chines, bankruptci).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
In the paper
“This mistake is also common in many of the emergent text-as-data discussions in political science.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
What the sources say
- Not checked yetcites Slapin and Proksch (2008)
Strength:plain· not yet calibratedSingle caseTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Working naively, it is unclear what we are to do with words that are spoken by only one party and therefore have infinite odds ratios.If we let the infinite values have infinite weight, the partisan word list consists of only those spoken by a single party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Without the prior, this is of course simply the observed log-odds-ratio.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Republican word choice reflects framing the debate from the point of view of the baby/child and emphasizing the details of the partial birth abortion procedure.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“If we wish to proceed directly to ML estimation, the lack of covariates results in an immediately available analytical solution for the MLE of b ðiÞ kw .”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.1 The likelihood
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The logic of tf.idf is that the words containing the greatest information about a particular document are the words that appear many times in that document, but in relatively few others.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.7 tf.idf (Computer Science)
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“That is, use of any particular Dirichlet prior defined by a affects the posterior exactly as if we had observed in the data an additional a w -1 instances of word w.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.2 Prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problem is again the failure to account for sampling variability.With log-oddsratios, the sampling variation goes down with increased frequency, as is clear in Figure 2. So, this measure will be inappropriately dominated by obscure words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.5 Log-odds-ratio
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So, WordScores demonstrates the same failure to account for sampling variation and the same overweighting of high-frequency words.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 15
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Generally, this simply removes the most problematic features without resolving the issue.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.6 Correction: eliminating low-frequency words
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“There is no implied directionality (e.g., Democratic vs. Republican), however, and the v 2 value itself does not have any obvious interpretation as an evaluation weight.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.4 Accounting for Variance
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our example, a stop list would eliminate a word like her, which almost definitely has political content in the context of abortion, 9 and a word like their, which might (e.g., women and their doctors).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.2.3 Correction: removing stop words
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“There are several approaches, but the notion in common is to put a strong conservative prior on the model.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5 Shrinkage and Regularization
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The problems we have seen to this point are typical of many contemporary data-rich problems, emerging not just from computational linguistics but from other similarly scaled or ''ill-posed'' machine and statistical learning problems in areas like image processing and gene expression.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5 Shrinkage and Regularization
Strength:plain· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variance is based on the absolute frequency of a word in all, or both, documents of interest, and its implied absolute frequency in the associated priors.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.3.4 Feature evaluation
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Third, the fact that these estimates could be recovered from appropriate standard Bayesian generalized linear models means the approach is easily extended beyond the setting of a strict partitioning of the document space into discrete groups.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
In the paper
“Rollcall-based work suggests that a single ideological dimension accounts for virtually all Congressional divisions (Poole 2005).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
What the sources say
- Not checked yetcites Poole 2005
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As with the above, the effect of any particular smoothing parameter will be affected by the relative volume of speech and the polarization of word use.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“This is important because of a general gender gap across the two parties both in core values (Rokeach 1973) and in framing (Lakoff 2004).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
What the sources say
- Not checked yetcites Rokeach 1973
- Not checked yetcites Lakoff 2004
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Variation in language use offers an alternative measure, one that can be differentiated across political topics and at a finer dynamic grain.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“It is well known that this corresponds to a uniform prior for log(c).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:boosted· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“On the theoretical issue of majority rule, where you stand depends on where you sit.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✗ Checkable: fails✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. It may be too vague to check against evidence. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can also extend our analysis across time to analyze dynamics across partisan (or extrapartisan) word use.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Strength:plain· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Republicans were pushing for elimination of the death tax, lowering of capital gains taxes, elimination of the marriage penalty, and flatter brackets; Democrats advocated for a tax credit for prescription drug coverage, for college, and against lowering the estate tax or giving breaks to the wealthiest.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Chong
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Not only can we look at the relative differences in language use between men and women, as we did with party, but we can also do so while controlling for party.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
- Rests on an assumption: Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“A higher smoothing parameter would result in even fewer nonzero features; a smaller one would result in more.19”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:hedged· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes? Load-bearing: unclear✓ Based: passesIt may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The nondifferentiability at the mode does considerably increase the difficulty and computational overhead associated with estimation.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.2 Laplace prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“One of the more prominent is the concern over whether women legislators behave differently than male legislators and whether they offer a fundamentally different form of representation by speaking ''in a different voice'' (Gilligan 1993;Williams 1998;Diaz 2005).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.2 Women's Representation
What the sources say
Strength:plain· not yet calibratedCausal:correlational· not yet calibratedTests✗ Asserted: fails✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The authors may not put it forward as true. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Second, this low computational overhead makes it relatively straightforward to develop more flexible variants, such as a dynamic version we discuss below.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
In the paper
“The primary measure of polarization at the elite level has come from rollcall analysis, following Poole and Rosenthal (1997).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.4 Polarization
What the sources say
- Not checked yetcites Poole and Rosenthal (1997)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This approach still suffers as a technique for feature selection: it is unclear where one draws the line between words that qualitatively matter and words that qualitatively do not.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 3.5.1 Informative Dirichlet prior
Load-bearingStrength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesProbably not a substantial claim.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In order to observe shifts in the tectonic plates of partisan conflict, we need both salience (the parties and presumably the public must care) and some exogenous shock or change in the status quo (to motivate change).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The analytic solution of the Dirichlet model makes dynamic extensions straightforward.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
Tests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. The paper may give no reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“A large-scale example clearly matching the criteria for a ''realignment'' or an ''issue evolution'' is the reversal of the parties on race and civil rights issues over the course of the 20th century; a small-scale example, which may or may not meet the criteria, is the polarization over time of the parties on the issue of abortion.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.3 Dynamics: Issue Evolutions and Semantic Realignment
What the sources say
- Not checked yetcites Carmines and Stimson 1990
- Not checked yetcites Poole and Rosenthal 1997
Strength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Maximization of the posterior densities in equations 28 and 29 is complicated by the discontinuities in the partial derivatives that are introduced by the Laplace prior.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Appendix-Estimation of the Laplace Model
Strength:plain· not yet calibratedCausal:direct· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These problems lead researchers to select and weight feature/word lists that have low semantic validity or vary wildly depending on arbitrary decisions.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 5 Discussion and Conclusion
Tests✓ Asserted: passes✗ Checkable: fails✓ Load-bearing: passes✗ Based: failsProbably not a substantial claim. It may be too vague to check against evidence. The paper may give no reason why it holds. It may state two separate claims as one.
- Rests on an assumption: More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.
None of the sources cited for this claim have been checked yet.
In the paper
“Using results from Cawley and Talbot (2006), it is easy to show how the value of b ðiÞ k that maximizes pðb ðiÞ k jy ðiÞ k Þ } Z p À y ðiÞ k jb ðiÞ k Þpðb ðiÞ k jcÞp J ðcÞdcðA1Þ can be calculated via a modified version of the iterative approach of Shevade and Keerthi (2003) in which, at each iteration, a working version, c, of c is defined to be:”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Marginalizing over c with a Jeffreys Prior
What the sources say
- Not checked yetcites Cawley and Talbot (2006)
- Not checked yetcites Shevade and Keerthi (2003)
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✓ Based: passesProbably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Note that when a 5 0, the results from this procedure are equivalent to those from the setup where c is given a Jeffreys prior and then integrated out of the posterior.”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > Marginalizing over c with an Exponential Prior
Strength:plain· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✗ Load-bearing: fails✗ Based: failsProbably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This is suggestive that there are measurable distributive (and therefore multidimensional) preference differences across Senators (which are unlikely to be present in rollcall votes once partisan coalitions have been built around particular bills).”
§ Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict > 4.5 Dimensionality
Load-bearingStrength:hedged· not yet calibratedTests✓ Asserted: passes✓ Checkable: passes✓ Load-bearing: passes✓ Based: passesIt may state two separate claims as one.
- Rests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.
The paper relies on this without support.
Different speech patterns across groups can flow from both partisan and nonpartisan sources, including idiosyncrasies of dialect and style.
“Additionally, we know a priori that different speech patterns across groups can flow from both partisan and nonpartisan sources, including idiosyncrasies of dialect and style.”
The authors state this as an assumption.
The approximations in Equations 17 and 19 assume y(i)_kw >> a(i)_kw, y_kw >> a_kw and ignore covariance terms that will typically be close to 0.
“Where the approximations in Equations 17 and 19 assume y ðiÞ kw ) a ðiÞ kw ; y kw ) a kw and ignore covariance terms that will typically be close to 0”
The authors state this as an assumption.
Equations 18 and 20 additionally assume that n(i)_k >> y(i)_kw and n_k >> y_kw.
“while Equations 18 and 20 additionally assume that n ðiÞ k ) y ðiÞ kw and n k ) y kw .”
The paper relies on this without support.
Sports speeches congratulating a home state team have no partisan structure.
“For example, our Senate analysis finds a large number of ''sports'' speeches congratulating a home state team.We would not expect these to have partisan structure.”
The paper relies on this without support.
Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.
“Democrats are both more likely than Republicans to address ''women's issues,'' and also have more women members serving in the Senate than do the Republicans.”
The paper relies on this without support.
The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.
“We conducted Lexis-Nexis searches of two newspapers, The New York Times and The Washington Times, and two broadcast news sources, National Public Radio and Fox News, widely thought to be, relatively speaking, at different partisan extremes.”
The paper relies on this without support.
A genuine partisan realignment or issue evolution generally occurs only over fairly long time frames.
“This generally occurred (or been agreed to have occurred) only over fairly long time frames.”
The paper relies on this without support.
Speech data offers leverage on analyzing partisan conflict.
“Speech offers interesting potential leverage on the problem.”
The paper relies on this without support.
Prior efforts to systematically analyze political conflict have focused on easily quantified behaviors that obscure substance.
“Efforts to systematically analyze political conflict, especially in the American context, have previously focused on behaviors that are easily quantified but obscure substance (e.g., votes).”
The paper relies on this without support.
Language use in political speech is a valid indicator of political conflict and preferences.
“We have seen that static snapshots can illuminate the content of a particular partisan conflict.”
The paper relies on this without support.
More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.
“More recent attempts to investigating political dynamics that summarize and organize text have fallen into other traps, including inefficiency (e.g., classification techniques), and overfitting idiosyncratic group differences in low-or high-frequency words (e.g., tf.idf or difference of proportions).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source says what the claim attributes to it.
In the citing paper
“Monroe, Colaresi, and Quinn (2008) describe a variety of methods that measure how well words separate already identified categories of interest”
From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Monroe, Colaresi, and Quinn (2008)
The cited source says what the claim attributes to it.
In the citing paper
“Although there have been efforts to understand differential word rate of use within topics pertaining to these terms (e.g., Monroe, Colaresi, and Quinn 2008), there has been relatively little work on whether the same words appear in different contexts.”
From Embedding Regression: Models for Context-Specific Description and Inference (2023) · cites this paper as Monroe, Colaresi, and Quinn 2008
In the citing paper
“For example, Monroe et al. explored a variety of techniques that compare two frequencies of words – one number for each of two groups [44]. In particular, they explored frequencies across democratic versus republican speeches and settled on a Bayesian model with regularization and shrinkage based on priors of word use.”
From Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach (2013) · cites this paper as [44]
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
The Nature of Statistical Learning Theory
Vapnik, V. (2000) · Statistics for Engineering and Information Science · doi:10.1007/978-1-4757-3264-1 · Cited by 46,387
The Nature of Statistical Learning Theory
Vapnik, V. N. (1995) · doi:10.1007/978-1-4757-2440-0 · Cited by 39,138
Content Analysis: An Introduction to its Methodology.
Shelley, M., & Krippendorff, K. (1984) · Journal of the American Statistical Association · doi:10.2307/2288384 · Cited by 24,346
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162
A decision-theoretic generalization of on-line learning and an application to boosting
Freund, Y., & Schapire, R. E. (1997) · Journal of Computer and System Sciences · doi:10.1006/jcss.1997.1504 · Cited by 20,466
Content Analysis: An Introduction to Its Methodology
Krippendorff, K. (2019) · doi:10.4135/9781071878781 · Cited by 15,448
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
(1973) · DigitalGeorgetown (Georgetown University Library) · Cited by 10,144
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
In a Different Voice: Psychological Theory and Women’s Development
GILLIGAN, C. (1993) · doi:10.4159/9780674037618 · Cited by 9,127
Agresti, A. (2002) · Wiley Series in Probability and Statistics · doi:10.1002/0471249688 · Cited by 6,746
Speech and language processing
Politt, R., Pollock, J., & Waller, E. (2010) · Taylor & Francis eBooks · doi:10.4324/9780203461891_chapter_3 · Cited by 3,594
Chong, D., & Druckman, J. N. (2007) · Annual Review of Political Science · doi:10.1146/annurev.polisci.10.072805.103054 · Cited by 3,071
The Theory of Committees and Elections.
Alan, S., & Black, D. (1959) · Economica · doi:10.2307/2550498 · Cited by 2,829
Congress: A Political-Economic History of Roll Call Voting
Poole, K. T., & Rosenthal, H. L. (2000) · Medical Entomology and Zoology · Cited by 2,798
Polarized America: the dance of ideology and unequal riches
(2007) · Choice Reviews Online · doi:10.5860/choice.44-3551 · Cited by 1,793
An information-theoretic perspective of tf–idf measures
Aizawa, A. (2003) · Information Processing & Management · doi:10.1016/s0306-4573(02)00021-3 · Cited by 1,419
Extracting Policy Positions from Political Texts Using Words as Data
LAVER, M., BENOIT, K., & GARRY, J. (2003) · American Political Science Review · doi:10.1017/s0003055403000698 · Cited by 1,377
Framing Public Opinion in Competitive Democracies
Chong, D., & Druckman, J. N. (2007) · American Political Science Review · doi:10.1017/s0003055407070554 · Cited by 1,127
Cox, G. W., & McCubbins, M. D. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511791123 · Cited by 1,126
Setting the agenda: responsible party government in the U.S. House of Representatives
(2006) · Choice Reviews Online · doi:10.5860/choice.43-4315 · Cited by 915
The Art of Political Manipulation, by William H. Riker
Garth, D. L. (1987) · Political Science Quarterly · doi:10.2307/2151420 · Cited by 894
Classifier Technology and the Illusion of Progress
Hand, D. J. (2006) · Statistical Science · doi:10.1214/088342306000000060 · Cited by 723
A Scaling Model for Estimating Time‐Series Party Positions from Texts
Slapin, J. B., & Proksch, S. (2008) · American Journal of Political Science · doi:10.1111/j.1540-5907.2008.00338.x · Cited by 684
How to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671
Self-citationParty Polarization in Congress
Theriault, S. M. (2008) · Cambridge University Press eBooks · doi:10.1017/cbo9780511790652 · Cited by 667
Spatial Models of Parliamentary Voting
Poole, K. T. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511614644 · Cited by 629
What Drives Media Slant? Evidence from U.S. Daily Newspapers
Gentzkow, M. A., & Shapiro, J. M. (2006) · National Bureau of Economic Research · doi:10.3386/w12707 · Cited by 579
Abortion: Evidence of an Issue Evolution
Adams, G. D. (1997) · American Journal of Political Science · doi:10.2307/2111673 · Cited by 549
Voice, Trust, and Memory: Marginalized Groups and the Failings of Liberal Representation
Williams, M. S. (1998) · doi:10.1515/9781400822782 · Cited by 533
A simple and efficient algorithm for gene selection using sparse logistic regression
Shevade, S., & Keerthi, S. S. (2003) · Bioinformatics · doi:10.1093/bioinformatics/btg308 · Cited by 411
Bayesian Regularization and Pruning Using a Laplace Prior
Williams, P. M. (1995) · Neural Computation · doi:10.1162/neco.1995.7.1.117 · Cited by 377
Voice, Trust, and Memory: Marginalized Groups and the Failings of Liberal Representation
Paxton, P. M., & Williams, M. S. (2001) · Contemporary Sociology A Journal of Reviews · doi:10.2307/3089779 · Cited by 355
The decline of the death penalty and the discovery of innocence
(2008) · Choice Reviews Online · doi:10.5860/choice.45-7050 · Cited by 320
Cawley, G. C., & Talbot, N. L. C. (2006) · Bioinformatics · doi:10.1093/bioinformatics/btl386 · Cited by 243
The Decline of the Death Penalty and the Discovery of Innocence
Baumgartner, F. R., De Boef, S. L., & Boydstun, A. E. (2001) · Cambridge University Press · doi:10.1017/cbo9780511790638 · Cited by 242 · cited 1×
“The result of this initial stage is something like the 65-argument coding scheme developed by Baumgartner et al. (2008) to exhaustively capture the proand antiframes used in debate over the death penality (Baumgartner et al. 2008, 103–12, 243–51).”
BackgroundThe Theory of Committees and Elections
Holmer, F., & Black, D. (1959) · The Western Political Quarterly · doi:10.2307/444007 · Cited by 230
Party wars: polarization and the politics of national policy making
(2006) · Choice Reviews Online · doi:10.5860/choice.44-2391 · Cited by 207
PARTY WARS: POLARIZATION AND THE POLITICS OF NATIONAL POLICY MAKING
Taylor, C. (2006) · Oklahoma politics · Cited by 192
A probabilistic justification for using tf×idf term weighting in information retrieval
Hiemstra, D. (2000) · International Journal on Digital Libraries · doi:10.1007/s007999900025 · Cited by 186
Language and Ideology in Congress
Diermeier, D. A., Godbout, J., Yu, B., & Kaufmann, S. C. (2011) · British Journal of Political Science · doi:10.1017/s0007123411000160 · Cited by 176
McCaffrey, D., & Keys, J. L. (2000) · Sociological Quarterly · doi:10.1111/j.1533-8525.2000.tb02365.x · Cited by 170
Lowe, W. (2008) · Political Analysis · doi:10.1093/pan/mpn004 · Cited by 166
Classifying Party Affiliation from Political Speech
Yu, B., Kaufmann, S., & Diermeier, D. A. (2008) · Journal of Information Technology & Politics · doi:10.1080/19331680802149608 · Cited by 143
Computer-Assisted Topic Classification for Mixed-Methods Social Science Research
Hillard, D., Purpura, S., & Wilkerson, J. (2008) · Journal of Information Technology & Politics · doi:10.1080/19331680801975367 · Cited by 138
Regulation, redistribution, and public choice
Aranson, P. H., & Ordeshook, P. C. (1981) · Public Choice · doi:10.1007/bf00124233 · Cited by 47
The Congressional Debate on Partial-Birth Abortion: Constitutional Gravitas and Moral Passion
Schonhardt-Bailey, C. (2008) · British Journal of Political Science · doi:10.1017/s0007123408000203 · Cited by 43
Extracting Systematic Social Science Meaning from Text 1
Hopkins, D. J., & King, G. (2007) · Cited by 24
Cowboy nation and American character
Kleinfeld, J., & Kleinfeld, A. (2004) · Society · doi:10.1007/bf02690182 · Cited by 5
BOHM, R. M. (2009) · International Journal of Public Opinion Research · doi:10.1093/ijpor/edp051 · Cited by 4
CAMPAIGNING IN POETRY: IS THERE INFORMATION CONVEYED IN THE CANDIDATES' CHOICE OF WORDS? *
Sacerdote, B., & Zidar, O. M. (2008) · Cited by 1
Flagged first, then by citations
Research works printed but not linked: 9
- Diaz, 2005, Representing women? Female legislators in West European parliaments
- Carmines, 1990, Issue evolution
- The counts of estate, prescription, wage, wealthiest, death, bracket, flat, penalty, respectively, are (2078, 620, 1159, 208, 1280, 164, 431, 743) for The New York Times, (1032, 376, 692, 84, 731, 189, 399, 720) for The Washington Times, (172, 214, 224, 68, 209, 47, 68, 174) for National Public Radio, and (167, 362, 238, 54, 450, 55, 197, 281) for Fox News.
- Monroe, 2004, Society for Political Methodology
- Mucciaroni, 2006, Deliberative choice: Debating public policy in congress
- Riker, 1986, The art of political manipulation
- Emily, 2004, From boys to men
- Lakoff, 2004, Don't think of an elephant! Know your values and frame the debate
- Monroe, 2006, United States congressional speech corpus
Cited by
Sorted by citations
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297
“…than a count (Pang, Lee, and Vaithyanathan 2002; Hopkins and King 2010); (2) including some commonly used stopwords such as gendered pronouns (Monroe, Colaresi, and Quinn 2008); (3) a subset of features (either by automated feature selection or a lower dimensional projection) (Hofmann 1999);…”
BackgroundInfluentialPersonality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., Shah, A., Kosinski, M., Stillwell, D., Seligman, M. E. P., et al. (2013) · PLoS ONE · doi:10.1371/journal.pone.0073791 · Cited by 1,700
GoEmotions: A Dataset of Fine-Grained Emotions
Demszky, D., Movshovitz-Attias, D., Ko, J., Cowen, A., Nemade, G., & Ravi, S. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.372 · Cited by 873
Computational Grounded Theory: A Methodological Framework
Nelson, L. K. (2017) · Sociological Methods & Research · doi:10.1177/0049124117729703 · Cited by 794
“More complicated lexical-based techniques include word scores that aim to identify important or distinctive words (Monroe et al. 2008), and relational semantic network techniques, such as mapping networks of words that occur near one another (Lee and Martin 2015).”
BackgroundHow to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671
Self-citation
Metadata
- Authors
- Monroe, B. L., Colaresi, M. P., & Quinn, K. M.
- Affiliations
- Pennsylvania State University (US)
- Michigan State University (US)
- Harvard University (US)
- Corresponding author
- Monroe, Burt L.
- Colaresi, Michael P.
- Quinn, Kevin M.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2008
- Volume
- 16
- Issue
- 4
- Pages
- 372-403
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2038411619
- Semantic Scholar
- ebc37575aa4e4afeb1dc94a18022cc0ebeb2fe09
- Cited by
- 542
- References
- 54
- Citations by year
- 2026: 44 · 2025: 41 · 2024: 51 · 2023: 52 · 2022: 28 · 2021: 69 · 2020: 38 · 2019: 38 · 2018: 32 · 2017: 30 · 2016: 29 · 2015: 16 · 2014: 16 · 2013: 20 · 2012: 13
- FWCI
- 42.88
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- public-domain
- www.cambridge.org/core/terms (unspecified, 3291 d)
- Locations
- Political Analysis (publishedVersion, public-domain)
- RePEc: Research Papers in Economics (submittedVersion)
- Funders
- National Science Foundation
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Social Sciences › Political Science and International Relations › Electoral Systems and Political Participation
- Keywords
- political conflict, Bayesian shrinkage, regularization, text-as-data, distributive politics, United States Senate, ideological space
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
