Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict

May combine two claims.

The measure with equivalent prior has the same problems, with the dominant words still the same list of obscurities.

“This has the same problems, with the dominant words still the same list of obscurities.”

May combine two claims.

In the abortion speeches, with infinite values given infinite weight, the most Democratic words are 'bankruptci,' 'Snow[e],' 'ratifi,' 'confidenti,' and 'church,' and the most Republican words are 'infant,' 'admit,' 'Chines,' 'industri,' and '40.'

“The most Democratic words are then bankruptci, Snow½e, ratifi, confidenti, and church, and the most Republican words are infant, admit, Chines, industri, and 40.”

May combine two claims.

Although function words no longer dominate the lists as with several previous techniques, some still appear too prominent in their implied partisanship, among them the, of, be, not, and my.

“Although function words no longer dominate the lists as with several previous techniques, some still appear to be too prominent in their implied partisanship, among them the, of, be, not, and my.”

Probably not a substantial claim: may not pass S3 (load-bearing).

In the running example, the WordScores implicit evaluation measure correlates with difference of proportions at over 0.998.

“In this example, they correlate at over 10.998.”

May combine two claims.

In the abortion speeches, the top Democratic word list by difference of proportions is dominated by 'to' and includes 'my,' 'and,' and 'for'; the top Republican word list is dominated by 'the' and includes 'of,' 'not,' 'be,' 'that,' 'you,' 'it,' and 'a.'

“The top Democratic word list is dominated by to and includes my, and, and for; the top Republican word list is dominated by the and includes of, not, be, that, you, it, and a.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Figure 4 shows feature weightings based on f(D-R) for the running example.

“Figure 4 shows the feature weightings based on the fðD 2 RÞ kw for our running example.”

May combine two claims.

In the abortion speeches, if words with zero counts in one party are thrown out, the most Democratic words are 'treati,' 'discrim,' 'abroad,' 'domest,' and 'privacy,' and the most Republican words are 'perfect,' 'subsid,' 'percent,' 'overrid,' and 'cell.'

“If we instead throw out the words with zero counts in one party, the most Democratic words are treati, discrim, abroad, domest, and privacy, and the most Republican words are perfect, subsid, percent, overrid, and cell.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Slapin and Proksch (2008) provide a plot demonstrating heteroskedasticity; their Figure 2 looks like Figure 2 turned on its side, and their Table 1 contains many obscurities.

“One example is given by Slapin and Proksch (2008), made more striking because they provide a plot that clearly demonstrates the heteroskedasticity.Their Figure 2 looks like Figure 2 turned on its side.The resulting lists of their Table 1 contain many obscurities.”

• cites Slapin and Proksch (2008)Not checked yet

  Not checked in the free run.

The quote only partly states this claim.

The weights from the unlogged tf.idf variant are correlated at 0.997 with those from raw difference of proportions.

“The weights are correlated (at 10.997 in this case)”

May combine two claims.

In the abortion speeches, the odds of a Republican using a variant of 'baby' are 5.4 times those of a Democrat, while the odds of a Republican using 'April' are 7.4 times those of a Democrat.

“In our data, the odds of a Republican using a variant of babi are 5.4 times those of a Democratic when talking about abortion, which seems informative.But, the odds of a Republican using the word April are 7.4 times those of a Democrat when talking about abortion, which seems less so.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The odds of The New York Times using Democratic over Republican frames are roughly 20% higher than those of The Washington Times.

“Sticking to the familiar metric of odds ratios, the odds of The New York Times using these Democratic over Republican frames are roughly 20% higher than those of the Washington Times.”

• AssumptionRests on an assumption: The New York Times and Washington Times, and NPR and Fox News, are at different partisan extremes.

May combine two claims.

We find several instances where Republicans own an issue or frame while in opposition to Clinton and are successful in obtaining a policy change during the Bush honeymoon period of early 2001 or in the rally period after 9/11, only to find Democrats owning the issue when the policy becomes unpopular; dramatic examples include debt (budget), oil (energy), and Iraq (defense).

“For example, we find several instances where Republicans ''own'' an issue or frame while in opposition to Clinton are successful in obtaining a policy change during the Bush honeymoon period of early 2001 or in the rally period after 9/11, only to find Democrats ''owning'' the issue when the policy becomes unpopular.”

• Also stated in: § 4.3 Dynamics: Issue Evolutions and Semantic Realignment · plain

  “Dramatic examples include debt when discussing the budget, oil when discussing energy policy, and Iraq when discussing defense policy.”

• AssumptionRests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.

May combine two claims. The quote only partly states this claim.

During the Clinton era, Republicans consistently blocked judicial nominees and Democrats demanded votes with increasing urgency; during the Bush era, Democrats consistently filibustered judicial nominees and Republicans supported presidential deference and up-or-down votes.

“During the Bush era, Democrats consistently filibustered his judicial nominees, and Republicans discovered a deeply held belief in presidential deference and up-or-down votes.”

• AssumptionRests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.

May combine two claims.

Iraq supplies both salience and exogenous shock, and we observe relative nonpartisanship pre-9/11 shift to a Republican talking point post-9/11, only to see events on the ground and the cost of the war increase Democratic leverage on the issue.

“Iraq supplies both of these, and in turn, we see relative nonpartisanship pre-9/11 shift to a Republican talking point post-9/11 only to see events on the ground and the cost of the war increase Democratic leverage on the issue.”

• AssumptionRests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.

May combine two claims.

There is clear variation in the intensity of preferences, with great intensity on multiple issues from the Alaska delegation, but limited intensity from the Alabama delegation.

“There is also clear variation in the intensity of preferences, with great intensity on multiple issues from the Alaska delegation, but limited intensity from the Alabama delegation.”

• AssumptionRests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.

Probably not a substantial claim: may not pass S3 (load-bearing).

In the running example with informative Dirichlet prior, the Republican top 20 list has shuffled, with the, you, not, of, and be being replaced by aliv, infant, brutal, brain, and necessari.

“For example, the Republican top 20 list has shuffled, with the, you, not, of, and be being replaced by the considerably more evocative aliv, infant, brutal, brain, and necessari.”

May combine two claims.

The prior used in the running example has almost no effect on estimates in topics with much more speech and strong partisan differences, and overwhelms estimates in topics with very little speech or no partisan differences.

“The prior used in this example has almost no effect on estimates in topics with much more speech and strong partisan differences (like defense, post-Iraq), and overwhelms estimates-as it should-in topics with very little speech or no partisan differences (like symbolic constituent tributes).”

May combine two claims.

Defense polarization reached near zero in the aftermath of 9/11 and dramatically increased through and after the invasion of Iraq.

“Defense is particularly interesting, reaching near zero polarization in the aftermath of 9/11 and dramatically increasing through and after the invasion of Iraq.”

• AssumptionRests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

Pooling all speech together has several undesirable consequences and topic modeling is a crucial prior step.

“For our particular substantive and theoretical interests, pooling all speech together has several undesirable consequences and topic modeling is a crucial prior step.”

Classified as probably not a substantial claim. May combine two claims.

The proposed solution relies on a model-based approach to avoid inefficiency and shrinkage and regularization to avoid both infinite estimates and overfitting the sample data.

“Our proposed solution relies on a model-based approach to avoid inefficiency and shrinkage and regularization to avoid both infinite estimates and overfitting the sample data.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The chi^2 approach was used by Gentzkow and Shapiro (2006).

“This was the approach used by Gentzkow and Shapiro (2006).”

• cites Gentzkow and Shapiro (2006)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Due to conjugacy, the full Bayesian estimate using the Dirichlet prior is analytically available.

“Due to the conjugacy, the full Bayesian estimate using the Dirichlet prior is also analytically available in analogous form:”

May combine two claims.

Every word in the vocabulary receives nonzero weight under the f(D-R) measure, which could lead to overfitting if these estimates are used in some subsequent model.

“A related problem is that, although there is variation, every word in the vocabulary receives nonzero weight, which could lead to overfitting if these estimates are used in some subsequent model.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

tf.idf is recommended in standard textbooks and is widely used in document search and information retrieval tasks.

“tf.idf is recommended in standard textbooks (Jurafsky and Martin (2000, 651-4) (Manning and Schu ¨tze 1999, 541-4) and is widely used in document search and information retrieval tasks.14”

• cites Jurafsky and Martin (2000, 651-4)Not checked yet

  Not checked in the free run.

• cites Manning and Schu ¨tze 1999, 541-4Not checked yet

  Not checked in the free run.

May combine two claims.

Eliminating function words inappropriately eliminates words like 'her' and inappropriately elevates high-frequency non-stop words.

“Eliminating function words not only eliminates words like her inappropriately but it also elevates high-frequency non-stop words inappropriately.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Perhaps the most prominent text-as-data approach in political science is the WordScores procedure, which embeds a feature evaluation technique in its algorithm.

“Perhaps the most prominent text-as-data approach in political science is the WordScores procedure (Laver et al. 2003), which embeds a feature evaluation technique in its algorithm.”

• cites Laver et al. 2003Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

With the prior, the log-odds-ratio between two groups is a relabeling of the smoothed log-odds-ratio discussed before.

“With the prior, this is a relabeling of the smoothed log-odds-ratio discussed before.”

May combine two claims.

The log-odds-ratio word lists do not seem like words that most fundamentally define the partisan division on abortion; plausible partisan words are overwhelmed by oddities.

“These do not seem like words that most fundamentally define the partisan division on abortion.Words with plausible partisan content on abortion (infant, church) are overwhelmed by oddities that require quite a bit more investigation to interpret (Chines, bankruptci).”

Probably not a substantial claim: may not pass S3 (load-bearing).

The mistake of eliminating low-frequency words is common in emergent text-as-data discussions in political science.

“This mistake is also common in many of the emergent text-as-data discussions in political science.”

• cites Slapin and Proksch (2008)Not checked yet

  Not checked in the free run.

May combine two claims.

With naive log-odds-ratio, words spoken by only one party have infinite odds ratios, and if infinite values are given infinite weight, the partisan word list consists only of words spoken by a single party.

“Working naively, it is unclear what we are to do with words that are spoken by only one party and therefore have infinite odds ratios.If we let the infinite values have infinite weight, the partisan word list consists of only those spoken by a single party.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Without the prior, the log-odds-ratio between two groups is simply the observed log-odds-ratio.

“Without the prior, this is of course simply the observed log-odds-ratio.”

May combine two claims.

Republican word choice reflects framing the debate from the point of view of the baby/child and emphasizing the details of the partial birth abortion procedure.

“Republican word choice reflects framing the debate from the point of view of the baby/child and emphasizing the details of the partial birth abortion procedure.”

Probably not a substantial claim: may not pass S3 (load-bearing).

If proceeding directly to ML estimation, the lack of covariates results in an immediately available analytical solution for the MLE of b.

“If we wish to proceed directly to ML estimation, the lack of covariates results in an immediately available analytical solution for the MLE of b ðiÞ kw .”

Probably not a substantial claim: may not pass S3 (load-bearing).

The logic of tf.idf is that the words containing the greatest information about a particular document are the words that appear many times in that document, but in relatively few others.

“The logic of tf.idf is that the words containing the greatest information about a particular document are the words that appear many times in that document, but in relatively few others.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

Use of any particular Dirichlet prior defined by a affects the posterior exactly as if we had observed in the data an additional a_w -1 instances of word w.

“That is, use of any particular Dirichlet prior defined by a affects the posterior exactly as if we had observed in the data an additional a w -1 instances of word w.”

May combine two claims.

The log-odds-ratio measure fails to account for sampling variability; its sampling variation goes down with increased frequency, so it will be inappropriately dominated by obscure words.

“The problem is again the failure to account for sampling variability.With log-oddsratios, the sampling variation goes down with increased frequency, as is clear in Figure 2. So, this measure will be inappropriately dominated by obscure words.”

May combine two claims.

WordScores demonstrates the same failure to account for sampling variation and the same overweighting of high-frequency words.

“So, WordScores demonstrates the same failure to account for sampling variation and the same overweighting of high-frequency words.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Setting a frequency threshold for features generally removes the most problematic features without resolving the issue.

“Generally, this simply removes the most problematic features without resolving the issue.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The chi^2 value itself does not have any obvious interpretation as an evaluation weight.

“There is no implied directionality (e.g., Democratic vs. Republican), however, and the v 2 value itself does not have any obvious interpretation as an evaluation weight.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In the abortion context, a stop list would eliminate 'her,' which almost definitely has political content, and 'their,' which might.

“In our example, a stop list would eliminate a word like her, which almost definitely has political content in the context of abortion, 9 and a word like their, which might (e.g., women and their doctors).”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

In regularization and Bayesian shrinkage, the notion in common is to put a strong conservative prior on the model.

“There are several approaches, but the notion in common is to put a strong conservative prior on the model.”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).

The problems seen to this point are typical of many contemporary data-rich problems, emerging not just from computational linguistics but from other similarly scaled or ill-posed machine and statistical learning problems in areas like image processing and gene expression.

“The problems we have seen to this point are typical of many contemporary data-rich problems, emerging not just from computational linguistics but from other similarly scaled or ''ill-posed'' machine and statistical learning problems in areas like image processing and gene expression.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Variance is based on the absolute frequency of a word in all, or both, documents of interest, and its implied absolute frequency in the associated priors.

“Variance is based on the absolute frequency of a word in all, or both, documents of interest, and its implied absolute frequency in the associated priors.”

May combine two claims.

The informative Dirichlet estimates could be recovered from appropriate standard Bayesian generalized linear models, so the approach is easily extended beyond a strict partitioning of the document space into discrete groups.

“Third, the fact that these estimates could be recovered from appropriate standard Bayesian generalized linear models means the approach is easily extended beyond the setting of a strict partitioning of the document space into discrete groups.”

Classified as probably not a substantial claim.

Rollcall-based work suggests that a single ideological dimension accounts for virtually all Congressional divisions (Poole 2005).

“Rollcall-based work suggests that a single ideological dimension accounts for virtually all Congressional divisions (Poole 2005).”

• cites Poole 2005Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The effect of any particular smoothing parameter will be affected by the relative volume of speech and the polarization of word use.

“As with the above, the effect of any particular smoothing parameter will be affected by the relative volume of speech and the polarization of word use.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

A general gender gap across the two parties exists both in core values and in framing.

“This is important because of a general gender gap across the two parties both in core values (Rokeach 1973) and in framing (Lakoff 2004).”

• cites Rokeach 1973Not checked yet

  Not checked in the free run.

• cites Lakoff 2004Not checked yet

  Not checked in the free run.

May combine two claims.

Variation in language use offers an alternative measure of polarization, one that can be differentiated across political topics and at a finer dynamic grain.

“Variation in language use offers an alternative measure, one that can be differentiated across political topics and at a finer dynamic grain.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The Jeffreys prior corresponds to a uniform prior for log(c).

“It is well known that this corresponds to a uniform prior for log(c).”

Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).

On the theoretical issue of majority rule, where you stand depends on where you sit.

“On the theoretical issue of majority rule, where you stand depends on where you sit.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing), S4 (based).

The Dirichlet model can be extended across time to analyze dynamics in partisan (or extrapartisan) word use.

“We can also extend our analysis across time to analyze dynamics across partisan (or extrapartisan) word use.”

May combine two claims.

Republicans were pushing for elimination of the death tax, lowering of capital gains taxes, elimination of the marriage penalty, and flatter brackets; Democrats advocated for a tax credit for prescription drug coverage, for college, and against lowering the estate tax or giving breaks to the wealthiest.

“Republicans were pushing for elimination of the death tax, lowering of capital gains taxes, elimination of the marriage penalty, and flatter brackets; Democrats advocated for a tax credit for prescription drug coverage, for college, and against lowering the estate tax or giving breaks to the wealthiest.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The models can be used to look at relative differences in language use between men and women while controlling for party.

“Not only can we look at the relative differences in language use between men and women, as we did with party, but we can also do so while controlling for party.”

• AssumptionRests on an assumption: Democrats are more likely than Republicans to address women's issues and have more women members serving in the Senate than Republicans.

May combine two claims.

A higher smoothing parameter would result in even fewer nonzero features; a smaller one would result in more.

“A higher smoothing parameter would result in even fewer nonzero features; a smaller one would result in more.19”

Probably not a substantial claim: may not pass S3 (load-bearing).

The nondifferentiability at the mode of the Laplace prior considerably increases the difficulty and computational overhead associated with estimation.

“The nondifferentiability at the mode does considerably increase the difficulty and computational overhead associated with estimation.”

Probably not a substantial claim: may not pass S1 (asserted), S3 (load-bearing). May combine two claims.

One prominent concern is whether women legislators behave differently than male legislators and whether they offer a fundamentally different form of representation by speaking in a different voice.

“One of the more prominent is the concern over whether women legislators behave differently than male legislators and whether they offer a fundamentally different form of representation by speaking ''in a different voice'' (Gilligan 1993;Williams 1998;Diaz 2005).”

• cites Gilligan 1993 sourceNot checked yet

  Not checked in the free run.

• cites Williams 1998 sourceNot checked yet

  Not checked in the free run.

• cites Diaz 2005Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The low computational overhead of the informative Dirichlet approach makes it relatively straightforward to develop more flexible variants, such as a dynamic version.

“Second, this low computational overhead makes it relatively straightforward to develop more flexible variants, such as a dynamic version we discuss below.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The primary measure of polarization at the elite level has come from rollcall analysis, following Poole and Rosenthal (1997).

“The primary measure of polarization at the elite level has come from rollcall analysis, following Poole and Rosenthal (1997).”

• cites Poole and Rosenthal (1997)Not checked yet

  Not checked in the free run.

Classified as probably not a substantial claim.

The informative Dirichlet approach still suffers as a technique for feature selection: it is unclear where one draws the line between words that qualitatively matter and words that qualitatively do not.

“This approach still suffers as a technique for feature selection: it is unclear where one draws the line between words that qualitatively matter and words that qualitatively do not.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

In order to observe shifts in partisan conflict, both salience (the parties and presumably the public must care) and some exogenous shock or change in the status quo (to motivate change) are needed.

“In order to observe shifts in the tectonic plates of partisan conflict, we need both salience (the parties and presumably the public must care) and some exogenous shock or change in the status quo (to motivate change).”

Probably not a substantial claim: may not pass S4 (based).

The analytic solution of the Dirichlet model makes dynamic extensions straightforward.

“The analytic solution of the Dirichlet model makes dynamic extensions straightforward.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The reversal of the parties on race and civil rights issues over the 20th century is a large-scale example clearly matching the criteria for a realignment or issue evolution; the polarization over time of the parties on abortion is a small-scale example that may or may not meet the criteria.

“A large-scale example clearly matching the criteria for a ''realignment'' or an ''issue evolution'' is the reversal of the parties on race and civil rights issues over the course of the 20th century; a small-scale example, which may or may not meet the criteria, is the polarization over time of the parties on the issue of abortion.”

• cites Carmines and Stimson 1990Not checked yet

  Not checked in the free run.

• cites Poole and Rosenthal 1997Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Maximization of the posterior densities in equations 28 and 29 is complicated by the discontinuities in the partial derivatives that are introduced by the Laplace prior.

“Maximization of the posterior densities in equations 28 and 29 is complicated by the discontinuities in the partial derivatives that are introduced by the Laplace prior.”

Probably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.

These problems lead researchers to select and weight feature/word lists that have low semantic validity or vary wildly depending on arbitrary decisions.

“These problems lead researchers to select and weight feature/word lists that have low semantic validity or vary wildly depending on arbitrary decisions.”

• AssumptionRests on an assumption: More recent attempts to analyze political dynamics by summarizing text have fallen into traps including inefficiency and overfitting idiosyncratic group differences in low- or high-frequency words.

Probably not a substantial claim: may not pass S3 (load-bearing).

Using results from Cawley and Talbot (2006), the value of b ðiÞ k that maximizes pðb ðiÞ k jy ðiÞ k Þ can be calculated via a modified version of the iterative approach of Shevade and Keerthi (2003).

“Using results from Cawley and Talbot (2006), it is easy to show how the value of b ðiÞ k that maximizes pðb ðiÞ k jy ðiÞ k Þ } Z p À y ðiÞ k jb ðiÞ k Þpðb ðiÞ k jcÞp J ðcÞdcðA1Þ can be calculated via a modified version of the iterative approach of Shevade and Keerthi (2003) in which, at each iteration, a working version, c, of c is defined to be:”

• cites Cawley and Talbot (2006)Not checked yet

  Not checked in the free run.

• cites Shevade and Keerthi (2003)Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

When a = 0, the results from the procedure with an exponential prior are equivalent to those from the setup where c is given a Jeffreys prior and then integrated out of the posterior.

“Note that when a 5 0, the results from this procedure are equivalent to those from the setup where c is given a Jeffreys prior and then integrated out of the posterior.”

May combine two claims.

There are measurable distributive (and therefore multidimensional) preference differences across Senators, which are unlikely to be present in rollcall votes once partisan coalitions have been built around particular bills.

“This is suggestive that there are measurable distributive (and therefore multidimensional) preference differences across Senators (which are unlikely to be present in rollcall votes once partisan coalitions have been built around particular bills).”

• AssumptionRests on an assumption: Language use in political speech is a valid indicator of political conflict and preferences.