Classification Accuracy as a Substantive Quantity of Interest: Measuring Polarization in Westminster Systems
Open accessThe DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Abstract
Measuring the polarization of legislators and parties is a key step in understanding how politics develops over time. But in parliamentary systems—where ideological positions estimated from roll calls may not be informative—producing valid estimates is extremely challenging. We suggest a new measurement strategy that makes innovative use of the “accuracy” of machine classifiers, i.e., the number of correct predictions made as a proportion of all predictions. In our case, the “labels” are the party identifications of the members of parliament, predicted from their speeches along with some information on debate subjects. Intuitively, when the learner is able to discriminate members in the two main Westminster parties well, we claim we are in a period of “high” polarization. By contrast, when the classifier has low accuracy—and makes a relatively large number of mistakes in terms of allocating members to parties based on the data—we argue parliament is in an era of “low” polarization. This approach is fast and substantively valid, and we demonstrate its merits with simulations, and by comparing the estimates from 78 years of House of Commons speeches with qualitative and quantitative historical accounts of the same. As a headline finding, we note that contemporary British politics is approximately as polarized as it was in the mid-1960s—that is, in the middle of the “postwar consensus”. More broadly, we show that the technical performance of supervised learning algorithms can be directly informative about substantive matters in social science.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Computation and analysis of multiple structural change models
Bai, J., & Perron, P. (2003) · Journal of Applied Econometrics · doi:10.1002/jae.659 · Cited by 5,771
A Method of Automated Nonparametric Content Analysis for Social Science
Hopkins, D. J., & King, G. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00428.x · Cited by 765
A Scaling Model for Estimating Time‐Series Party Positions from Texts
Slapin, J. B., & Proksch, S. (2008) · American Journal of Political Science · doi:10.1111/j.1540-5907.2008.00338.x · Cited by 684
Causes and Consequences of Polarization
Barber, M. J., & McCarty, N. (2015) · Solutions to Political Polarization in America · doi:10.1017/cbo9781316091906.002 · Cited by 375 · cited 2×
“To define terms explicitly: in keeping with the Americanist literature (e.g. Barber and McCarty 2015), we understand “polarization” to mean the (average) difference between the positions of Authors’ note: We are grateful to Niels Goet, Justin Grimmer and Ben Lauderdale for comments on an earlier…”
BackgroundGentzkow, M., Shapiro, J., & Taddy, M. (2016) · doi:10.3386/w22423 · Cited by 159
UK OC OK? Interpreting Optimal Classification Scores for the U.K. House of Commons
Spirling, A., & McLean, I. (2007) · Political Analysis · doi:10.1093/pan/mpl009 · Cited by 94 · cited 1×
“…literature (e.g. Barber and McCarty 2015), we cannot generally use roll calls to infer relative partisan difference because (a) parties tend to vote extremely cohesively in the UK and (b) even when they do not, it can be difficult to interpret deviations substantively (Spirling and McLean 2007).”
BackgroundSelf-citation
Cited by
Gentzkow, M., Shapiro, J. M., & Taddy, M. (2019) · Econometrica · doi:10.3982/ecta16566 · Cited by 419
Gentzkow, M., Shapiro, J., & Taddy, M. (2016) · doi:10.3386/w22423 · Cited by 159
Playing to the Gallery: Emotive Rhetoric in Parliaments
OSNABRÜGGE, M., HOBOLT, S. B., & RODON, T. (2021) · American Political Science Review · doi:10.1017/s0003055421000356 · Cited by 118
Myrick, R. (2021) · International Organization · doi:10.1017/s0020818321000175 · Cited by 91
“Peterson and Spirling 2018.”
Metadata
- Authors
- Peterson, A., & Spirling, A.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2018-01
- Volume
- 26
- Issue
- 1
- Pages
- 120-128
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W2753354158
- Cited by
- 82
- References
- 6
- FWCI
- 34.07
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Classification Accuracy as a Substantive Quantity of Interest: Measuring Polarization in Westminster Systems. https://app.cochat.ai/papers/10.1017/pan.2017.39
- The paper
- Peterson, A., & Spirling, A. (2018). Classification Accuracy as a Substantive Quantity of Interest: Measuring Polarization in Westminster Systems. Political Analysis. https://doi.org/10.1017/pan.2017.39
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.