The DOI matches its Crossref record; 42 of the 42 references it cites were found in a registry or an index.
Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It
Open accessAbstract
Despite the popularity of unsupervised techniques for political science text-as-data research, the importance and implications of preprocessing decisions in this domain have received scant systematic attention. Yet, as we show, such decisions have profound effects on the results of real models for real data. We argue that substantive theory is typically too vague to be of use for feature selection, and that the supervised literature is not necessarily a helpful source of advice. To aid researchers working in unsupervised settings, we introduce a statistical procedure and software that examines the sensitivity of findings under alternate preprocessing regimes. This approach complements a researcher’s substantive understanding of a problem by providing a characterization of the variability changes in preprocessing choices may induce when analyzing a particular dataset. In making scholars aware of the degree to which their results are likely to be sensitive to their preprocessing decisions, it aids replication efforts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
An Introduction to Statistical Learning
James, G., Witten, D., Hastie, T., & Tibshirani, R. (2013) · Springer Texts in Statistics · doi:10.1007/978-1-4614-7138-7 · Cited by 12,106
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Foundations of statistical natural language processing
Manning, C. D., & Schütze, H. (1999) · Cited by 9,816
An algorithm for suffix stripping
Porter, M. F. (1980) · Program · doi:10.1108/eb046814 · Cited by 8,096 · cited 1×
“Stemming refers to the process of reducing a word to its most basic form (Porter, 1980).”
BackgroundMachine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,926
Jurafsky, D. S., & Martin, J. (2000) · Cited by 4,132
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · cited 3×
“A commonly used rule of thumb is to discard terms that appear in less than 0.5-1% of documents (Grimmer, 2010; Yano, Smith and Wilkerson, 2012; Grimmer and Stewart, 2013), however, there has been no systematic study of the effects this preprocessing choice has on downstream analyses.”
BackgroundMethodInfluential
Cited by
Gentzkow, M., Kelly, B., & Taddy, M. (2019) · Journal of Economic Literature · doi:10.1257/jel.20181020 · Cited by 1,236
Smart literature review: a practical topic modelling approach to exploratory literature review
Asmussen, C. B., & Møller, C. (2019) · Journal of Big Data · doi:10.1186/s40537-019-0255-7 · Cited by 455
Rhetoric and frame analysis of ExxonMobil's climate change communications
Supran, G., & Oreskes, N. (2021) · One Earth · doi:10.1016/j.oneear.2021.04.014 · Cited by 374
Machine Learning for Social Science: An Agnostic Approach
Grimmer, J., Roberts, M. E., & Stewart, B. M. (2021) · Annual Review of Political Science · doi:10.1146/annurev-polisci-053119-015921 · Cited by 323
Large-Scale Computerized Text Analysis in Political Science: Opportunities and Challenges
Wilkerson, J., & Casas, A. (2017) · Annual Review of Political Science · doi:10.1146/annurev-polisci-052615-025542 · Cited by 300
Metadata
- Authors
- Denny, M. J., & Spirling, A.
- Venue
- Political Analysis
- Publisher
- Cambridge University Press (CUP)
- Published
- 2018-03-19
- Volume
- 26
- Issue
- 2
- Pages
- 168-189
- ISSN
- 1047-1987 (ISSN-L)
- 1476-4989
- OpenAlex
- W3125733373
- Cited by
- 463
- References
- 42
- FWCI
- 158.38
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It. https://app.cochat.ai/papers/10.1017/pan.2017.44
- The paper
- Denny, M. J., & Spirling, A. (2018). Text Preprocessing For Unsupervised Learning: Why It Matters, When It Misleads, And What To Do About It. Political Analysis. https://doi.org/10.1017/pan.2017.44
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.