The DOI matches its Crossref record; 64 of the 65 references it cites were found in a registry or an index.
Large-Scale Computerized Text Analysis in Political Science: Opportunities and Challenges
Open accessAbstract
Text has always been an important data source in political science. What has changed in recent years is the feasibility of investigating large amounts of text quantitatively. The internet provides political scientists with more data than their mentors could have imagined, and the research community is providing accessible text analysis software packages, along with training and support. As a result, text-as-data research is becoming mainstream in political science. Scholars are tapping new data sources, they are employing more diverse methods, and they are becoming critical consumers of findings based on those methods. In this article, we first describe the four stages of a typical text-as-data project. We then review recent political science applications and explore one important methodological challenge—topic model instability—in greater detail.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Citations
Cites
Breiman, L. (2001) · Machine Learning · doi:10.1023/a:1010933404324 · Cited by 133,538
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
The Coding Manual for Qualitative Researchers
Saldaña, J. (2025) · SAGE Publications Ltd eBooks · doi:10.4135/9781036235611 · Cited by 19,193
Measures of the Amount of Ecologic Association Between Species
Dice, L. R. (1945) · Ecology · doi:10.2307/1932409 · Cited by 12,187
Needleman, S. B., & Wunsch, C. D. (1970) · Journal of Molecular Biology · doi:10.1016/0022-2836(70)90057-4 · Cited by 11,288
A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection
Kohavi, R. (1995) · Zenodo · doi:10.5281/zenodo.19712698 · Cited by 10,909
Identification of common molecular subsequences
Smith, T. F., & Waterman, M. S. (1981) · Journal of Molecular Biology · doi:10.1016/0022-2836(81)90087-5 · Cited by 9,962
Thumbs up?: sentiment classification using machine learning techniques
Pang, B., Lee, L., & Vaithyanathan, S. (2002) · Proceedings of the ACL-02 conference on Empirical methods in natural language processing - EMNLP '02 · doi:10.3115/1118693.1118704 · Cited by 7,024
Opinion Mining and Sentiment Analysis
Pang, B., & Lee, L. (2008) · now publishers, Inc. eBooks · doi:10.1561/9781601981516 · Cited by 6,957
Congress, The Electoral Connection
Ross, M. J., & Mayhew, D. R. (1975) · The Western Political Quarterly · doi:10.2307/447379 · Cited by 4,094
A survey of cross-validation procedures for model selectionPreprint
Arlot, S., & Célisse, A. (2009) · Cited by 3,627
A survey of cross-validation procedures for model selection
Arlot, S., & Celisse, A. (2010) · Statistics Surveys · doi:10.1214/09-ss054 · Cited by 3,522
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · cited 2×
“Validation is a critical component of every text as data project (Saldana 2009; Grimmer and Stewart 2013).”
BackgroundCongress: The Electoral Connection, by David R. Mayhew
Page, B. I. (1975) · Political Science Quarterly · doi:10.2307/2148248 · Cited by 3,141
Issue Ownership in Presidential Elections, with a 1980 Case Study
Petrocik, J. R. (1996) · American Journal of Political Science · doi:10.2307/2111797 · Cited by 2,921
Advances in Neural Information Processing Systems 20
Bonilla, E. V., Chai, K. M. A., & Williams, C. K. I. (2008) · neural information processing systems · Cited by 2,701
How Censorship in China Allows Government Criticism but Silences Collective Expression
KING, G., PAN, J., & ROBERTS, M. E. (2013) · American Political Science Review · doi:10.1017/s0003055413000014 · Cited by 2,415 · cited 1×
“King et al. (2013) use ReadMe to classify millions of social media posts for topic in a study of government censorship in China.”
BackgroundStructural Topic Models for Open‐Ended Survey Responses
Roberts, M. E., Stewart, B., Tingley, D. H., Lucas, C., Leder-Luis, J., Gadarian, S. K., Albertson, B. L., & Rand, D. G. (2014) · American Journal of Political Science · doi:10.1111/ajps.12103 · Cited by 2,103
(2009) · Cited by 1,900
Home Style: House Members in Their Districts, by Richard F. Fenno, Jr.
Leuthold, D. A. (1979) · Political Science Quarterly · doi:10.2307/2150167 · Cited by 1,602
Meme-tracking and the dynamics of the news cycle
Leskovec, J., Bäckström, L., & Kleinberg, J. M. (2009) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/1557019.1557077 · Cited by 1,567
Extracting Policy Positions from Political Texts Using Words as Data
LAVER, M., BENOIT, K., & GARRY, J. (2003) · American Political Science Review · doi:10.1017/s0003055403000698 · Cited by 1,377
Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data
Barberá, P. (2015) · Political Analysis · doi:10.1093/pan/mpu011 · Cited by 982 · cited 1×
“Barbera (2015) uses Twitter data and information about their followers to estimate the ideological positions of politicians, parties, and individual citizens.”
BackgroundGoogle Trends: A Web‐Based Tool for Real‐Time Surveillance of Disease Outbreaks
Carneiro, H. A., & Mylonakis, E. E. (2009) · Clinical Infectious Diseases · doi:10.1086/630200 · Cited by 815
The master algorithm: how the quest for the ultimate learning machine will remake our world
(2016) · Choice Reviews Online · doi:10.5860/choice.194685 · Cited by 810
A Method of Automated Nonparametric Content Analysis for Social Science
Hopkins, D. J., & King, G. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00428.x · Cited by 765
A Scaling Model for Estimating Time‐Series Party Positions from Texts
Slapin, J. B., & Proksch, S. (2008) · American Journal of Political Science · doi:10.1111/j.1540-5907.2008.00338.x · Cited by 684
How to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671
Corporate funding and ideological polarization about climate change
Farrell, J. (2015) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1509433112 · Cited by 606
Cerón, A., Curini, L., Iacus, S. M., & Porro, G. (2013) · New Media & Society · doi:10.1177/1461444813480466 · Cited by 509
Denny, M. J., & Spirling, A. (2017) · Political Analysis · doi:10.1017/pan.2017.44 · Cited by 486
Network Analysis and Political Science
Ward, M. D., Stovel, K. W., & Sacks, A. (2011) · Annual Review of Political Science · doi:10.1146/annurev.polisci.12.040907.115949 · Cited by 307
Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data
Benoit, K. R., Conway, D., Lauderdale, B. E., Laver, M., & Mikhaylov, S. (2016) · American Political Science Review · doi:10.1017/s0003055416000058 · Cited by 296
Text-mining the signals of climate change doubt
Boussalis, C., & Coan, T. (2015) · Global Environmental Change · doi:10.1016/j.gloenvcha.2015.12.001 · Cited by 283
Measuring Interest Group Influence Using Quantitative Text Analysis
Klüver, H. (2009) · European Union Politics · doi:10.1177/1465116509346782 · Cited by 238
Grimmer, J. (2013) · American Journal of Political Science · doi:10.1111/ajps.12000 · Cited by 203
Text Analysis with R for Students of Literature
Jockers, M. L. (2014) · Quantitative methods in the humanities and social sciences · doi:10.1007/978-3-319-03164-4 · Cited by 198
Machine Coding of Event Data Using Regional and International Sources
Gerner, D. J., Schrodt, P. A., Francisco, R. Α., & Weddle, J. L. (1994) · International Studies Quarterly · doi:10.2307/2600873 · Cited by 188
Handbook of Mixed Membership Models and Their Applications
(2014) · Chapman & Hall/CRC eBooks · doi:10.1201/b17520 · Cited by 187
Automated Data Collection with R
Munzert, S., Rubba, C., Meißner, P., & Nyhuis, D. (2014) · Wiley eBooks · doi:10.1002/9781118834732 · Cited by 183
Validity Assessment of a Machine-Coded Event Data Set for the Middle East, 1982-92
Schrodt, P. A., & Gerner, D. J. (1994) · American Journal of Political Science · doi:10.2307/2111609 · Cited by 180
General purpose computer-assisted clustering and conceptualization
Grimmer, J., & King, G. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1018067108 · Cited by 179
Language and Ideology in Congress
Diermeier, D. A., Godbout, J., Yu, B., & Kaufmann, S. C. (2011) · British Journal of Political Science · doi:10.1017/s0007123411000160 · Cited by 176
Measuring Political Positions from Legislative Speech
Lauderdale, B. E., & Herzog, A. (2016) · Political Analysis · doi:10.1093/pan/mpw017 · Cited by 173
Lowe, W. (2008) · Political Analysis · doi:10.1093/pan/mpn004 · Cited by 166 · cited 1×
“Some of the earliest applications of automated text analysis in political science focused on using speeches and manifestos to locate European political parties in continuous ideological space (Laver et al. 2003; Slapin and Proksh 2008; Lowe 2008).”
BackgroundTracing the Flow of Policy Ideas in Legislatures: A Text Reuse Approach
Wilkerson, J., Smith, D. A., & Stramp, N. (2015) · American Journal of Political Science · doi:10.1111/ajps.12175 · Cited by 154
Self-citationScaling Politically Meaningful Dimensions Using Texts and Votes
Lauderdale, B. E., & Clark, T. S. (2014) · American Journal of Political Science · doi:10.1111/ajps.12085 · Cited by 147
Navigating the Local Modes of Big Data: The Case of Topic Models
Roberts, M. E., Stewart, B., & Tingley, D. H. (2016) · Computational Social Science · doi:10.1017/cbo9781316257340.004 · Cited by 145
Predicting and Interpolating State‐Level Polls Using Twitter Textual Data
Beauchamp, N. (2016) · American Journal of Political Science · doi:10.1111/ajps.12274 · Cited by 140
Lobbying as a collective enterprise: winners and losers of policy formulation in the European Union
Klüver, H. (2012) · Journal of European Public Policy · doi:10.1080/13501763.2012.699661 · Cited by 119
The Dynamics of Bureaucracy in the U.S. Government
Workman, S. (2015) · Cambridge University Press eBooks · doi:10.1017/cbo9781107447752 · Cited by 108
Introduction to the Special Issue: The Statistical Analysis of Political Text
Monroe, B. L., & Schrodt, P. A. (2008) · Political Analysis · doi:10.1093/pan/mpn017 · Cited by 99
(2016) · Cambridge University Press eBooks · doi:10.1017/cbo9781316257340 · Cited by 90
Denny, M. J., & Spirling, A. (2017) · SSRN Electronic Journal · doi:10.2139/ssrn.2849145 · Cited by 79
Infectious texts: Modeling text reuse in nineteenth-century newspapers
Smith, D. A., Cordell, R., & Dillon, E. M. (2013) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata.2013.6691675 · Cited by 60
TopicCheck: Interactive Alignment for Assessing Topic Model Stability
Chuang, J., Roberts, M. E., Stewart, B., Weiss, R., Tingley, D. H., Grimmer, J., & Heer, J. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1018 · Cited by 50
Text Annotation for Political Science Research
Cardie, C., & Wilkerson, J. (2008) · Journal of Information Technology & Politics · doi:10.1080/19331680802149590 · Cited by 46
Self-citationvan Atteveldt, W., Sheafer, T., Shenhav, S. R., & Fogel-Dror, Y. (2017) · Political Analysis · doi:10.1017/pan.2016.12 · Cited by 42
Bayesian Optimization of Text Representations
Yogatama, D., Kong, L., & Smith, N. A. (2015) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d15-1251 · Cited by 38
Eggers, A. C., & Spirling, A. (2016) · British Journal of Political Science · doi:10.1017/s0007123416000016 · Cited by 32
The Inside View: Using the E nron E‐mail Archive to Understand Corporate Political Attention
Drutman, L., & Hopkins, D. J. (2013) · Legislative Studies Quarterly · doi:10.1111/lsq.12001 · Cited by 21
One-Minute Speeches: Current House Practices
Schneider, J. (1999) · University of North Texas Digital Library (University of North Texas) · Cited by 10
Baldwin, S. E. (1908) · American Political Science Review · doi:10.2307/1944794 · Cited by 2
Copy and Paste Lawmaking: The Diusion of Policy Language across American State Legislatures
Jansa, J. M. (2015) · Cited by 2
Research works printed but not linked: 12
- Bird S, 2009, Natural Language Processing with Python—Analyzing Text with the Natural Language Toolkit
- Blei D, 2009, Text Mining: Classification, Clustering, and Applications, 71
- Blei DM, 2003, J. Mach. Learn. Res., 3, 993
- Casas A, 2016, Rev. Espan. Investig. Sociol., 155, 73
- Chuang J, Wilkerson JD, Weiss R, Tingley D, Stewart BM, et al. 2014. Computer-assisted content analysis: topic models for exploring multiple subjective interpretations. Presented at Advances in Neural Information Processing Systems Workshop on Human-Propelled Machine Learning, Montreal, Dec. 8–13
- Collingwood L, 2011, J. Inf. Technol. Polit., 4, 1
- Grimmer J, 2016, The unreliability of measures of intercoder reliability, and what to do about it.
- Huang A, 2008, Proc. Sixth New Zealand Computer Science Research Student Conference, 49
- Kohavi R, 1995, Proc. Int. Joint Conf. Artificial Intelligence, 1137
- Laver M, 2003, Am. Polit. Sci. Rev., 2, 311
- Wallach H, Dicker L, Jensen S. 2010. An alternative prior for nonparametric Bayesian clustering. In Proc. Thirteenth International Conference on Artificial Intelligence and Statistics, May 13–15, 2010, Chia Laguna Resort, Sardinia, Italy , ed. YW Teh, M Titterington, 9:892–99. http://www.jmlr.org/proceedings/papers/v9/
- Ward M, 2013, Analysis, 21, 267
Cited by
Sorted by citations
Barberá, P., Casas, A., Nagler, J., Egan, P. J., Bonneau, R., Jost, J. T., & Tucker, J. A. (2019) · American Political Science Review · doi:10.1017/s0003055419000352 · Cited by 584
Self-citationvan Atteveldt, W., van der Velden, M. A. C. G., & Boukes, M. (2021) · Communication Methods and Measures · doi:10.1080/19312458.2020.1869198 · Cited by 470
GPT is an effective tool for multilingual psychological text analysis
Rathje, S., Mirea, D.-M., Sucholutsky, I., Marjieh, R., Robertson, C. E., & Van Bavel, J. J. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2308950121 · Cited by 297
Hase, V., Mahl, D., Schäfer, M. S., & Keller, T. R. (2021) · Global Environmental Change · doi:10.1016/j.gloenvcha.2021.102353 · Cited by 342
More than Bags of Words: Sentiment Analysis with Word Embeddings
Rudkowsky, E., Haselmayer, M., Wastian, M., Jenny, M., Emrich, Š., & Sedlmair, M. (2018) · Communication Methods and Measures · doi:10.1080/19312458.2018.1455817 · Cited by 239
Metadata
- Authors
- Wilkerson, J., & Casas, A.
- Affiliations
- University of Washington (US)
- ORCID
- Casas, Andreu
- Venue
- Annual Review of Political Science
- Publisher
- Annual Reviews
- Published
- 2017-05-11
- Volume
- 20
- Issue
- 1
- Pages
- 529-544
- ISSN
- 1094-2939 (ISSN-L)
- 1545-1577
- OpenAlex
- W2515855575
- Semantic Scholar
- 0d345c2fb459e6ecc28328917ab37a4707e4a502
- Cited by
- 300
- References
- 65
- Citations by year
- 2026: 29 · 2025: 48 · 2024: 37 · 2023: 38 · 2022: 34 · 2021: 50 · 2020: 25 · 2019: 25 · 2018: 12 · 2017: 2
- FWCI
- 7.83
- Type
- journal article
- Language
- en
- Open access
- green
- research.vu.nl
- Licence
- other-oa
- Locations
- Annual Review of Political Science (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Topics
- Social Sciences › General Social Sciences › Computational and Text Analysis Methods
- Social Sciences › Political Science and International Relations › Political Science Research and Education
- Social Sciences › Arts and Humanities › Literature and Literary Theory › Digital Humanities and Scholarship
- Keywords
- text-as-data, computational text analysis, topic modeling, online information sources, quantitative text analysis
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
