The DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Structural Topic Models for Open‐Ended Survey Responses
Abstract
Collection and especially analysis of open‐ended survey responses are relatively rare in the discipline and when conducted are almost exclusively done through human coding. We present an alternative, semiautomated approach, the structural topic model (STM) (Roberts, Stewart, and Airoldi 2013; Roberts et al. 2013), that draws on recent developments in machine learning based analysis of textual data. A crucial contribution of the method is that it incorporates information about the document, such as the author's gender, political affiliation, and treatment assignment (if an experimental study). This article focuses on how the STM is helpful for survey researchers and experimentalists. The STM makes analyzing open‐ended responses easier, more revealing, and capable of being used to estimate treatment effects. We illustrate these innovations with analysis of text from surveys and experiments.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Introduction to information retrieval
(2009) · Choice Reviews Online · doi:10.5860/choice.46-2715 · Cited by 12,278
Introduction to Information Retrieval
Manning, C. D., Raghavan, P., & Schütze, H. (2008) · doi:10.1017/cbo9780511809071 · Cited by 11,288
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts
Grimmer, J., & Stewart, B. M. (2013) · Political Analysis · doi:10.1093/pan/mps028 · Cited by 3,297 · cited 2×
“…recent efforts in digitalization of text, researchers are rapidly reinventing the evidence base of the social sciences by introducing new measures of political and social behavior constructed from large text archives (Grimmer and Stewart 2013; Hopkins and King 2012; King 2009; Lazer et al. 2009).”
BackgroundSelf-citationContent Analysis in Mass Communication: Assessment and Reporting of Intercoder Reliability
Lombard, M., Snyder-Duch, J., & Bracken, C. C. (2002) · Human Communication Research · doi:10.1111/j.1468-2958.2002.tb00826.x · Cited by 3,068
Inter-Coder Agreement for Computational Linguistics
Artstein, R., & Poesio, M. (2008) · Computational Linguistics · doi:10.1162/coli.07-034-r2 · Cited by 1,665
Extracting Policy Positions from Political Texts Using Words as Data
LAVER, M., BENOIT, K., & GARRY, J. (2003) · American Political Science Review · doi:10.1017/s0003055403000698 · Cited by 1,377
Spontaneous giving and calculated greed
Rand, D. G., Greene, J. D., & Nowak, M. A. (2012) · Nature · doi:10.1038/nature11467 · Cited by 1,334
Self-citationHow to analyze political attention with minimal assumptions and costs
Quinn, K. M., Monroe, B. L., Colaresi, M., Crespin, M. H., & Radev, D. R. (2010) · American Journal of Political Science · doi:10.1111/j.1540-5907.2009.00427.x · Cited by 671
Anxiety, Immigration, and the Search for Information
Gadarian, S. K., & Albertson, B. (2014) · Political Psychology · doi:10.1111/pops.12034 · Cited by 231
Self-citation
Cited by
stm : An R Package for Structural Topic Models
Roberts, M. E., Stewart, B. M., & Tingley, D. (2019) · Journal of Statistical Software · doi:10.18637/jss.v091.i02 · Cited by 1,714
Self-citationPrescriptive analytics: Literature review and research challenges
Lepenioti, K., Bousdekis, A., Apostolou, D., & Mentzas, G. (2020) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2019.04.003 · Cited by 501
“…making problems in different contexts are: Artificial Neural Networks (Domingos, Burguillo, & Lenaerts, 2017), Bayesian Networks (Said, Shahzad, Zamai, Hubac, & Tollenaere, 2016), topic models (Roberts et al., 2014), fuzzy sets (Pal & Ceglarek, 2013), Markov models (Bousdekis et al., 2018).”
Data integration enables global biodiversity synthesis
Heberling, J. M., Miller, J. T., Noesgaard, D., Weingart, S. B., & Schigel, D. (2021) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2018093118 · Cited by 380
Computer-Assisted Text Analysis for Comparative Politics
Lucas, C., Nielsen, R. A., Roberts, M. E., Stewart, B. M., Storer, A., & Tingley, D. (2015) · Political Analysis · doi:10.1093/pan/mpu019 · Cited by 366
Self-citationLarge-Scale Computerized Text Analysis in Political Science: Opportunities and Challenges
Wilkerson, J., & Casas, A. (2017) · Annual Review of Political Science · doi:10.1146/annurev-polisci-052615-025542 · Cited by 300
Metadata
- Authors
- Roberts, M. E., Stewart, B. M., Tingley, D., Lucas, C., Leder‐Luis, J., Gadarian, S. K., Albertson, B., & Rand, D. G.
- Venue
- American Journal of Political Science
- Publisher
- Wiley
- Published
- 2014-03-06
- Volume
- 58
- Issue
- 4
- Pages
- 1064-1082
- ISSN
- 0092-5853 (ISSN-L)
- 1540-5907
- OpenAlex
- W2096974619
- Cited by
- 1,952
- References
- 10
- FWCI
- 230.94
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Structural Topic Models for Open‐Ended Survey Responses. https://app.cochat.ai/papers/10.1111/ajps.12103
- The paper
- Roberts, M. E., Stewart, B. M., Tingley, D., Lucas, C., Leder‐Luis, J., Gadarian, S. K., Albertson, B., & Rand, D. G. (2014). Structural Topic Models for Open‐Ended Survey Responses. American Journal of Political Science. https://doi.org/10.1111/ajps.12103
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
