The DOI matches its Crossref record; 6 of the 6 references it cites were found in a registry or an index.
Computer‐Assisted Keyword and Document Set Discovery from Unstructured Text
Abstract
The (unheralded) first step in many applications of automated text analysis involves selecting keywords to choose documents from a large text corpus for further study. Although all substantive results depend on this choice, researchers usually pick keywords in ad hoc ways that are far from optimal and usually biased. Most seem to think that keyword selection is easy, since they do Google searches every day, but we demonstrate that humans perform exceedingly poorly at this basic task. We offer a better approach, one that also can help with following conversations where participants rapidly innovate language to evade authorities, seek political advantage, or express creativity; generic web searching; eDiscovery; look‐alike modeling; industry and intelligence analysis; and sentiment and topic analysis. We develop a computer‐assisted (as opposed to fully automated or human‐only) statistical approach that suggests keywords from available text without needing structured data as inputs. This framing poses the statistical problem in a new way, which leads to a widely applicable algorithm. Our specific approach is based on training classifiers, extracting information from (rather than correcting) their mistakes, and summarizing results with easy‐to‐understand Boolean search strings. We illustrate how the technique works with analyses of English texts about the Boston Marathon bombings, Chinese social media posts designed to evade censorship, and others.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Elements of Statistical Learning
Hastie, T., Tibshirani, R., & Friedman, J. (2009) · Springer Series in Statistics · doi:10.1007/978-0-387-84858-7 · Cited by 25,970
Neural Networks for Pattern Recognition
Bishop, C. M. (1995) · doi:10.1093/oso/9780198538493.001.0001 · Cited by 12,287
How Censorship in China Allows Government Criticism but Silences Collective Expression
KING, G., PAN, J., & ROBERTS, M. E. (2013) · American Political Science Review · doi:10.1017/s0003055413000014 · Cited by 2,415
Self-citationLetham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766
Identifying Mislabeled Training Data
Brodley, C. E., & Friedl, M. A. (1999) · Journal of Artificial Intelligence Research · doi:10.1613/jair.606 · Cited by 612
Roese, N. J., & Vohs, K. D. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612454303 · Cited by 457
Cited by
Firm‐Level Climate Change Exposure
SAUTNER, Z., VAN LENT, L., VILKOV, G., & ZHANG, R. (2023) · The Journal of Finance · doi:10.1111/jofi.13219 · Cited by 1,328
KING, G., PAN, J., & ROBERTS, M. E. (2017) · American Political Science Review · doi:10.1017/s0003055417000144 · Cited by 1,028
Self-citationLetham, B., Rudin, C., McCormick, T. H., & Madigan, D. (2015) · The Annals of Applied Statistics · doi:10.1214/15-aoas848 · Cited by 766
Denny, M. J., & Spirling, A. (2018) · Political Analysis · doi:10.1017/pan.2017.44 · Cited by 463
Racial Bias in Hate Speech and Abusive Language Detection Datasets
Davidson, T., Bhattacharya, D., & Weber, I. (2019) · Proceedings of the Third Workshop on Abusive Language Online · doi:10.18653/v1/w19-3504 · Cited by 414
Metadata
- Authors
- King, G., Lam, P., & Roberts, M. E.
- Venue
- American Journal of Political Science
- Publisher
- Wiley
- Published
- 2017-04-03
- Volume
- 61
- Issue
- 4
- Pages
- 971-988
- ISSN
- 0092-5853 (ISSN-L)
- 1540-5907
- OpenAlex
- W2604700393
- Cited by
- 229
- References
- 6
- FWCI
- 76.41
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Computer‐Assisted Keyword and Document Set Discovery from Unstructured Text. https://app.cochat.ai/papers/10.1111/ajps.12291
- The paper
- King, G., Lam, P., & Roberts, M. E. (2017). Computer‐Assisted Keyword and Document Set Discovery from Unstructured Text. American Journal of Political Science. https://doi.org/10.1111/ajps.12291
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.