CoChat
Sign Up
Cited by 179Advanced Clustering Algorithms ResearchData Visualization and AnalyticsData Mining and Machine Learning Applications

General purpose computer-assisted clustering and conceptualization

Open access
Grimmer, Justin, King, Gary·2011·Proceedings of the National Academy of Sciences
DOI 10.1073/pnas.1018067108PMC PMC3041127
Open this paper in CoChat — free account

Abstract

We develop a computer-assisted method for the discovery of insightful conceptualizations, in the form of clusterings (i.e., partitions) of input objects. Each of the numerous fully automated methods of cluster analysis proposed in statistics, computer science, and biology optimize a different objective function. Almost all are well defined, but how to determine before the fact which one, if any, will partition a given set of objects in an “insightful” or “useful” way for a given user is unknown and difficult, if not logically impossible. We develop a metric space of partitions from all existing cluster analysis methods applied to a given dataset (along with millions of other solutions we add based on combinations of existing clusterings) and enable a user to explore and interact with it and quickly reveal or prompt useful or insightful conceptualizations. In addition, although it is uncommon to do so in unsupervised learning problems, we offer and implement evaluation designs that make our computer-assisted approach vulnerable to being proven suboptimal in specific data types. We demonstrate that our approach facilitates more efficient and insightful discovery of useful information than expert human coders or many existing fully automated methods.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1073/pnas.1018067108

Cites

2 of 2 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Providing guidance on when specific measures of similarity, objective functions, or optimization algorithms are likely to lead to useful or insightful partitions of data without specific knowledge of the data set may be impossible.

    “Indeed, it appears that providing this guidance without specific knowledge of the data set may be impossible (Grimmer and King 2011).”

    From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Grimmer and King 2011

  • Grimmer and King (2011) introduce a method for exploring the space, easily searching over included methods and millions of other clusterings.

    “Using this space, Grimmer and King (2011) introduce a method for exploring it, easily searching over the included methods and millions of other clusterings that are the result of combinations of similar organizations of the data.”

    From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Grimmer and King 2011

  • Using experimentally elicited human input allows direct comparison of the quality of clusterings, but it is unable to provide an absolute measure of semantic validity.

    “Using this method allows direct comparison of the quality of clusterings, but it is unable to provide an absolute measure of semantic validity.”

    From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Grimmer and King 2011

  • Grimmer and King (2011) show how to embed partitions into a two-dimensional space such that two clusterings are close if they organize documents in similar ways.

    “Then, Grimmer and King (2011) show how to embed the partitions into a two-dimensional space such that two clusterings are close in the space if they organize the documents in similar ways.”

    From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Grimmer and King 2011

  • CAC is a method for efficiently searching over a large space of clusterings.

    “With this idea in mind, CAC is a method for efficiently searching over a large space of clusterings (Grimmer and King 2011).”

    From Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts (2013) · cites this paper as Grimmer and King 2011

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 23

  1. [1]Bailey KD. Typologies and Taxonomies: An Introduction to Classification Techniques. Beverly Hills, CA: Sage; 1994. [Google Scholar]
  2. [2]Manning CD, Raghavan P, Schütze H. Introduction to Information Retrieval. New York: Cambridge Univ Press; 2008. [Google Scholar]
  3. [4]Shawe-Taylor J, Cristianini N. Kernel Methods for Pattern Analysis. Cambridge: Cambridge Univ Press; 2004. [Google Scholar]
  4. [5]Jordan Michael, Ghahramani Z, Jaakkola T, Saul L. An introduction to variational methods for graphical models. J Mach Learn Res. 1999;37:183–233. [Google Scholar]
  5. [6]Meila M. Comparing clusterings: An information based distance. J Multivariate Anal. 2007;98:873–895. [Google Scholar]
  6. [7]Sammon J. A nonlinear mapping for data structure analysis. IEEE T Comput. 1969;C-18:401–409. [Google Scholar]
  7. [8]Strehl Alexander, Grosh J. Cluster ensembles: A knowledge reuse framework for combining multiple partitions. J Mach Learn Res. 2003;3:583–617. [Google Scholar]
  8. [9]Fern X, Brodley C. Random project for high dimensional data clustering: A cluster ensemble approach; Proceedings of the Twentieth International Conference on Machine Learning; Washington: International Machine Learning Society; 2003. [Google Scholar]
  9. [10]Law M, Topchy A, Jain A. Multi-objective data clustering; IEEE Computer Society Conference on Computer Vision and Pattern Recognition; Washington: IEEE Computer Society; 2004. [Google Scholar]
  10. [11]Caruana R, Elhawary M, Nguyen N, Smith C. Meta clustering; ICDM'06. Sixth International Conference on Data Mining; Bethesda, MD: SIAM; 2006. pp. 107–118. [Google Scholar]
  11. [12]Gionis A, Mannila H, Tsaparas P. Clustering aggregation; Proceedings of the 21st International Conference on Data Engineering; Tokyo: IEEE Computer Society; 2005. [Google Scholar]
  12. [13]Topchy A, Jain AK, Punch W. Combining multiple weak clusterings; Proceedings IEEE International Conference on Data Mining; Melbourne: IEEE Computer Society; 2003. [Google Scholar]
  13. [14]Diaconis P, Goel S, Holmes S. Horseshoes in multidimensional scaling and local kernel methods. Ann Appl Stat. 2008;2:777–807. [Google Scholar]
  14. [16]Blei D, Jordan M. Variational inference for dirichlet process mixtures. Bayesian Analysis. 2006;1:121–144. [Google Scholar]
  15. [17]Guttman L. The problem of attitude and opinion measurement. Measurement and Prediction. 1950;4:46–59. [Google Scholar]
  16. [18]Mayhew D. The Electoral Connection. New Haven, CT: Yale Univ Press; 1974. [Google Scholar]
  17. [19]Fiorina M. Congress, Keystone of the Washington Establishment. New Haven, CT: Yale Univ Press; 1989. [Google Scholar]
  18. [20]Eulau H, Karps P. The puzzle of representation: Specifying components of responsiveness. Legis Stud Quart. 1977;2:233–254. [Google Scholar]
  19. [21]Yiannakis DE. House members communication styles: Newsletters and press releases. J Polit. 1982;44:1049–1071. [Google Scholar]
  20. [22]Hopkins D, King G. A method of automated nonparametric content analysis for social science. Am J Polit Sci. 2010;54:229–247. http://gking.harvard.edu/files/abs/words-abs.shtml. [Google Scholar]
  21. [23]Gutmann A, Thompson D. Democracy and Disagreement. Cambridge, MA: Harvard Univ Press; 1996. [Google Scholar]
  22. [24]Miller WE, Stokes DE. Constituency influence in Congress. Am Polit Sci Rev. 1963;57:45–56. [Google Scholar]
  23. [25]Pitkin HF. The Concept of Representation. Berkeley, CA: Univ of California Press; 1972. [Google Scholar]

Cited by

Sorted by citations

Authors
Grimmer, J., & King, G.
Affiliations
  • Stanford University (US)
  • Harvard University (US)
ORCID
Venue
Proceedings of the National Academy of Sciences
Publisher
National Academy of Sciences
Published
2011-02-03
Volume
108
Issue
7
Pages
2643-2650
ISSN
  • 0027-8424 (ISSN-L)
  • 1091-6490
DOI
10.1073/pnas.1018067108
PMC
PMC3041127
OpenAlex
W2149978102
Cited by
179
References
2
Citations by year
2026: 1 · 2025: 11 · 2024: 7 · 2023: 8 · 2022: 9 · 2021: 10 · 2020: 24 · 2019: 11 · 2018: 11 · 2017: 17 · 2016: 14 · 2015: 18 · 2014: 12 · 2013: 13 · 2012: 12
FWCI
11.19
Type
journal article
Language
en
Open access
Locations
  • Proceedings of the National Academy of Sciences (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
  • Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
Topics
  • Physical Sciences › Computer Science › Artificial Intelligence › Advanced Clustering Algorithms Research
  • Physical Sciences › Computer Science › Computer Vision and Pattern Recognition › Data Visualization and Analytics
  • Physical Sciences › Computer Science › Information Systems › Data Mining and Machine Learning Applications
Keywords
conceptualization, clustering, interactive data exploration, unsupervised learning
MeSH terms
Algorithms, Artificial Intelligence, Classification, Medical Informatics, Pattern Recognition, Automated, Cluster Analysis

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review