The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Characterizing the Google Books Corpus: Strong Limits to Inferences of Socio-Cultural and Linguistic Evolution
Open accessAbstract
It is tempting to treat frequency trends from the Google Books data sets as indicators of the "true" popularity of various words and phrases. Doing so allows us to draw quantitatively strong conclusions about the evolution of cultural perception of a given topic, such as time or gender. However, the Google Books corpus suffers from a number of limitations which make it an obscure mask of cultural popularity. A primary issue is that the corpus is in effect a library, containing one of each book. A single, prolific author is thereby able to noticeably insert new phrases into the Google Books lexicon, whether the author is widely read or not. With this understood, the Google Books corpus remains an important data set to be considered more lexicon-like than text-like. Here, we show that a distinct problematic feature arises from the inclusion of scientific texts, which have become an increasingly substantive portion of the corpus throughout the 1900 s. The result is a surge of phrases typical to academic articles but less common in general, such as references to time in the form of citations. We use information theoretic methods to highlight these dynamics by examining and comparing major contributions via a divergence measure of English data sets between decades in the period 1800-2000. We find that only the English Fiction data set from the second version of the corpus is not heavily affected by professional texts. Overall, our findings call into question the vast majority of existing claims drawn from the Google Books corpus, and point to the need to fully characterize the dynamics of the corpus before using these data sets to draw broad conclusions about cultural and linguistic evolution.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Mathematical Theory of Communication
Shannon, C. E. (1948) · Bell System Technical Journal · doi:10.1002/j.1538-7305.1948.tb00917.x · Cited by 48,949
On Information and Sufficiency
Kullback, S., & Leibler, R. A. (1951) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177729694 · Cited by 20,193
Divergence measures based on the Shannon entropy
Lin, J. (1991) · IEEE Transactions on Information Theory · doi:10.1109/18.61115 · Cited by 5,140
Price, D. J. D. S. (1963) · Columbia University Press eBooks · doi:10.7312/pric91844 · Cited by 4,264
ASH, R. B. (1966) · Cited by 3,990
Quantitative Analysis of Culture Using Millions of Digitized Books
Michel, J.-B., Shen, Y. K., Aiden, A. P., Veres, A., Gray, M. K., The Google Books Team, Pickett, J. P., Hoiberg, D., Clancy, D., Norvig, P., et al. (2011) · Science · doi:10.1126/science.1199644 · Cited by 3,077
Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market
Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006) · Science · doi:10.1126/science.1121066 · Cited by 2,231
Self-citationThe Annals of Mathematical Statistics
King, W. I. (1930) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177733256 · Cited by 827
Temporal patterns of happiness and information in a global social network: hedonometrics and Twitter
Dodds, P. S., Harris, K. D., Kloumann, I. M., Bliss, C. A., & Danforth, C. M. (2011) · PLoS ONE · doi:10.1371/journal.pone.0026752 · Cited by 809
Self-citationSyntactic Annotations for the Google Books NGram Corpus
Lin, Y., Michel, J., Aiden, E. L., Orwant, J., Brockman, W., & Petrov, S. (2012) · Cited by 423
The Changing Psychology of Culture From 1800 Through 2000
Greenfield, P. M. (2013) · Psychological Science · doi:10.1177/0956797613479387 · Cited by 391
Proceedings of ACL 2018, System Demonstrations
(2018) · doi:10.18653/v1/p18-4 · Cited by 234
Languages cool as they expand: Allometric scaling and the decreasing need for new words
Petersen, A. M., Tenenbaum, J. N., Havlin, S., Stanley, H. E., & Perc, M. (2012) · Scientific Reports · doi:10.1038/srep00943 · Cited by 190
Languages cool as they expand: Allometric scaling and the decreasing need for new words
Petersen, A. M., Tenenbaum, J., Havlin, S., Stanley, E., & Perc, M. (2012) · University of Maribor digital library (University of Maribor) · Cited by 172
Stochastic Model for the Vocabulary Growth in Natural Languages
Gerlach, M., & Altmann, E. G. (2013) · Physical Review X · doi:10.1103/physrevx.3.021006 · Cited by 158
Statistical Laws Governing Fluctuations in Word Use from Word Birth to Word Death
Petersen, A. M., Tenenbaum, J., Havlin, S., & Stanley, E. (2012) · Scientific Reports · doi:10.1038/srep00313 · Cited by 158
Increases in Individualistic Words and Phrases in American Books, 1960–2008
Twenge, J., Campbell, W. K., & Gentile, B. (2012) · PLoS ONE · doi:10.1371/journal.pone.0040181 · Cited by 140
Koplenig, A. (2015) · Digital Scholarship in the Humanities · doi:10.1093/llc/fqv037 · Cited by 118
Male and Female Pronoun Use in U.S. Books Reflects Women’s Status, 1900–2008
Twenge, J. M., Campbell, W. K., & Gentile, B. (2012) · Sex Roles · doi:10.1007/s11199-012-0194-7 · Cited by 94
Books Average Previous Decade of Economic Misery
Bentley, R. A., Acerbi, A., Ormerod, P., & Lampos, V. (2014) · PLoS ONE · doi:10.1371/journal.pone.0083147 · Cited by 61
Cited by
Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change
Hamilton, W. L., Leskovec, J., & Jurafsky, D. (2016) · Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · doi:10.18653/v1/p16-1141 · Cited by 943
Statistics in Corpus Linguistics: A Practical Guide
Brezina, V. (2018) · Cambridge University Press eBooks · doi:10.1017/9781316410899 · Cited by 648
The Geometry of Culture: Analyzing the Meanings of Class through Word Embeddings
Kozlowski, A. C., Taddy, M., & Evans, J. A. (2019) · American Sociological Review · doi:10.1177/0003122419877135 · Cited by 499
Inducing Domain-Specific Sentiment Lexicons from Unlabeled Corpora
Hamilton, W. L., Clark, K., Leskovec, J., & Jurafsky, D. (2016) · Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d16-1057 · Cited by 355
Twentieth century morality: The rise and fall of moral concepts from 1900 to 2007
Wheeler, M. A., McGrath, M. J., & Haslam, N. (2019) · PLOS ONE · doi:10.1371/journal.pone.0212267 · Cited by 98
“Second, some researchers have suggested that the Google Ngram English corpus has included progressively more scientific literature throughout the 20 century [33].”
Background
Metadata
- Authors
- Pechenick, E. A., Danforth, C. M., & Dodds, P. S.
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2015-10-07
- Volume
- 10
- Issue
- 10
- Pages
- e0137041
- ISSN
- 1932-6203 (ISSN-L)
- OpenAlex
- W1808906688
- Cited by
- 436
- References
- 20
- FWCI
- 703.75
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.