Quantifying and contextualizing the impact of bioRxiv preprints through automated social media audience segmentation
Open accessPublished version of a preprint
The DOI matches its Crossref record; 60 of the 60 references it cites were found in a registry or an index.
Abstract
Engagement with scientific manuscripts is frequently facilitated by Twitter and other social media platforms. As such, the demographics of a paper's social media audience provide a wealth of information about how scholarly research is transmitted, consumed, and interpreted by online communities. By paying attention to public perceptions of their publications, scientists can learn whether their research is stimulating positive scholarly and public thought. They can also become aware of potentially negative patterns of interest from groups that misinterpret their work in harmful ways, either willfully or unintentionally, and devise strategies for altering their messaging to mitigate these impacts. In this study, we collected 331,696 Twitter posts referencing 1,800 highly tweeted bioRxiv preprints and leveraged topic modeling to infer the characteristics of various communities engaging with each preprint on Twitter. We agnostically learned the characteristics of these audience sectors from keywords each user's followers provide in their Twitter biographies. We estimate that 96% of the preprints analyzed are dominated by academic audiences on Twitter, suggesting that social media attention does not always correspond to greater public exposure. We further demonstrate how our audience segmentation method can quantify the level of interest from nonspecialist audience sectors such as mental health advocates, dog lovers, video game developers, vegans, bitcoin investors, conspiracy theorists, journalists, religious groups, and political constituencies. Surprisingly, we also found that 10% of the preprints analyzed have sizable (>5%) audience sectors that are associated with right-wing white nationalist communities. Although none of these preprints appear to intentionally espouse any right-wing extremist messages, cases exist in which extremist appropriation comprises more than 50% of the tweets referencing a given preprint. These results present unique opportunities for improving and contextualizing the public discourse surrounding scientific research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“a finding echoed by a recent study on the spread of the wider (i.e., not limited to preprints) COVID-19 research field on Twitter, which found that COVID-19 research was being widely disseminated and driven largely by academic Twitter users [33,34].”
From The evolving role of preprints in the dissemination of COVID-19 research and their impact on the science communication landscape (2021) · cites this paper as [34]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Inference of Population Structure Using Multilocus Genotype Data
Pritchard, J. K., Stephens, M., & Donnelly, P. (2000) · Genetics · doi:10.1093/genetics/155.2.945 · Cited by 33,908
(2003) · Journal of Machine Learning Research · doi:10.5555/944919.944937 · Cited by 32,337
Birds of a Feather: Homophily in Social Networks
McPherson, M., Smith-Lovin, L., & Cook, J. M. (2001) · Annual Review of Sociology · doi:10.1146/annurev.soc.27.1.415 · Cited by 18,567 · cited 1×
“A cornerstone of sociological research is the principle of network homophily, which states that individuals tend to connect and engage within a social network of others who share similar characteristics [18].”
Backgroundgenalex 6: genetic analysis in Excel. Population genetic software for teaching and research
PEAKALL, R., & SMOUSE, P. E. (2006) · Molecular Ecology Notes · doi:10.1111/j.1471-8286.2005.01155.x · Cited by 16,589
Blei, D. M., Ng, A. Y., & Jordan, M. I. (2002) · Advances in Neural Information Processing Systems 14 · doi:10.7551/mitpress/1120.003.0082 · Cited by 12,223
A Bayesian Hierarchical Model for Learning Natural Scene Categories
Li, F.-F., & Perona, P. (2005) · 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05) · doi:10.1109/cvpr.2005.16 · Cited by 3,613 · cited 1×
“Probabilistic topic modeling is a powerful and 150 flexible method for revealing the granular composition of all sorts of complex datasets—not only 151 the present application of text mining [36], but also inference of population ancestry using 152 genetic markers [32], computer vision [33], and many other areas of st…”
MethodPredicting protein structures with a multiplayer online game
Cooper, S., Khatib, F., Treuille, A., Barbero, J., Lee, J., Beenen, M., Leaver-Fay, A., Baker, D., Popović, Z., & players, F. (2010) · Nature · doi:10.1038/nature09304 · Cited by 1,368 · cited 1×
“their work in grant applications, enable new citizen science collaborations (such as fold-it, a 357 crowdsourced computer game where players predict protein structures [46]) or crowdfunded 358 research (such as the ~1,000 research projects that have been successfully funded on the 359 scientific crowdfunding platform,…”
BackgroundBirds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data
Barberá, P. (2015) · Political Analysis · doi:10.1093/pan/mpu011 · Cited by 1,298
Eysenbach, G. (2011) · Journal of Medical Internet Research · doi:10.2196/jmir.2012 · Cited by 1,083 · cited 1×
“…might provide an immediate indicator of an article’s potential downstream impacts, including (1) pre-dicting—and perhaps even causally increasing—the number of times one’s research is cited in the scientific literature [7,8]; (2) facilitating engagement with news media and public audiences [9,10]”
BackgroundA Review of Theory and Practice in Scientometrics
Mingers, J., & Leydesdorff, L. (2015) · European Journal of Operational Research · doi:10.1016/j.ejor.2015.04.002 · Cited by 1,002 · cited 1×
“These citation-based metrics (hereafter referred to as “traditional scientometrics”) 42 are not without their flaws, as they can be gamed and exploited [2] and are susceptible to 43 systematic biases [3].”
Background
Research works printed but not linked: 5
- To tweet or not to tweet? In: Science | AAAS [Internet]. 20 Nov 2014. [cited 2019 Dec 10]. Available from: https://www.sciencemag.org/careers/2014/10/tweet-or-not-tweet
- Kang J-H, Lerman K. Using Lists to Measure Homophily on Twitter. Workshops at the Twenty-Sixth AAAI Conference on Artificial Intelligence. 2012. [cited 2019 Nov 5]. Available from: https://www.aaai.org/ocs/index.php/WS/AAAIW12/paper/viewPaper/5425/5628
- Birney E. Race, genetics and pseudoscience: an explainer. In: Ewan’s Blog: Bioinformatician at large [Internet]. 24 10 2019 [cited 2019 Oct 31]. Available from: http://ewanbirney.com/2019/10/race-genetics-and-pseudoscience-an-explainer.html
- Sommer W. Why a Red “X” Is the New Symbol of Conservative Twitter. In: The Daily Beast [Internet]. 10 8 2018 [cited 2019 Aug 30]. Available from: https://www.thedailybeast.com/why-a-red-x-is-the-new-symbol-of-conservative-twitter
- “It”s a toxic place.’ How the online world of white nationalists distorts population genetics. In: Science | AAAS [Internet]. 22 May 2018. [cited 2020 Jan 16]. Available from: https://www.sciencemag.org/news/2018/05/it-s-toxic-place-how-online-world-white-nationalists-distorts-population-genetics
Cited by
Fraser, N., Brierley, L., Dey, G., Polka, J. K., Pálfy, M., Nanni, F., & Coates, J. A. (2021) · PLOS Biology · doi:10.1371/journal.pbio.3000959 · Cited by 555
“Hashtags associated with individual, highly tweeted preprints reveal some emergent themes that suggest communication of certain preprints can also extend well beyond scientific audiences (S4A Fig) [34].”
BackgroundKarlsson Linnér, R., Mallard, T. T., Barr, P. B., Sanchez-Roige, S., Madole, J. W., Driver, M. N., Poore, H. E., de Vlaming, R., Grotzinger, A. D., Tielbeek, J. J., et al. (2021) · Nature Neuroscience · doi:10.1038/s41593-021-00908-3 · Cited by 317
Self-citationA systematic review of the use of topic models for short text social media analysis
Doogan, C., Buntine, W., & Linger, H. (2023) · Artificial Intelligence Review · doi:10.1007/s10462-023-10471-x · Cited by 202
“For example, replacing characters with their word form such as ‘$’ to ‘dollar’ (Gregoriades and Pampaka 2020) and ‘#’ to ‘hashtag‘ before appending this word to the hashtag-word itself (Carlson and Harris 2020).”
BackgroundKhan, A., Gogarten, S. M., McHugh, C., Stilp, A. M., Sofer, T., Bowers, M., Wong, Q., Cupples, L. A., Hidalgo, B., Johnson, A. D., et al. (2022) · Cell Genomics · doi:10.1016/j.xgen.2022.100155 · Cited by 124
Counter the weaponization of genetics research by extremists
Carlson, J., Henn, B. M., Al-Hindi, D. R., & Ramachandran, S. (2022) · Nature · doi:10.1038/d41586-022-03252-z · Cited by 96
Self-citation
Metadata
- Authors
- Carlson, J., & Harris, K.
- Affiliations
- University of Washington (US)
- Corresponding author
- Carlson, Jedidiah
- Venue
- PLoS Biology
- Publisher
- Public Library of Science (PLoS)
- Published
- 2020-09-22
- Volume
- 18
- Issue
- 9
- Pages
- e3000860
- ISSN
- 1544-9173 (ISSN-L)
- 1545-7885
- PMC
- PMC7508356
- OpenAlex
- W3088297402
- Cited by
- 61
- References
- 55
- Citations by year
- 2026: 6 · 2025: 4 · 2024: 11 · 2023: 9 · 2022: 17 · 2021: 10 · 2020: 4
- FWCI
- 9.91
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS Biology (publishedVersion, cc-by)
- bioRxiv (Cold Spring Harbor Laboratory) (acceptedVersion, cc-by-nd)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- National Institute of General Medical Sciences (1R35GM133428-01)
- Burroughs Wellcome Fund
- Pew Charitable Trusts
- Searle Scholars Program
- Alfred P. Sloan Foundation
- Fred Hutchinson Cancer Research Center
- Topics
- Social Sciences › Library and Information Sciences › Academic Publishing and Open Access
- Social Sciences › Library and Information Sciences › scientometrics and bibliometrics research
- Social Sciences › Health › Social Media in Health Education
- Keywords
- Twitter engagement, audience segmentation, social media audiences, public perception of science, topic modeling, right-wing extremism, conspiracy theorists, science communication
- MeSH terms
- Academies and Institutes, Political Activism, Preprints as Topic, Electronic Data Processing, Humans, Publications, Science, Social Change, Databases as Topic, Internet, Access to Information, Information Literacy, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Quantifying and contextualizing the impact of bioRxiv preprints through automated social media audience segmentation. https://app.cochat.ai/papers/10.1371/journal.pbio.3000860
- The paper
- Carlson, J., & Harris, K. (2020). Quantifying and contextualizing the impact of bioRxiv preprints through automated social media audience segmentation. PLoS Biology. https://doi.org/10.1371/journal.pbio.3000860
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.