The spread of low-credibility content by social bots
Open accessAbstract
The massive spread of digital misinformation has been identified as a major threat to democracies. Communication, cognitive, social, and computer scientists are studying the complex causes for the viral diffusion of misinformation, while online platforms are beginning to deploy countermeasures. Little systematic, data-based evidence has been published to guide these efforts. Here we analyze 14 million messages spreading 400 thousand articles on Twitter during ten months in 2016 and 2017. We find evidence that social bots played a disproportionate role in spreading articles from low-credibility sources. Bots amplify such content in the early spreading moments, before an article goes viral. They also target users with many followers through replies and mentions. Humans are vulnerable to this manipulation, resharing content posted by bots. Successful low-credibility sources are heavily supported by social bots. These results suggest that curbing social bots may be an effective strategy for mitigating the spread of online misinformation.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Bots are maliciously used on social media platforms to spread disinformation and manipulate user popularity and engagement metrics.
“However, bots are also maliciously used on social media platforms to spread disinformation and manipulate user popularity and engagement metrics [6,39,48].”
From Unsupervised detection of coordinated fake-follower campaigns on social media (2024) · cites this paper as [39]
The Introduction and Discussion report that bots spread low-credibility content and amplify its reach. Results > Bot strategies describes bots creating the appearance that content is widely shared, and Results > Bot impact reports increased human engagement. These passages do not establish that bots manipulate users’ own popularity or engagement metrics. (LLM, conf=0.86)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
1 retracted
Limited individual attention and online virality of low-quality information
Qiu, X., Oliveira, D. F. M., Shirazi, A. S., Flammini, A., & Menczer, F. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0132 · Cited by 186 · see citing works
Self-citationRetractedThe spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · cited 4× · see citing works
“It has also been employed in other studies [57, 61] and is widely adopted, serving hundreds of thousand requests daily.”
InfluentialError and attack tolerance of complex networks
Albert, R., Jeong, H., & Barabási, A.-L. (2000) · Nature · doi:10.1038/35019019 · Cited by 7,534 · see citing works
Social Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767 · see citing works
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160 · see citing works
Self-citationThe spreading of misinformation online
Del Vicario, M., Bessi, A., Zollo, F., Petroni, F., Scala, A., Caldarelli, G., Stanley, H. E., & Quattrociocchi, W. (2016) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1517441113 · Cited by 2,407 · see citing works
Experimental Study of Inequality and Unpredictability in an Artificial Cultural Market
Salganik, M. J., Dodds, P. S., & Watts, D. J. (2006) · Science · doi:10.1126/science.1121066 · Cited by 2,219 · see citing works
Beyond misinformation: Understanding and coping with the “post-truth” era.
Lewandowsky, S., Ecker, U. K. H., & Cook, J. (2017) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2017.07.008 · Cited by 1,985 · see citing works
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973 · see citing works
Self-citationCAPTCHA: Using Hard AI Problems for Security
von Ahn, L., Blum, M., Hopper, N., & Langford, J. C. (2003) · Lecture notes in computer science · doi:10.1007/3-540-39200-9_18 · Cited by 1,450 · see citing works
Ideology, motivated reasoning, and cognitive reflection
Kahan, D. M. (2013) · Judgment and Decision Making · doi:10.1017/s1930297500005271 · Cited by 1,302 · see citing works
Jagatic, T. N., Johnson, N. A., Jakobsson, M., & Menczer, F. (2007) · Communications of the ACM · doi:10.1145/1290958.1290968 · Cited by 1,156 · see citing works
Self-citationSocial bots distort the 2016 U.S. Presidential election online discussion
Bessi, A., & Ferrara, E. (2016) · First Monday · doi:10.5210/fm.v21i11.7090 · Cited by 993 · see citing works
Stroud, N. J. (2011) · doi:10.1093/acprof:oso/9780199755509.001.0001 · Cited by 950 · see citing works
Online Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 928 · see citing works
Self-citationPolitical Polarization on Twitter
Conover, M., Ratkiewicz, J., Francisco, M., Goncalves, B., Menczer, F., & Flammini, A. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14126 · Cited by 626 · cited 1× · see citing works
“These include information overload and finite attention [41], novelty of false news [57], the selective exposure [52, 40, 39] caused by polarized and segregated online social networks [9, 8], algorithmic popularity bias [45, 20, 37], and other cognitive vulnerabilities such as confirmation bias and motivated reasoning…”
Self-citationWhy Do Partisan Media Polarize Viewers?
Levendusky, M. S. (2013) · American Journal of Political Science · doi:10.1111/ajps.12008 · Cited by 587 · see citing works
Detecting and Tracking Political Abuse in Social Media
Ratkiewicz, J., Conover, M., Meiss, M., Gonçalves, B., Flammini, A., & Menczer, F. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14127 · Cited by 580 · see citing works
Self-citationDisinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524 · see citing works
Sunstein, C. R. (2009) · doi:10.1093/oso/9780195378016.001.0001 · Cited by 509 · see citing works
The DARPA Twitter Bot Challenge
Subrahmanian, V. S., Azaria, A., Durst, S., Kagan, V., Galstyan, A., Lerman, K., Zhu, L., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Computer · doi:10.1109/mc.2016.183 · Cited by 480 · see citing works
Self-citation(2018) · Oxford University Press eBooks · doi:10.1093/oso/9780190931407.001.0001 · Cited by 418 · see citing works
Anatomy of an online misinformation network
Shao, C., Hui, P.-M., Wang, L., Jiang, X., Flammini, A., Menczer, F., & Ciampaglia, G. L. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196087 · Cited by 282 · see citing works
Self-citationPartisan asymmetries in online political activity
Conover, M. D., Gonçalves, B., Flammini, A., & Menczer, F. (2012) · EPJ Data Science · doi:10.1140/epjds6 · Cited by 234 · see citing works
Self-citationHow algorithmic popularity bias hinders or promotes quality
Ciampaglia, G. L., Nematzadeh, A., Menczer, F., & Flammini, A. (2018) · Scientific Reports · doi:10.1038/s41598-018-34203-2 · Cited by 204 · see citing works
Self-citationMarkines, B., Cattuto, C., & Menczer, F. (2009) · doi:10.1145/1531914.1531924 · Cited by 173 · see citing works
Self-citationHow Visibility and Divided Attention Constrain Social Contagion
Hodas, N. O., & Lerman, K. (2012) · 2012 International Conference on Privacy, Security, Risk and Trust and 2012 International Confernece on Social Computing · doi:10.1109/socialcom-passat.2012.129 · Cited by 136 · see citing works
Perceived social presence reduces fact-checking
Jun, Y., Meng, R., & Johar, G. V. (2017) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1700175114 · Cited by 121 · see citing works
Texas and Its Measles Epidemics
Hotez, P. J. (2016) · PLoS Medicine · doi:10.1371/journal.pmed.1002153 · Cited by 92 · see citing works
Flagged first, then by citations
Research works printed but not linked: 5
- [4]Mustafaraj, E. & Metaxas, P. T. From obscurity to prominence in minutes: Political speech and real-time search. In Proc. Web Science Conference: Extending the Frontiers of Society On-Line (Raleigh, 2010).
- [14]Pariser, E. The Filter Bubble: How the New Personalized Web Is Changing What We Read and How We Think (Penguin, New York, 2011).
- [15]Nikolov, D., Lalmas, M., Flammini, A. & Menczer, F. Quantifying biases in online information exposure. J. Am. Soc. Inform. Sci. Technol. Preprint at https://arxiv.org/abs/1807.06958 (2018). · arXiv
- [21]Stroud, N. Niche News: The Politics of News Choice (Oxford University Press, Oxford, 2011).
- [35]Mosseri, A. News feed fyi: showing more informative links in news feed. Press release, Facebook (2017). https://newsroom.fb.com/news/2017/06/news-feed-fyi-showing-more-informative-links-in-news-feed/
Cited by
Sorted by citations
Fake news on Twitter during the 2016 U.S. presidential election
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019) · Science · doi:10.1126/science.aau2706 · Cited by 1,803 · see citing works
Prevalence of health misinformation on social media: systematic review
Suarez-Lledo, V., & Alvarez-Galvez, J. (2021) · Journal of Medical Internet Research · doi:10.2196/17187 · Cited by 1,564 · see citing works
Rise of Machine Agency: A Framework for Studying the Psychology of Human–AI Interaction (HAII)
Sundar, S. S. (2020) · Journal of Computer-Mediated Communication · doi:10.1093/jcmc/zmz026 · Cited by 871 · see citing works
Fake news, disinformation and misinformation in social media: a review
Aïmeur, E., Amri, S., & Brassard, G. (2023) · Social Network Analysis and Mining · doi:10.1007/s13278-023-01028-5 · Cited by 772 · see citing works
Influence of fake news in Twitter during the 2016 US presidential election
Bovet, A., & Makse, H. A. (2019) · Nature Communications · doi:10.1038/s41467-018-07761-2 · Cited by 701 · see citing works
Assessing the risks of ‘infodemics’ in response to COVID-19 epidemics
Gallotti, R., Valle, F., Castaldo, N., Sacco, P., & De Domenico, M. (2020) · Nature Human Behaviour · doi:10.1038/s41562-020-00994-6 · Cited by 591 · see citing works
Tangcharoensathien, V., Calleja, N., Nguyen, T., Purnat, T., D’Agostino, M., Garcia-Saiso, S., Landry, M., Rashidian, A., Hamilton, C., AbdAllah, A., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/19659 · Cited by 561 · see citing works
Masood, M., Nawaz, M., Malik, K. M., Javed, A., Irtaza, A., & Malik, H. (2023) · Applied Intelligence · doi:10.1007/s10489-022-03766-z · Cited by 543 · see citing works
“Fake News” Is Not Simply False Information: A Concept Explication and Taxonomy of Online Content
Molina, M. D., Sundar, S. S., Le, T., & Lee, D. (2021) · American Behavioral Scientist · doi:10.1177/0002764219878224 · Cited by 485 · see citing works
Arming the public with artificial intelligence to counter social bots
Yang, K., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.115 · Cited by 419 · see citing works
“A study based on 14 million tweets posted during and after the 2016 U.S. presidential election revealed that bots played a key role in the spread of low-credibility content (Shao et al., 2018a,b).”
BackgroundSelf-citationSocial Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373 · see citing works
COVID-19-related misinformation on social media: a systematic review
Gabarron, E., Oyeyemi, S. O., & Wynn, R. (2021) · Bulletin of the World Health Organization · doi:10.2471/blt.20.276782 · Cited by 345 · see citing works
The False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184 · see citing works
Social media’s contribution to political misperceptions in U.S. Presidential elections
Garrett, R. K. (2019) · PLOS ONE · doi:10.1371/journal.pone.0213500 · Cited by 151 · see citing works
The Moral Psychology of Artificial Intelligence
Bonnefon, J.-F., Rahwan, I., & Shariff, A. (2024) · Annual Review of Psychology · doi:10.1146/annurev-psych-030123-113559 · Cited by 151 · see citing works
Quantifying echo chamber effects in information spreading over political communication networks
Cota, W., Ferreira, S. C., Pastor-Satorras, R., & Starnini, M. (2019) · EPJ Data Science · doi:10.1140/epjds/s13688-019-0213-9 · Cited by 146 · see citing works
A Survey on the Detection and Impacts of Deepfakes in Visual, Audio, and Textual Formats
Mubarak, R., Alsboui, T., Alshaikh, O., Inuwa-Dutse, I., Khan, S., & Parkinson, S. (2023) · IEEE Access · doi:10.1109/access.2023.3344653 · Cited by 132 · see citing works
The Future of False Information Detection on Social Media: New Perspectives and Trends
Guo, B., Ding, Y., Yao, L., Liang, Y., & Yu, Z. (2021) · ACM Computing Surveys · doi:10.1145/3393880 · Cited by 122 · see citing works
Investigating Italian disinformation spreading on Twitter in the context of 2019 European elections
Pierri, F., Artoni, A., & Ceri, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227821 · Cited by 93 · see citing works
Exposure to social engagement metrics increases vulnerability to misinformation
Avram, M., Micallef, N., Patil, S., & Menczer, F. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-033 · Cited by 83 · see citing works
Self-citationEmotions explain differences in the diffusion of true vs. false social media rumors
Pröllochs, N., Bär, D., & Feuerriegel, S. (2021) · Scientific Reports · doi:10.1038/s41598-021-01813-2 · Cited by 77 · see citing works
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71 · see citing works
Self-citationJournalists on Twitter: self-branding, audiences, and involvement of bots
Varol, O., & Uluturk, I. (2020) · Journal of Computational Social Science · doi:10.1007/s42001-019-00056-6 · Cited by 39 · see citing works
Self-citationBruno, M., Lambiotte, R., & Saracco, F. (2022) · EPJ Data Science · doi:10.1140/epjds/s13688-022-00330-0 · Cited by 40 · see citing works
Trust within human-machine collectives depends on the perceived consensus about cooperative norms
Makovi, K., Sargsyan, A., Li, W., Bonnefon, J.-F., & Rahwan, T. (2023) · Nature Communications · doi:10.1038/s41467-023-38592-5 · Cited by 36 · see citing works
Toward Mitigating Misinformation and Social Media Manipulation in LLM Era
Zhang, Y., Sharma, K., Du, L., & Liu, Y. (2024) · Companion Proceedings of the ACM Web Conference 2024 · doi:10.1145/3589335.3641256 · Cited by 25 · see citing works
Unsupervised detection of coordinated fake-follower campaigns on social media
Zouzou, Y., & Varol, O. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-024-00499-6 · Cited by 19 · see citing works
Self-citationInvestigating Fake and Reliable News Sources Using Complex Networks Analysis
Mazzeo, V., & Rapisarda, A. (2022) · Frontiers in Physics · doi:10.3389/fphy.2022.886544 · Cited by 6 · see citing works
Tucker, J. A. (2023) · Handbook of Computational Social Science for Policy · doi:10.1007/978-3-031-16624-2_20 · Cited by 3 · see citing works
He, H., Yang, H., Burckhardt, P., Kapravelos, A., Vasilescu, B., & Kästner, C. (2026) · Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering · doi:10.1145/3744916.3764531 · Cited by 2 · see citing works
The Moltbook Illusion: Separating Human Influence from Emergent Behavior in AI Agent Societies
Li, N. (2026) · arXiv · doi:10.48550/arxiv.2602.07432 · Cited by 1 · see citing works
“Studies have documented how coordinated inauthentic behavior distorts political discourse 16,17 , amplifies low-credibility content 18 , and undermines the integrity of online information ecosystems 19,20 .”
- Authors
- Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F.
- Affiliations
- National University of Defense Technology (CN)
- Indiana University Bloomington (US)
- Corresponding author
- Menczer, Filippo
- ORCID
- Venue
- Nature Communications
- Published
- 2018-11-20
- Volume
- 9
- Issue
- 1
- Pages
- 4787-4787
- ISSN
- 2041-1723 (ISSN-L)
- PMC
- PMC6246561
- OpenAlex
- W2804365752
- Cited by
- 1,125
- References
- 29
- Citations by year
- 2026: 133 · 2025: 151 · 2024: 172 · 2023: 139 · 2022: 175 · 2021: 129 · 2020: 157 · 2019: 66 · 2018: 3
- FWCI
- 201.92
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- Nature Communications (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- IUScholarWorks Open (Indiana University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1735095)
- China Scholarship Council
- Defense Advanced Research Projects Agency (W911NF-12-1-0037)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Physics and Astronomy › Statistical and Nonlinear Physics › Complex Network Analysis Techniques
- Keywords
- social bots, misinformation, viral diffusion, Twitter, user engagement, misinformation mitigation, 2016 US presidential election
- MeSH terms
- Communication, Data Collection, Humans, Information Dissemination, Social Networking, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
