Machine learning language models: Achilles heel for social media platforms and a possible solution
Open accessAbstract
Any uptick in new misinformation that casts doubt on COVID-19 mitigation strategies, such as vaccine boosters and masks, could reverse society's recovery from the pandemic both nationally and globally.This study demonstrates how machine learning language models can automatically generate new COVID-19 and vaccine misinformation that appears fresh and realistic (i.e.human-generated) even to subject matter experts.The study uses the latest version of the GPT model that is public and freely available, GPT-2, and inputs publicly available text collected from social media communities that are known for their high levels of health misinformation.The same team of subject matter experts that classified the original social media data used as input, are then asked to categorize the GPT-2 output without knowing about its automated origin.None of them successfully identified all the synthetic text strings as being a product of the machine model.This presents a clear warning for social media platforms: an unlimited volume of fresh and seemingly human-produced misinformation can be created perpetually on social media using current, off-the-shelf machine learning algorithms that run continually.We then offer a solution: a statistical approach that detects differences in the dynamics of this output as compared to typical human behavior.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The ability of LLMs to generate human-like text could be used to create fake news articles or other seemingly legitimate but fabricated or misleading content without the reader realizing it is AI-produced.
“The ability of LLMs to generate texts similar to those composed by humans could be used to create fake news articles or other seemingly legitimate but actually fabricated or misleading content, (45, 46) without the reader realizing that the text is produced by AI (47).”
From ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health (2023) · cites this paper as [47]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Transformers: State-of-the-Art Natural Language Processing
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., et al. (2020) · Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations · doi:10.18653/v1/2020.emnlp-demos.6 · Cited by 8,741 · see citing works
Experimental evidence for tipping points in social convention
Centola, D., Becker, J., Brackbill, D., & Baronchelli, A. (2018) · Science · doi:10.1126/science.aas8827 · Cited by 750 · see citing works
The online competition between pro- and anti-vaccination views
Johnson, N. F., Velásquez, N., Restrepo, N. J., Leahy, R., Gabriel, N., Oud, S. E., Zheng, M., Manrique, P. D., Wuchty, S., & Lupu, Y. (2020) · Nature · doi:10.1038/s41586-020-2281-1 · Cited by 704 · see citing works
Self-citationScience vs Conspiracy: Collective Narratives in the Age of Misinformation
Bessi, A., Coletto, M., Davidescu, G. A., Scala, A., Caldarelli, G., & Quattrociocchi, W. (2015) · PLOS ONE · doi:10.1371/journal.pone.0118093 · Cited by 645 · see citing works
Burstiness and memory in complex systems
Goh, K.-I., & Barabási, A.-Ĺ. (2008) · Europhysics Letters (EPL) · doi:10.1209/0295-5075/81/48002 · Cited by 547 · see citing works
The epic battle against coronavirus misinformation and conspiracy theories
Ball, P., & Maxmen, A. (2020) · Nature · doi:10.1038/d41586-020-01452-z · Cited by 286 · see citing works
Epidemiology of vaccine hesitancy in the United States
Siddiqui, M., Salmon, D. A., & Omer, S. B. (2013) · Human Vaccines & Immunotherapeutics · doi:10.4161/hv.27243 · Cited by 248 · see citing works
Moon, R. Y., Mathews, A., Oden, R. P., & Carlin, R. F. (2019) · Journal of Medical Internet Research · doi:10.2196/14289 · Cited by 236 · see citing works
Hidden resilience and adaptive dynamics of the global online hate ecology
Johnson, N. F., Leahy, R., Restrepo, N. J., Velásquez, N., Zheng, M., Manrique, P. D., Devkota, P., & Wuchty, S. (2019) · Nature · doi:10.1038/s41586-019-1494-7 · Cited by 224 · see citing works
Disinformation’s spread: bots, trolls and all of us
Starbird, K. (2019) · Nature · doi:10.1038/d41586-019-02235-x · Cited by 203 · see citing works
Quantifying COVID-19 Content in the Online Health Opinion War Using Machine Learning
Sear, R. F., Velásquez, N., Leahy, R., Restrepo, N. J., Oud, S. E., Gabriel, N., Lupu, Y., & Johnson, N. F. (2020) · IEEE Access · doi:10.1109/access.2020.2993967 · Cited by 133 · see citing works
Self-citationA Knowledge-Enhanced Pretraining Model for Commonsense Story GenerationPreprint
Guan, J., Huang, F., Zhao, Z., Zhu, X., & Huang, M. (2020) · arXiv (Cornell University) · doi:10.48550/arxiv.2001.05139 · Cited by 18 · see citing works
Transition in the waiting-time distribution of price-change events in a global socioeconomic system
Zhao, G., McDonald, M., Fenn, D., et al. (2013) · Physica A Statistical Mechanics and its Applications · doi:10.1016/j.physa.2013.08.036 · Cited by 5 · see citing works
Extreme alien light allows survival of terrestrial bacteria
Johnson, N. F., Zhao, G., Caycedo, F., et al. (2013) · Scientific Reports · doi:10.1038/srep02198 · Cited by 5 · see citing works
Temporal network approach to unraveling collective neuron firings
Ruszczycki, B., Zhao, Z., Johnson, N. F., et al. (2013) · Journal of Complex Networks · doi:10.1093/comnet/cnt017 · Cited by 2 · see citing works
An explanation for the universal 3.5 power-law observed in currency markets
Johnson, N. A., Johnson, N. A., Johnson, N. F., et al. (2016) · Results in Physics · doi:10.1016/j.rinp.2016.01.009 · Cited by 1 · see citing works
Flagged first, then by citations
Research works printed but not linked: 7
- Larson H. A Lack of Information Can Become Misinformation. Nature. 2020;580:306.
- Lazer DMJ, Baum MA, Benkler Y, Berinsky AJ, Greenhill KM, et. al. The science of fake news. Science. 2018;359, 1094-1096.
- Ammari T, Schoenebeck S. "Thanks for Your Interest in Our Facebook Group, but It's Only for Dads": Social Roles of Stay-At-Home Dads. In CSCW '16: Proc. 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing.2016;1363-1375.
- LawsR, Walsh AD, Hesketh KD, Downing KL, Kuswara K, et. al. Differences Between Mothers and Fathers of Young Children in Their Use of the Internet to Support Healthy Family Lifestyle Behaviors: Cross-Sectional StudyJournal of Medical Internet Research. 2019;21:e11454.
- Centola D, Becker J, Brackbill D, Baronchelli D. Experimental Evidence for Tipping Points in Social Convention. Science. 2018;360:1116-1119.
- Hutson M. The Language Machines. Nature. 2021;591:22-25.
- Stiennon N, Ouyang L, Wu J, Ziegler D, Lowe R, et al.Learning to Summarize with Human Feedback.In:LarochelleH, Ranzato M, Hadsell R, Balcan M F, Lin H, editors.In ProceedingsAdvances in Neural Information Processing Systems33. 2020;3008-3021.
Cited by
Sorted by citations
ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · Frontiers in Public Health · doi:10.3389/fpubh.2023.1166120 · Cited by 708 · see citing works
ChatGPT and the Rise of Large Language Models: The New AI-Driven Infodemic Threat in Public Health
De Angelis, L., Baglivo, F., Arzilli, G., Privitera, G. P., Ferragina, P., Tozzi, A. E., & Rizzo, C. (2023) · doi:10.2139/ssrn.4352931 · Cited by 708 · see citing works
- Authors
- Rhys Leahy, R. F. S., J. Restrepo, N., Lupu, Y., & F. Johnson, N.
- Affiliations
- George Washington University (US)
- ORCID
- Lupu, Yonatan
- Venue
- Advances in Artificial Intelligence and Machine Learning
- Published
- 2021
- Volume
- 01
- Issue
- 03
- Pages
- 191-202
- ISSN
- 2582-9793 (ISSN-L)
- OpenAlex
- W4210713476
- Cited by
- 5
- References
- 16
- Citations by year
- 2024: 2 · 2023: 2 · 2022: 1
- FWCI
- 1.73
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Advances in Artificial Intelligence and Machine Learning (publishedVersion)
- Funders
- U.S. Air Force
- Air Force Office of Scientific Research
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Keywords
- AI-generated text detection, GPT-2, COVID-19 misinformation, vaccine misinformation, health misinformation, social media, synthetic text detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
