AI model GPT-3 (dis)informs us better than humans
Open accessThe DOI matches its Crossref record; 26 of the 26 references it cites were found in a registry or an index.
Abstract
Artificial intelligence (AI) is changing the way we create and evaluate information, and this is happening during an infodemic, which has been having marked effects on global health. Here, we evaluate whether recruited individuals can distinguish disinformation from accurate information, structured in the form of tweets, and determine whether a tweet is organic or synthetic, i.e., whether it has been written by a Twitter user or by the AI model GPT-3. The results of our preregistered study, including 697 participants, show that GPT-3 is a double-edge sword: In comparison with humans, it can produce accurate information that is easier to understand, but it can also produce more compelling disinformation. We also show that humans cannot distinguish between tweets generated by GPT-3 and written by real Twitter users. Starting from our results, we reflect on the dangers of AI for disinformation and on how information campaigns can be improved to benefit global health.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Figures




Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Finally, AI-generated misinformation influences attitudes and decisions (Kim et al. 2023; Spitale et al. 2023).”
From Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation (2026) · cites this paper as Spitale et al. 2023
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“As such, challenges arise as, by now, outputs are indistinguishable from authentic content and may present misinformation or deceive users (Spitale et al. 2023).”
From Generative AI (2024) · cites this paper as Spitale et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“it was revealed that GPT-3 is a double-edged sword: it produces accurate information that is easier to understand than human-written content but also generates disinformation that is more compelling and harder to detect.”
From Generative AI and misinformation: a scoping review of the role of generative AI in the generation, detection, mitigation, and impact of misinformation (2026) · cites this paper as Spitale et al. 2023
The cited source says what the claim attributes to it.
In the citing paper
“and vast amounts of compelling mis/disinformation with little human involvement (Buchanan et al., 2021;Spitale et al., 2023).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Spitale et al., 2023
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Language Models are Few-Shot Learners
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al. (2020) · arXiv · doi:10.48550/arxiv.2005.14165 · Cited by 23,372
Kruger, J., & Dunning, D. (1999) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.77.6.1121 · Cited by 7,731 · cited 1×
“An alternative view is proposed by the Dunning-Kruger effect, which can also help in interpreting our findings ( 25 , 26 ).”
Cook, J., Lewandowsky, S., & Ecker, U. K. H. (2017) · PLOS ONE · doi:10.1371/journal.pone.0175799 · Cited by 971
Dale, R. (2021) · Natural Language Engineering · doi:10.1017/s1351324920000601 · Cited by 432 · cited 2×
“GPT-3, the latest iteration of the generative pretrained trans-formers developed by OpenAI, is arguably the most advanced system of pretrained language representations ( 2 ).”
Plagiarism in the age of massive generative pre-trained transformers (GPT-3)
Dehouche, N. (2021) · Ethics in Science and Environmental Politics · doi:10.3354/esep00195 · Cited by 296
All That’s ‘Human’ Is Not Gold: Evaluating Human Evaluation of Generated Text
Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021) · Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · doi:10.18653/v1/2021.acl-long.565 · Cited by 290 · cited 1×
“In line with previous research ( 21 ), we found that both respondents and GPT-3 were not able to distinguish whether a tweet was organic or synthetic (data on GPT-3 ’ s assessment are available in the study ’ s repository) ( 15 ).”
All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text
Clark, E., August, T., Serrano, S., Haduong, N., Gururangan, S., & Smith, N. A. (2021) · arXiv · doi:10.48550/arxiv.2107.00061 · Cited by 290 · cited 1×
“In line with previous research ( 21 ), we found that both respondents and GPT-3 were not able to distinguish whether a tweet was organic or synthetic (data on GPT-3 ’ s assessment are available in the study ’ s repository) ( 15 ).”
Cabanac, G., Labbé, C., & Magazinov, A. (2021) · arXiv · doi:10.48550/arxiv.2107.06751 · Cited by 224 · cited 1×
“The system relies on statistical representations of language for how it is used in real-life by real humans or “ a simulacrum of the interaction between people and the world ” ( 4 ).”
Mahmood, K., & University of the Punjab (2016) · Comminfolit · doi:10.15760/comminfolit.2016.10.2.24 · Cited by 212
Can GPT-3 pass a writer’s turing test?
Elkins, K., & Chun, J. (2020) · Journal of Cultural Analytics · doi:10.22148/001c.17212 · Cited by 202 · cited 1×
“…tool for machine translations, text classification, dialogue/chatbot systems, knowledge summarizing, question answering, creative writing ( 2 , 5 , 6 ), detecting hate speech ( 7 ), and automatic code writing ( 2 , 8 ), it can also be used to produce “ misinformation, spam, phishing, abuse of…”
Research works printed but not linked: 1
- T. Rosenstiel J. Sonderman K. Loker M. Ivancin N. Kjarval “How people use Twitter in general.” American Press Institute (2015); www.americanpressinstitute.org/publications/reports/survey-research/how-people-use-twitter-in-general/.
Cited by
Items 201–245 of 245
Al-Muhammed, M. J. (2025) · Knowledge-Based Systems · doi:10.1016/j.knosys.2025.115199 · Cited by 1
When AI Takes a Seat at the Desk: Innovating Business Education
Baldazo, M. G. (2026) · Springer eBooks · doi:10.1007/978-3-032-08213-8_16 · Cited by 1
Generative AI in Consumer Behaviour and Marketing: Applications, Opportunities and Challenges
Dar, A. R., & Gul, M. (2025) · Contributions to management science · doi:10.1007/978-3-032-02526-5_14
Bring Wisdom into AI Education (Before it’s Too Late)
Williams‐Ceci, S., & Williams, W. M. (2026) · Roeper Review · doi:10.1080/02783193.2025.2593014
“One recent example is a survey by Spitale et al. (2023), in which they asked participants to judge the accuracy (true/false) and source (human/AI) of several tweets on misinforma-tion-prone issues, such as Covid-19.”
Potencjał sztucznej inteligencji (AI) w budowaniu skutecznych narracji politycznych
Darmoń, K., & Marmola, M. (2025) · Studia Politicae Universitatis Silesiensis · doi:10.31261/spus.22851
Recent Trends in AI Law and Ethics and Their Implications for Military and Humanitarian Healthcare
März, J. W., & Biller‐Andorno, N. (2026) · Military and humanitarian health ethics · doi:10.1007/978-3-032-11331-3_3
Self-citationTrust in Falsehoods: Definitions, Sources, Challenges
Šoć, A. (2026) · Springer eBooks · doi:10.1007/978-3-031-96790-0_2
Large language models show Dunning-Kruger-like effects in multilingual fact-checking
Qazi, I. A., Khan, Z., Ghani, A., Raza, A. A., Qazi, I. A., Sajjad, W., Ali, A., Javaid, A., Sohail, M. A., & Azeemi, A. H. (2026) · Scientific Reports · doi:10.1038/s41598-026-39046-w · Cited by 10
“This risk is heightened by the fact that AI-generated text is often indistinguishable from human-written content, while people consistently overestimate their ability to detect it 23–25 .”
Schück, F. (2026) · Frontiers in Water · doi:10.3389/frwa.2026.1749745
“Spitale et al. (2023) called this combination of certainty and incorrectness compelling disinformation and showed that GPT-3 is better at creating believable disinformation than humans.”
Von Resilienz zu Vigilanz. Über die Verantwortung von Maschinen
Hummert, C. (2026) · Springer eBooks · doi:10.1007/978-3-662-72661-7_3
The Impact of AI‐Generated Content on Decision Making for Topics Requiring Expertise
Li, S., Chen, T., & Demartini, G. (2026) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/3410621
Breaking the Imitation Game: Can LLMs Fool Humans and Machines Alike?
Ullah, U., Laudanna, S., Sorbo, A. D., & Visaggio, C. A. (2026) · International Journal of Intelligent Systems · doi:10.1155/int/8117975 · Cited by 1
More Than Human: The Capital-Market Effects of AI-Generated Corporate DisclosurePreprint
Chen, M. A., & Wang, J. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5959496
Shao, A., & Scheufele, D. A. (2026) · Journal of Broadcasting & Electronic Media · doi:10.1080/08838151.2026.2669539 · Cited by 2
On not being a robot: AI as a threat to rational agency
Sparrow, R., & Flenady, G. E. (2026) · Inquiry · doi:10.1080/0020174x.2026.2673086
Silenced by design: Marginal voices and algorithmic bias in AI-generated climate discourse
Brancaccio, M. (2026) · Discourse & Society · doi:10.1177/09579265261449258
“…have become influential intermediaries in shaping public understandings of climate change, as their outputs are increasingly being used in producing scientific explanations, and in informing ethical debates and policy framing (Fore et al., 2024; Leippold et al., 2025; Spitale et al., 2023).”
ÜRETKEN YAPAY ZEKADA SİBER GÜVENLİK VE ETİK: CHATGPT ÖRNEĞİ
Gürsoy, S., & Yalçın, N. (2026) · Güvenlik Bilimleri Dergisi · doi:10.28956/gbd.1707204
From AI-generated content to agentic action: Security and safety threats in generative AI
Zhang, Z., Li, Q., Cao, J., Liu, L., & Ni, J. (2026) · Journal of Information and Intelligence · doi:10.1016/j.jiixd.2026.05.002 · Cited by 4
“…is at least as effective as human-written content at changing beliefs: GPT-3-generated content was recognized as accurate more frequently than human-authored disinformation [121], and a 27-country study ( 𝑁 =27,000) confirmed higher perceived veracity and sharing intent for AI-generated fake news.”
Digital Maturity Understanding Human–AI Interaction in Online Social Media
Valdez, A. S. C. (2026) · Springer eBooks · doi:10.1007/978-3-658-49118-5_9
الكفاية التشريعية لمواجهة الجرائم الناشئة عن التزييف العميق: دراسة تحليلية مقارنة
Fouad, A. A. (2026) · ARAB JOURNAL FOR SECURITY STUDIES · doi:10.26735/uqhp9428
Distinguishing Human and AI-Generated Text: A BiLSTM-Based Deep Learning Approach
Sumaya, S. T., Arnob, H. I., Ahmed, A., Sultana, A., & Rifat, A. A. S. (2026) · IEEE International Conference on Quantum Photonics, Artificial Intelligence & Networking (QPAIN) · doi:10.1109/qpain69676.2026.11546461
“[14], [15] Humans vs Machine Prediction System: Spitale et al. suggest that GPT-3 may be able to provide better and more relevant content than human beings do and that raises doubts about detection systems.”
Hybrid Tri-Layer Framework for Robust AI-Generated Text Detection in Spanish
Sanjurjo-González, H., & Bringas, P. G. (2026) · Lecture notes in computer science · doi:10.1007/978-3-032-29292-6_20
Synthetic social media creation for social-cybersecurity training
Hicks, M., Farag, M., & Carley, K. M. (2026) · Computational and Mathematical Organization Theory · doi:10.1007/s10588-026-09421-z
“Spitale et al. (2023 ) This is reflected in bot detection moving beyond just text analysis and requir - ing meta-data and network analysis for bot detection.”
Algorithms of trust in the political and economic development of Central Asia
Mendybayev, B., BURBAYEVA, P., & Ainayeva, A. (2026) · Central Asian Economic Review · doi:10.52821/2789-4401-2026-2-17-34
Arias, C. F., & Silvente, V. B. (2026) · Edutec Revista Electrónica de Tecnología Educativa · doi:10.21556/edutec.2026.96.4385
Postmodern epistemik galat paradoksu: Üretken yapay zekâ kullanımı için bir etiketleme çerçevesi
Öncü, S. E., Bozkurt, A., Elibol, H., & Özer, F. (2026) · Açıköğretim Uygulamaları ve Araştırmaları Dergisi · doi:10.51948/auad.1922925
Zhang, Z. W., Morgan, A. J., Armstrong, G., & Reavley, N. J. (2026) · Human Behavior and Emerging Technologies · doi:10.1155/hbe2/9069016
Zhu, Y., Zhang, B., Gong, M., Wang, G., Xiong, B., & Xie, C. (2026) · 中国科学基金 · doi:10.3724/bnsfc-2025-0267 · Cited by 1
When AI speaks: user imaginaries of generative AI and freedom of expression
Brandtzæg, P. B., & Wang, X. (2026) · AI & Society · doi:10.1007/s00146-026-03188-2
“The sophisticated realism of AI-generated content may also accelerate the scale of misinformation and deepfakes (Lim 2023; Spitale et al. 2023; Zhou et al. 2023).”
Khazova, N., & Artemenko, E. D. (2026) · Virtual Communication and Social Networks · doi:10.21603/vcsn-2026-5-2-111-122
Research Progress on Rumor Propagation Models and Simulation Methods
徐, 嘉. (2026) · Modeling and Simulation · doi:10.12677/mos.2026.157111
Threats to Russian values in relation to digital sovereignty
Agafonov, A. S. (2026) · Učënye zapiski Kazanskogo universiteta. Seriâ Gumanitarnye nauki/Učenye zapiski Kazanskogo gosudarstvennogo universiteta. Seriâ Gumanitarnye nauki/Učenye zapiski Kazanskogo universiteta. Seriâ Gumanitarnye nauki · doi:10.26907/2541-7738.2026.3.173-187
When AI Talks Back: Behavioral Consequences of AI-Generated Social Interaction on PlatformsPreprint
Li, B., & Lin, Z. (2026) · SSRN Electronic Journal · doi:10.2139/ssrn.7041298
López, R. T. (2026) · Lex Social Revista de Derechos Sociales · doi:10.46661/lexsocial.13868
Generative AI may bias crop protection advice away from agroecological approaches
Wyckhuys, K. A. G., Dangles, O. J., Krupke, C. H., Sánchez‐Bayo, F., Tooker, J. F., & den Brink, P. J. V. (2026) · Journal of Environmental Management · doi:10.1016/j.jenvman.2026.130669
Perspective of the Counter Disinformation System Development in Ukraine
Marushchak, A., Khoperiya, A., Balaban, M., & Petrov, S. (2026) · Terrorism, security and computation · doi:10.1007/978-3-032-24731-5_9
Five Myths About Influence Operations: What 25 Million Tweets Across Seven State Campaigns Reveal
Ferrara, E. (2026) · Systems · doi:10.3390/systems14080970 · Cited by 1
The fallacy of AI-generated podcasts: the case of Google NotebookLM
Musi, E., Harris, R., & Yates, S. J. (2026) · Discourse Context & Media · doi:10.1016/j.dcm.2026.101049
Teichmann, F. M. J. (2026) · International Cybersecurity Law Review · doi:10.1365/s43439-026-00174-8
“…converges with the broader experimental literature on machine-generated content: in a preregistered study, text produced by a large language model was not only indistinguishable from human-written text but, in the case of disinformation, more compelling than its human-written counterpart [18].”
Alluri, N., Gottumukkala, P. S. V., & Indukuri, H. (2026) · AI · doi:10.3390/ai7080319
Leveling the Playing Field? Generative AI and Entrepreneurial Storytelling
Taeuscher, K., & Lounsbury, M. D. (2026) · Entrepreneurship Theory and Practice · doi:10.1177/10422587261472523 · Cited by 1
AI-enabled health communication to address misinformation on social media: a mini review
Hu, Y. (2026) · Frontiers in Communication · doi:10.3389/fcomm.2026.1907956
Teaching What the Machine Told Us: Critical Agency and Epistemic Trust in the Algorithmic Classroom
Genç, L. G., & Uygun, E. (2026) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-2600-1232-1.ch014
Kumar, A. (2026) · Journal of Knowledge Management · doi:10.1108/jkm-03-2026-0627 · Cited by 1
Xian, R. P., Lee, A. J., Lolla, S., Wang, V., Cui, Q., Ro, R., & Abbasi-Asl, R. (2024) · arXiv · doi:10.48550/arxiv.2402.10527
“They are increasingly used in knowledge-intensive tasks by the general public without sufficient assessment of inherent issues in trustworthiness and safety (Anonymous, 2023; Spitale et al., 2023; Si et al., 2023).”
Background
Metadata
- Authors
- Spitale, G., Biller-Andorno, N., & Germani, F.
- Affiliations
- University of Zurich (CH)
- Institute of Biomedical Ethics and History of Medicine, University of Zurich, Zurich, Switzerland.
- Corresponding author
- Germani, Federico
- Venue
- Science Advances
- Published
- 2023-06-30
- Volume
- 9
- Issue
- 26
- Pages
- eadh1850-eadh1850
- ISSN
- 2375-2548 (ISSN-L)
- PMC
- PMC10306283
- OpenAlex
- W4382361534
- Cited by
- 241
- References
- 23
- Citations by year
- 2026: 64 · 2025: 89 · 2024: 65 · 2023: 23
- FWCI
- 103.56
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by-nc
- Locations
- Science Advances (publishedVersion, cc-by-nc)
- PubMed Central (publishedVersion, cc-by-nc)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Keywords
- GPT-3, disinformation, synthetic content detection, AI-generated text, infodemic, information campaigns, global health communication
- MeSH terms
- Artificial Intelligence, Humans, Social Media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.