The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 9 of the 23 sources its claims cite were checked for support.
Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown
Open accessAbstract
Many observers of the current explosion of generative AI worry about its impact on our information environment, with concerns being raised about the increased quantity, quality, and personalization of misinformation. We assess these arguments with evidence from communication studies, cognitive science, and political science. We argue that current concerns about the effects of generative AI on the misinformation landscape are overblown.
Overview
CoChat Summary
TL;DR
The commentary argues that fears about generative AI worsening misinformation are overblown, because misinformation consumption is limited more by demand and finite attention than by supply. It reviews claims about increased quantity, quality, and personalization of AI-generated misinformation and finds existing evidence suggests at best modest effects.
Key findings
- The paper identifies four common worries—increased quantity, quality, personalization, and a fourth argument not addressed in detail—and reviews the first three, arguing they are speculative and that existing research suggests at best modest effects of generative AI on the misinformation landscape.
- On quantity, misinformation already abounds online and is rarely behind paywalls, but the average internet user consumes very little of it; consumption is concentrated in a small portion of very active and vocal users, so increased supply is unlikely to greatly increase diffusion because attention is finite.
- On quality, most people are barely exposed to misinformation and overwhelmingly consume mainstream sources; for generative AI to tip the scales in favor of misinformation, the increase in the appeal of misinformation would have to be 20 to 100 times larger than the increase in the appeal of reliable content (Acerbi et al., 2022).
- On personalization, the infrastructures enabling micro-targeting are not directly impacted by generative AI, evidence on personalized political advertising is mixed with at best small and context-dependent effects, and current LLMs cannot represent the full range of users’ preferences and values or hold direct information about users themselves.
Methods
- This is a commentary/argumentative review, not an empirical study; the authors synthesize evidence from communication studies, cognitive science, and political science.
- No original data or sample are reported; the paper assesses three arguments—quantity, quality, and personalization—and explicitly does not address the fourth argument in detail.
- The main analysis is conceptual: it evaluates claims about generative AI and misinformation against existing literature and reasoning about supply and demand, attention, persuasion, and micro-targeting.
Limitations
- Stated in the paper: the argument mostly relies on evidence about the media environment of wealthy, democratic countries with rich and competitive media ecosystems; less data is available in other countries, and the authors cannot rule out that generative AI might have a larger negative effect there, although it could also have a larger positive effect.
- Evident from the text: it is a commentary with no new empirical data, and conclusions about future effects rest on existing evidence and argument; some discussed possibilities, such as using chatbots to form relationships and then feed misinformation, remain entirely speculative.
- Evident from the text: the fourth argument is not addressed in detail by design, so the review is not comprehensive across all proposed harms.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
7 Valid · 2 Not valid · 9 Not checked · 20 Own results and interpretations
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Yet, despite the quantity and accessibility of misinformation, the average internet user consumes very little of it (e.g., Allen et al., 2020; for review, see Acerbi et al., 2022).”
What the sources say
- Partly supportedcites Allen et al., 2020 sourceExperimental — accuracy not yet measured
The Results say fake-news consumption is a negligible fraction of Americans’ daily information diet, supporting the claim for the kind of misinformation the study measured. But the claim extends that finding to the average internet user and to misinformation generally; the Results say broader forms of misinformation in conventional news were beyond the paper’s analysis. (LLM, conf=0.84)
- Partly supportedcites Acerbi et al., 2022 sourceExperimental — accuracy not yet measured
The Implications section (p. 2) reviews studies finding very low online misinformation consumption in the United States and Europe. It explicitly limits that conclusion to the global north, cautions that it may not hold in the global south, and notes that the estimates cover news sources rather than all forms of misinformation. The unqualified claim about the average internet user is therefore broader than the review supports. (LLM, conf=0.86)
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Instead, most people overwhelmingly consume content from mainstream sources, typically the same handful of popular media outlets (Altay et al., 2022; Guess et al., 2021).”
What the sources say
- Partly supportedcites Altay et al., 2022 sourceExperimental — accuracy not yet measured
The Results (page not available) say the most trustworthy news outlets largely dominated visits and Facebook engagement in the four countries studied. The outlet rankings in RQ1 list popular sources, but do not show that most people typically consume content from the same handful of outlets; the paper measures aggregate news-site traffic and Facebook engagement during the pandemic, not individuals’ overall content consumption. (LLM, conf=0.84)
- Not matchedcites Guess et al., 2021
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“If misinformation is so rare in the information environment of wealthy, democratic countries, it is thanks to the hard work of professionals -journalists, fact checkers, experts, etc. -and to the norms and know-how that have developed over time in these professions (e.g., Paris and Donovan, 2019; Silverman, 2014).”
What the sources say
- Partly supportedcites Paris and Donovan, 2019 sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Not matchedcites Silverman, 2014
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“What makes misinformation consumers special is not that they have privileged access to misinformation but traits that make them more likely to seek out misinformation (Broniatowski et al., 2023; Motta et al., 2023), such as having low trust in institutions or being strong partisans (Osmundsen et al., 2021).”
What the sources say
- Partly supportedcites Broniatowski et al., 2023 sourceExperimental — accuracy not yet measured
The Results section, “Motivated users may still seek out misinformation” (page unavailable), reports that removals may not have dissuaded audiences from seeking antivaccine misinformation; its model suggests demand shifted to remaining pages and groups. This supports the role of motivation, but the paper does not establish that misinformation consumers differ from others because of their traits rather than privileged access. The examples of low institutional trust and strong partisanship are attached to a different citation in the citing sentence. (LLM, conf=0.82)
- Not matchedcites Motta et al., 2023
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Partly supportedcites Osmundsen et al., 2021 sourceExperimental — accuracy not yet measured
In “Do people share news from fake news sources?” (page unavailable), the paper reports that strong party identifiers were more likely to share politically aligned fake-news sources. It does not establish that consumers have low trust in institutions or that they seek out misinformation rather than merely share it; the privileged-access comparison is attributed to other citations in the sentence. (LLM, conf=0.86)
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“Given the rarity of misleading content compared to reliable content, the increase in the appeal of misinformation would have to be 20 to 100 times larger than the increase in the appeal of reliable content for the effects of generative AI to tip the scales in favor of misinformation (Acerbi et al., 2022).”
What the sources say
- Partly supportedcites Acerbi et al., 2022 sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 2 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
hedgedstates a possible cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
17 more claims
Create a free account to see all 38 claims and to verify the 2 references still open.
- It may state two separate claims as one. 13
- Probably not a substantial claim. 7
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Introduction · p. 1
“(Gold & Fisher, 2023)”
§ Bibliography · p. 8
“Gold, A. & Fischer, S. (2023, February 21).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Increased quality of misinformation · p. 4
“(Schiff et al., 2023)”
§ Bibliography · p. 10
“Schiff, K. J., Schiff, D. S., & Bueno, N. (2022, May 11).”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Increased quality of misinformation · p. 4
“(Altay et al., 2022; Guess et al., 2021)”
§ Bibliography · p. 8
“Guess, A. M. (2021). (Almost) everything in moderation: New evidence on Americans' online media diets.”
Citations
Cites
Fake news on Twitter during the 2016 U.S. presidential election
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019) · Science · doi:10.1126/science.aau2706 · Cited by 1,803 · cited 1×
“Instead, misinformation consumption is heavily concentrated in a small portion of very active and vocal users (Grinberg et al. 2019).”
THE SOCIOLOGY OF SCIENCE:: Theoretical and Empirical Investigations
Merton, R. Κ., & Storer, N. W. (2017) · Cited by 1,462
The psychological drivers of misinformation belief and its resistance to correction
Ecker, U. K. H., Lewandowsky, S., Cook, J., Schmid, P., Fazio, L. K., Brashier, N., Kendeou, P., Vraga, E. K., & Amazeen, M. A. (2022) · Nature Reviews Psychology · doi:10.1038/s44159-021-00006-y · Cited by 1,372 · cited 1×
“…Arguably, the major factor deciding the effectiveness of such attempts is not the plausibility of a given technology being able to generate some content but other factors, such as partisanship or people’s preexisting trust in the individual attempting to discredit the evidence (Ecker et al., 2022).”
Metzger, M. J. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20672 · Cited by 871 · cited 1×
“Experimental studies have shown that the credibility of online sources can be affected by such features (e.g., Metzger, 2007), lending support to the argument.”
Art and the science of generative AI
Epstein, Z., Hertzmann, A., the Investigators of Human Creativity, Akten, M., Farid, H., Fjeld, J., Frank, M. R., Groh, M., Herman, L., Leach, N., et al. (2023) · Science · doi:10.1126/science.adh4451 · Cited by 788
OSMUNDSEN, M., BOR, A., VAHLSTRUP, P. B., BECHMANN, A., & PETERSEN, M. B. (2021) · American Political Science Review · doi:10.1017/s0003055421000290 · Cited by 614 · cited 1×
“…misinformation consumers special is not that they have privileged access to misinformation but traits that make them more likely to seek out misinformation (Broniatowski et al., 2023; Motta et al., 2023), such as having low trust in institutions or being strong partisans (Osmundsen et al., 2021).”
Evaluating the fake news problem at the scale of the information ecosystem
Allen, J., Howland, B., Mobius, M., Rothschild, D., & Watts, D. J. (2020) · Science Advances · doi:10.1126/sciadv.aay3539 · Cited by 528 · cited 1×
“Yet, despite the quantity and accessibility of misinformation, the average internet user consumes very little of it (e.g., Allen et al., 2020; for review, see Acerbi et al., 2022).”
(Almost) Everything in Moderation: New Evidence on Americans' Online Media Diets
Guess, A. M. (2021) · American Journal of Political Science · doi:10.1111/ajps.12589 · Cited by 342 · cited 1×
“Instead, most people overwhelmingly consume content from mainstream sources, typically the same handful of popular media outlets (Altay et al., 2022; Guess et al., 2021).”
The Sisyphean Cycle of Technology Panics
Orben, A. (2020) · Perspectives on Psychological Science · doi:10.1177/1745691620919372 · Cited by 316 · cited 1×
“These concerns are part of an old and broad family of moral panics surrounding new technologies (Jungherr & Schroeder, 2021b; Orben, 2020; Simon & Camargo, 2021).”
Not Born Yesterday: The Science of Who We Trust and What We Believe
Mercier, H. (2020) · Social Forces · doi:10.1093/sf/soaa040 · Cited by 289
Self-citation
Research works printed but not linked: 2
- Marcus, G. (2023, February 8). AI's Jurassic Park moment. Communications of the ACM. https://cacm.acm.org/blogs/blog-cacm/267674-ais-jurassic-park-moment/fulltext
- Silverman, C. (Ed.). (2014). Verification handbook: An ultimate guideline on digital age sourcing for emergency coverage. European Journalism Centre. https://datajournalism.com/read/handbook/verification-3
Cited by
Altay, S., & Gilardi, F. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae403 · Cited by 119
Self-citationHow large language models can reshape collective intelligence
Burton, J. W., Lopez-Lopez, E., Hechtlinger, S., Rahwan, Z., Aeschbach, S., Bakker, M. A., Becker, J. A., Berditchevskaia, A., Berger, J., Brinkmann, L. U. L., et al. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-01959-9 · Cited by 238
“149 for counter-arguments).”
Akhtar, Z. B. (2024) · Journal of Electrical Systems and Information Technology · doi:10.1186/s43067-024-00145-1 · Cited by 117
“However, economists like Daron Acemoglu have raised warnings about potential adverse consequences, including data harvesting, customer manipulation, and labor market disparities, underscoring the complex impact of AI on society [6, 23].”
BackgroundHaider, J., Söderström, K. R., Ekström, B., & Rödl, M. (2024) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-156 · Cited by 70
“…intermediaries between science, but also certain types of misinformation, and the public, namely Google Scholar, also in response to the legitimate concerns that the discussion of generative AI and misinformation needs to be more nuanced and empirically substantiated (Simon et al., 2023).”
BackgroundOn the way to deep fake democracy? Deep fakes in election campaigns in 2023
Łabuz, M., & Nehring, C. (2024) · European Political Science · doi:10.1057/s41304-024-00482-9 · Cited by 110
“A recent study proved this by pointing to the “relatively narrow conceptualization and understanding of deepfakes and their impact on society at large in journalistic dis-course” (Weikmann & Lecheler 2023, p. 13).”
Metadata
- Authors
- Simon, F. M., Altay, S., & Mercier, H.
- Affiliations
- University of Oxford (GB)
- University of Zurich (CH)
- Centre National de la Recherche Scientifique (FR)
- Venue
- Harvard Kennedy School Misinformation Review
- Published
- 2023-10-18
- ISSN
- 2766-1652 (ISSN-L)
- OpenAlex
- W4387633104
- Semantic Scholar
- aeddde96937507fb04104017ff5cdf3e29f90fa1
- Cited by
- 85
- References
- 47
- Citations by year
- 2026: 24 · 2025: 33 · 2024: 24 · 2023: 4
- FWCI
- 36.50
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by
- Locations
- Harvard Kennedy School Misinformation Review (publishedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich) (acceptedVersion, cc-by-nc-nd)
- HAL (submittedVersion, other-oa)
- Zurich Open Repository and Archive (University of Zurich) (submittedVersion, cc-by)
- Zurich Open Repository and Archive (University of Zurich)
- Funders
- University of Oxford
- HORIZON EUROPE European Research Council (883121)
- Agence Nationale de la Recherche (ANR-21-CE28-0016-01, ANR-17-EURE-0017, ANR-10-IDEX-0001-02)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Ethics and Social Impacts of AI
- Social Sciences › Communication › Social Media and Politics
- Keywords
- generative AI, misinformation, information environment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-10, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-10 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (gpt-6-sol).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. https://app.cochat.ai/papers/10.37016/mr-2020-127
- The paper
- Simon, F. M., Altay, S., & Mercier, H. (2023). Misinformation reloaded? Fears about the impact of generative AI on misinformation are overblown. Harvard Kennedy School Misinformation Review. https://doi.org/10.37016/mr-2020-127
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.