How malicious AI swarms can threaten democracy
Open accessAbstract
The fusion of agentic AI and LLMs marks a new frontier in information warfare
AI snapshot
Written from the paper’s full text.
TL;DR
The paper argues that malicious AI swarms—coordinated LLM agents—can conduct persistent, adaptive influence operations that threaten democracy through synthetic consensus, fragmentation, harassment, data poisoning, and institutional distrust. It proposes a three-pronged response spanning platform defenses, model safeguards, and a UN-backed AI Influence Observatory.
Key findings
- Identifies five core swarm capabilities: decentralized orchestration, community infiltration, detection evasion, continuous optimization, and persistence; these can radiate into at least nine democratic harms, including synthetic consensus cascades, undermined collective intelligence, fragmented epistemic commons, LLM training-data poisoning, mass harassment, algorithmic overcompensation, FUD-induced disengagement, institutional legitimacy erosion, and engineered norm shifts/mass mobilization.
- Contrasts older botnets with LLM swarms: the Russian Internet Research Agency’s 2016 Twitter operation reached 1% of users with 70% of its content but had no measurable effects on opinions or turnout; LLMs remove prior caps by generating persuasive, tailored text at scale and have shifted deeply held beliefs in laboratory settings.
- Warns that swarms can run millions of micro-A/B tests, propagate winning variants at machine speed, and maintain round-the-clock presence, enabling voter micro-suppression or mobilization and contamination of future AI training data.
- Recommends a three-pronged response: platform-side always-on swarm-detection dashboards, pre-election high-fidelity swarm-simulation stress-tests, transparency audits, and optional client-side “AI shields”; model-side standardized persuasion-risk tests, provenance-authenticating passkeys, and watermarking; and system-level oversight via a UN-backed AI Influence Observatory.
Methods
- This is a conceptual/policy analysis and call to action, not an empirical study; it synthesizes prior literature and proposes governance and technical defenses.
- No primary data, sample, or formal statistical analysis is reported.
Limitations
- The paper does not include a formal limitations section.
- Evident from the text: the argument is forward-looking and speculative about imminent swarm capabilities, with many harms projected rather than empirically demonstrated; it also cites field experiments finding only modest attitude shifts and no measurable IRA effects, which may constrain near-term persuasion claims.
- The paper itself notes limits of proposed fixes: “proof-of-human” alone is no panacea because millions lack formal IDs, biometrics raise privacy risks, and verified accounts can be hijacked; counter-messaging must be calibrated because defensive agents flooding a platform could drown out human voices.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
1 of 27 cited sources checked · 0 supported · 0 flagged · accuracy not yet measured
- Interpretationnone given
A single adversary can operate thousands of AI personas, scheduling content and updating narrative frames instantaneously across an entire fleet of agents.
“A single adversary can operate thousands of AI personas, scheduling content and updating narrative frames instantaneously across an entire fleet of agents.”
Quote in the paperNot a substantial claim - Interpretationcitation
Agent swarms may experience spontaneous or adversarially-induced norm shifts, abandoning their original engineered constraints for unpredictable new behavioral patterns through critical mass dynamics.
“these societies may experience spontaneous or adversarially-induced norm shifts, abandoning their original engineered constraints for unpredictable new behavioral patterns through critical mass dynamics [33].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [33]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
High-fidelity photorealistic avatars, context-aware slang, and heterogeneous posting rhythms circumvent the tell-tale synchrony that older detectors flag.
“High-fidelity photorealistic avatars, context-aware slang, and heterogeneous posting rhythms circumvent the tell-tale synchrony that older detectors flag [39,40].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [39]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [40]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
These engineered realities are designed to keep groups apart except when not desired, making cross-cleavage consensus increasingly unfeasible.
“Unlike passive “filter bubbles,” these engineered realities are designed to keep groups apart except when not desired, making cross-cleavage consensus increasingly unfeasible.”
Quote in the paperNot a substantial claim - Premisecitation
Once initiated, such streams can spread via social contagion, with the effect of bots potentially cascading beyond the people to whom the bots are directly connected, to two or three degrees of separation.
“Once initiated, such streams can also spread via social contagion, with the effect of bots potentially cascading beyond the people to whom the bots are directly connected, to two or three degrees of separation, as is often empirically the case [47].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [47]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
When AI agents in a swarm coordinate to simulate diverse voices across platforms, citizens can overestimate the informational value of this artificial consensus.
“When AI agents in a swarm coordinate to simulate diverse voices across platforms-appearing as distinct citizens, experts, and community leaders while actually representing a singular strategic objective-citizens can overestimate the informational value of this artificial consensus.”
Quote in the paperSubstantial claim - Interpretationreasoning
A coordinated chorus of AI voices erodes the independence that collective intelligence requires, and which democracies rely on.
“This coordinated chorus also erodes the independence that collective intelligence requires, and which democracies rely on.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Algorithmic over-compensation can elevate celebrity voices while sidelining ordinary citizens.
“Algorithmic over-compensation can then elevate celebrity voices while sidelining ordinary citizens.”
Quote in the paperSubstantial claim - Interpretationnone given
Collaborating AI agents can tailor streams of misleading information to each sub-community's linguistic, cultural, and emotional markers, weaving partially overlapping but strategically segmented realities.
“Collaborating AI agents can tailor streams of misleading information to each sub-community's linguistic, cultural, and emotional markers, weaving partially overlapping but strategically segmented realities.”
Quote in the paperNot a substantial claim - Interpretationreasoning
FUD and content saturation could drive users to disengage or retreat into gated channels, shrinking the shared public sphere on which deliberative democracy relies.
“Together, FUD and content saturation could drive users to disengage or retreat into gated channels, shrinking the shared public sphere on which deliberative democracy relies.”
Quote in the paperNot a substantial claim- Rests on an assumption: Genuine users will inevitably be misidentified as bots.
- Interpretationreasoning
A chorus of seemingly independent voices creates a mirage of grassroots consensus with enhanced speed and persuasiveness.
“This chorus of seemingly independent voices creates a mirage of grassroots consensus with enhanced speed and persuasiveness.”
Quote in the paperNot a substantial claim - Interpretationreasoning
Swarms may seed harmonized narratives across disparate niches, creating the illusion of majority agreement within days.
“Swarms may seed harmonized narratives across disparate niches, creating the illusion of majority agreement within days.”
Quote in the paperNot a substantial claim- Rests on an assumption: Citizens update opinions based on perceived peer norms, not evidence.
- Interpretationreasoning
By coordinating subtle yet increasingly profound doubts about electoral commissions, courts, or statistics bureaus, swarms could corrode procedural trust.
“By coordinating subtle yet increasingly profound doubts about electoral commissions, courts, or statistics bureaus, swarms could corrode procedural trust.”
Quote in the paperNot a substantial claim - Premisecitation
Social-graph mapping can also work in a decentralized fashion while still having global, network-wide efficacy.
“This can also work in a decentralized fashion while still having global, network-wide efficacy [36].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [36]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
As trust collapses, fear-uncertainty-doubt (FUD) can drive users into gated channels and silence.
“Generally, as trust collapses, fear-uncertainty-doubt (FUD) can drive users into gated channels and silence.”
Quote in the paperNot a substantial claim - Premisecitation
Methods for detecting coordinated inauthentic behavior generally rely on activity patterns that are suspiciously similar between accounts and therefore statistically unlikely to result from independent behaviors.
“Methods for detecting coordinated inauthentic behavior, such as influence operations, generally rely on activity patterns that are suspiciously similar between accounts and therefore statistically unlikely to result from independent behaviors [37,38].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [37]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [38]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationreasoning
AI swarms can rig the epistemic substrate on which future deliberation and future AI tools will rely, compounding democratic erosion.
“Thus, AI swarms can rig the epistemic substrate on which future deliberation and future AI tools will rely, compounding democratic erosion.”
Quote in the paperSubstantial claim - Premisecitation
The wisdom of crowds phenomenon depends critically on independence between judgments.
“the “wisdom of crowds” phenomenon, where aggregated judgments outperform individual experts, depends critically on independence between judgments [44,45,46].”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [44]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [45]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [46]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
Human-level mimicry lets swarms evade detectors that once caught copy-paste bots.
“human-level mimicry lets swarms evade detectors that once caught copy-paste bots.”
Quote in the paperSubstantial claim - Interpretationcitation
The unprecedented capabilities of AI swarm-driven influence campaigns pose profound threats to democracy by shaping public opinion formation.
“The unprecedented capabilities of AI swarm-driven influence campaigns pose profound threats to democracy by shaping public opinion formation [42,43].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [42]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [43]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationnone given
Continuous A/B testing refines messages at machine speed.
“continuous A/B testing refines messages at machine speed.”
Quote in the paperNot a substantial claim - Premisecitation
These agents could operate on the periphery of social networks, where early mobilization often begins.
“Rather than occupying central or influential positions, these agents could operate on the periphery of social networks, where early mobilization often begins [50].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [50]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Premisecitation
Prosocial media architectures on interoperable platforms can offer structural resilience.
““Prosocial media” architectures on interoperable platforms can offer structural resilience [57].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [57]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- Interpretationcitation
By flooding the web with fabricated chatter, agent swarms can contaminate the data from which future models learn.
“By flooding the web with fabricated chatter, agent swarms can contaminate the data from which future models learn.”
Quote in the paperSubstantial claim - Interpretationreasoning
Attention may concentrate around celebrities and major brands, while ordinary participants fade from view.
“Attention may concentrate around celebrities and major brands, while ordinary participants fade from view.”
Quote in the paperSubstantial claim - Interpretationreasoning
When citizens realize that vast portions of online speech may be AI-generated, baseline trust in social platforms and their users could sharply decline.
“When citizens realize that vast portions of online speech may be AI-generated, baseline trust in social platforms and their users could sharply decline.”
Quote in the paperNot a substantial claim
- May combine two claims. 20
- Classified as probably not a substantial claim. 28
May combine two claims.
InterpretationreasoningAs confidence falters, previously unthinkable emergency measures may become palatable, especially when deep fake endorsements from fabricated civic leaders amplify the call.
“As confidence falters, previously unthinkable “emergency” measures (e.g., postponing elections, rejecting certified results) may become palatable, especially when deep fake endorsements from fabricated civic leaders amplify the call [51,5].”
Quote in the paperSubstantial claim1 of 2 sources checked- cites [51]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [5] source
The Discussion (page unavailable in our copy) suggests that mistrust in the electoral system may be related to willingness to accept claims of voter fraud. It does not address support for postponing elections or rejecting certified results, or deepfake endorsements by fabricated civic leaders. (LLM, conf=0.83)
Experimental — accuracy not yet measured
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone givenLocal adaptation plus periodic hub-sync blurs the line between command-and-control and emergent hive behavior.
“Local adaptation plus periodic hub-sync blurs the line between command-and-control and emergent “hive” behavior.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone givenAn imminent development is the emergence of malicious AI swarms.
“an imminent development is the emergence of malicious AI swarms.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone givenAdvances in AI portend a new era of sophisticated disinformation operations.
“Advances in AI portend a new era of sophisticated disinformation operations.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
Interpretationnone givenSome threat actors may welcome their synthetic interventions being exposed, reasoning that the mere revelation of manipulation can sow as much confusion as successful deception.
“Some threat actors may even welcome their synthetic interventions being exposed, reasoning that the mere revelation of manipulation can sow as much confusion as successful deception.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenSwarms can at a low cost unleash coordinated synthetic harassment campaigns that relentlessly target politicians, dissidents, whistleblowers, journalists, and their family and social networks with overwhelming, psychologically tailored abuse.
“Separate from fragmentation, swarms can at a low cost unleash coordinated synthetic harassment campaigns (i.e., “synthetic shitstorms”) that relentlessly target politicians, dissidents, whistleblowers, journalists, and their family and social networks with overwhelming, psychologically tailored abuse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenIf platforms deploy always-on swarm detectors, frontier labs submit models to standardized persuasion stress tests, and governments launch an AI Influence Observatory that publishes open incident telemetry, we can blunt the most destabilizing tactics before critical political future events, without freezing innovation.
“If platforms deploy always-on swarm detectors, frontier labs submit models to standardized persuasion stress tests, and governments launch an AI Influence Observatory that publishes open incident telemetry, we can blunt the most destabilizing tactics before critical political future events, without freezing innovation.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing).
InterpretationcitationStronger provenance may help societies harden identity without muting speech as persuasive swarms proliferate.
“Stronger provenance may help societies harden identity without muting speech as persuasive swarms proliferate [52].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [52]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable). May combine two claims.
InterpretationreasoningCalibrated use of AI can help build trust before crisis points emerge and ensure that credible, evidence-based information sources are embedded within public discourse, keeping online communities resilient without drowning out authentic participation.
“Such calibrated use of AI can help build trust before crisis points emerge and ensure that credible, evidence-based information sources are embedded within public discourse, keeping online communities resilient without drowning out authentic participation.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationLLMs ingest fabricated chatter; at the next retraining cycle, fabricated narratives calcify inside the model weights.
“LLMs then ingest this chatter; at the next retraining cycle, fabricated narratives calcify inside the model weights [48].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [48]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
Interpretationnone givenPre-bunking campaigns that educate citizens about AI tactics can build cognitive immunity, but the very alertness they foster can still be exploited by threat actors to deepen FUD.
“Pre-bunking campaigns that educate citizens about AI tactics can build cognitive immunity, but paradoxically, the very alertness they foster can still be exploited by threat actors to deepen FUD.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningAgent swarms can persist for protracted durations, embedding themselves within communities over long timescales and gradually shifting discourse.
“agent swarms can persist for protracted durations, embedding themselves within communities over long timescales and gradually shifting discourse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningBy the time monitoring teams distinguish these AI campaigns from organic criticism, targets may have already withdrawn from public life, delivering significant victories for campaign operators while systematically excluding critical voices from democratic discourse.
“By the time monitoring teams distinguish these AI campaigns from organic criticism, targets may have already withdrawn from public life, delivering significant victories for campaign operators while systematically excluding critical voices from democratic discourse.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningSynthetic harassment operations appear as spontaneous public outrage while actually representing orchestrated action by thousands of AI personas adapting dynamically to target responses.
“Unlike conventional trolling, these operations appear as spontaneous public outrage while actually representing orchestrated action by thousands of AI personas adapting dynamically to target responses.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningCoordinated doubt may corrode institutional legitimacy and invite emergency rule.
“Taken to extremes, coordinated doubt may corrode institutional legitimacy and invite “emergency” rule.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
Interpretationnone givenThese capabilities convert into at least five cascading harms.
“These capabilities convert into at least five cascading harms (see Figure 1).”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
Interpretationnone givenThe emergence of sophisticated, collaborative, and increasingly autonomous swarms of AI agents capable of orchestrating influence operations marks a critical juncture.
“The emergence of sophisticated, collaborative, and increasingly autonomous swarms of AI agents capable of orchestrating influence operations marks a critical juncture.”
Quote in the paperNot a substantial claimMay combine two claims.
InterpretationreasoningEquipped with social-graph mapping capabilities, agent swarms can position themselves for maximum impact and tailor messages to the beliefs and cultural cues of each community, enabling more precise targeting than previous botnet approaches.
“Equipped with such capabilities, agent swarms can position themselves for maximum impact and tailor messages to the beliefs and cultural cues of each community, enabling more precise targeting than previous botnet approaches.”
Quote in the paperSubstantial claimMay combine two claims.
PremisecitationAgents now can map social graphs and slip into vulnerable communities with tailored appeals.
“agents now can map social graphs and slip into vulnerable communities with tailored appeals.”
Quote in the paperSubstantial claim0 of 3 sources checked- cites [34]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [35]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [36]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
PremisecitationSwarms can tip latent norms into action or dampen conformity, accelerating anti-democratic political action.
“Swarms can even tip latent norms into action [49] or dampen conformity, accelerating anti-democratic political action [30].”
Quote in the paperSubstantial claim0 of 2 sources checked- cites [49]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [30]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based). May combine two claims.
InterpretationreasoningIterative red-teaming pits hostile swarms against these defenses, mapping persuasion ceilings and long-term persona strategies, and continuously hardening guardrails.
“Iterative red-teaming then pits hostile swarms against these defenses, mapping persuasion ceilings and long-term persona strategies, and continuously hardening guardrails.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningRound-the-clock presence turns influence into a long-term, low-friction infrastructure.
“round-the-clock presence turns influence into a long-term, low-friction infrastructure.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningReal-time monitors are effective only when they anticipate the tactics they will face.
“Real-time monitors are effective only when they anticipate the tactics they will face.”
Quote in the paperNot a substantial claim- Rests on an assumption: Real-time monitors are effective only when they anticipate the tactics they will face.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
Interpretationnone givenReinforcement-learning agents run thousands of messaging experiments per hour, iteratively adjusting content while classifiers mine engagement metrics and textual responses to infer voting intent and gauge each tactic's success.
“Reinforcement-learning agents run thousands of messaging experiments per hour, iteratively adjusting content while classifiers mine engagement metrics and textual responses to infer voting intent and gauge each tactic's success.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningWhen timelines flood with AI-authored posts, both ranking algorithms and users may retreat to obvious trust proxies such as big follower numbers, blue checks, and legacy clout.
“When timelines flood with AI-authored posts, both ranking algorithms and users may retreat to obvious trust proxies-big follower numbers, blue checks, and legacy clout.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S4 (based).
InterpretationreasoningAI agents seeded into synthetic networks can replicate a platform's graph structure, content cadence, and recommender logic, yielding traces that recalibrate live detectors.
“AI agents seeded into synthetic networks can replicate a platform's graph structure, content cadence, and recommender logic, yielding traces that recalibrate live detectors.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningBecause defenders rarely have access to the underlying code or even the decision-making logic of malicious swarms, high-fidelity, agent-based simulation may be the only reliable window into how these systems might behave.
“Because defenders rarely have access to the underlying code or even the decision-making logic of malicious swarms, high-fidelity, agent-based simulation may be the only reliable window into how these systems might behave.”
Quote in the paperNot a substantial claim- Rests on an assumption: Defenders rarely have access to the underlying code or even the decision-making logic of malicious swarms.
Probably not a substantial claim: may not pass S4 (based). May combine two claims.
InterpretationreasoningGiven sufficient signals, swarms can run millions of micro-A/B tests, propagate the winning variants at machine speed, and iterate far faster than human campaigners.
“Given sufficient signals, though, swarms can run millions of micro-A/B tests, propagate the winning variants at machine speed, and iterate far faster than human campaigners.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
Interpretationnone givenThe causal arrow runs both ways: while malicious swarms endanger democratic norms, the quality of democratic governance will in turn shape how potent, or containable, those swarms become.
“The causal arrow runs both ways: while malicious swarms endanger democratic norms, the quality of democratic governance will in turn shape how potent, or containable, those swarms become.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable). May combine two claims.
PremisecitationAs threats evolve, defensive AI agents can strategically disseminate accurate information and pro-democratic counter-narratives, engage constructively with and warn targeted communities, and promote media literacy and resilience at scale.
“As threats evolve, defensive AI agents can strategically disseminate accurate information and pro-democratic counter-narratives, engage constructively with and warn targeted communities, and promote media literacy and resilience at scale [23,54].”
Quote in the paperNot a substantial claim0 of 2 sources checked- cites [23]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
- cites [54]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
May combine two claims.
InterpretationreasoningThe public sphere contracts from many-to-many dialogue to a few-to-many broadcast, a shift that erodes democratic pluralism and encourages further cynicism or migration to closed groups.
“The public sphere contracts from many-to-many dialogue to a few-to-many broadcast, a shift that erodes democratic pluralism and encourages further cynicism or migration to closed groups.”
Quote in the paperSubstantial claimProbably not a substantial claim: may not pass S2 (checkable), S4 (based). May combine two claims.
InterpretationreasoningSimilar strategies can be weaponized for micro-targeted voter suppression or mobilization on behalf of specific parties and causes.
“Similar strategies can be weaponized for micro-targeted voter suppression or mobilization on behalf of specific parties and causes.”
Quote in the paperNot a substantial claimMay combine two claims.
PremisecitationAI agents can employ systems that map social graphs at scale, identify key communities and beliefs, and track trending topics.
“AI agents can employ systems that map social graphs at scale, identify key communities and beliefs, and track trending topics [35].”
Quote in the paperSubstantial claim0 of 1 sources checked- cites [35]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
Probably not a substantial claim: may not pass S2 (checkable), S4 (based).
InterpretationreasoningIn the coming years, democracies have a brief but crucial opportunity to pull AI-enabled influence operations back from the brink.
“In the coming years, democracies have a brief but crucial opportunity to pull AI-enabled influence operations back from the brink.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing), S4 (based).
InterpretationreasoningBy committing now to principled action, upcoming societal decisions could even become a proving ground rather than a casualty of democratic AI governance.
“By committing now to principled action, upcoming societal decisions could even become a proving ground-rather than a casualty-of democratic AI governance.”
Quote in the paperNot a substantial claimProbably not a substantial claim: may not pass S2 (checkable), S3 (load-bearing). May combine two claims.
PremisecitationLike earlier calls to confront emerging bio-risks, success hinges on acting collaboratively without unnecessarily impeding scientific research, while keeping the public square resilient and accountable.
“Like earlier calls to confront emerging bio-risks, success hinges on acting “collaboratively . . .without unnecessarily impeding scientific research,” while keeping the public square resilient and accountable [58].”
Quote in the paperNot a substantial claim0 of 1 sources checked- cites [58]
Couldn’t be matched to a paper in Crossref, so it can’t be checked.
The paper relies on this without support.
Citizens update opinions based on perceived peer norms, not evidence.
“Citizens update opinions based on perceived peer norms, not evidence.”
The paper relies on this without support.
Genuine users will inevitably be misidentified as bots.
“Compounding this, genuine users will inevitably be misidentified as bots, weaponizing false accusations to discredit individuals and intensify FUD.”
The paper relies on this without support.
Millions lack formal IDs, biometrics raise privacy risks, and verified accounts can be hijacked.
“However, “proof-of-human” alone is no panacea: millions lack formal IDs, biometrics raise privacy risks, and verified accounts can be hijacked.”
The paper relies on this without support.
Defenders rarely have access to the underlying code or even the decision-making logic of malicious swarms.
“Because defenders rarely have access to the underlying code or even the decision-making logic of malicious swarms, high-fidelity, agent-based simulation may be the only reliable window into how these systems might behave.”
The paper relies on this without support.
Real-time monitors are effective only when they anticipate the tactics they will face.
“Real-time monitors are effective only when they anticipate the tactics they will face.”
The paper relies on this without support.
In the near term, human operators will still steer agent swarms while letting the agents perform limited in-context adaptation.
“In the near term, human operators will still steer agent swarms while letting the agents perform limited in-context adaptation.”
Cites
Social Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767 · see citing works
Chain-Of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, J., Wang, X., Schuurmans, D., Bosma, M., Ichter, B., Xia, F., Chi, E., Le, Q. V., & Zhou, D. (2022) · Advances in Neural Information Processing Systems 35 · doi:10.52202/068431-1800 · Cited by 4,505 · see citing works
Lazer, D. M. J., Baum, M. A., Benkler, Y., Berinsky, A. J., Greenhill, K. M., Menczer, F., Metzger, M. J., Nyhan, B., Pennycook, G., Rothschild, D., et al. (2018) · Science · doi:10.1126/science.aao2998 · Cited by 4,160 · see citing works
Self-citationGenerative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J., Cai, C. J., Morris, M. R., Liang, P., & Bernstein, M. S. (2023) · Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology · doi:10.1145/3586183.3606763 · Cited by 2,187 · see citing works
A survey on large language model based autonomous agents
Wang, L., Ma, C., Feng, X., Zhang, Z., Yang, H., Zhang, J., Chen, Z., Tang, J., Chen, X., Lin, Y., et al. (2024) · Frontiers of Computer Science · doi:10.1007/s11704-024-40231-1 · Cited by 1,941 · see citing works
Rahwan, I., Cebrian, M., Obradovich, N., Bongard, J., Bonnefon, J.-F., Breazeal, C., Crandall, J. W., Christakis, N. A., Couzin, I. D., Jackson, M. O., et al. (2019) · Nature · doi:10.1038/s41586-019-1138-y · Cited by 1,364 · cited 1× · see citing works
“They can also boost this illusion by liking posts, making narratives appear widely supported.”
Self-citationA survey on large language model (LLM) security and privacy: The Good, The Bad, and The Ugly
Yao, Y., Duan, J., Xu, K., Cai, Y., Sun, Z., & Zhang, Y. (2024) · High-Confidence Computing · doi:10.1016/j.hcc.2024.100211 · Cited by 1,083 · see citing works
The rise and potential of large language model based agents: a survey
Xi, Z., Chen, W., Guo, X., He, W., Ding, Y., Hong, B., Zhang, M., Wang, J., Jin, S., Zhou, E., et al. (2025) · Science China Information Sciences · doi:10.1007/s11432-024-4222-0 · Cited by 970 · see citing works
Experimental evidence for tipping points in social convention
Centola, D., Becker, J., Brackbill, D., & Baronchelli, A. (2018) · Science · doi:10.1126/science.aas8827 · Cited by 750 · see citing works
Self-citationHow do social media feed algorithms affect attitudes and behavior in an election campaign?
Guess, A. M., Malhotra, N., Pan, J., Barberá, P., Allcott, H., Brown, T., Crespo-Tenorio, A., Dimmery, D., Freelon, D., Gentzkow, M., et al. (2023) · Science · doi:10.1126/science.abp9364 · Cited by 363 · see citing works
Durably reducing conspiracy beliefs through dialogues with AI
Costello, T. H., Pennycook, G., & Rand, D. G. (2024) · Science · doi:10.1126/science.adq1814 · Cited by 354 · see citing works
Self-citationManaging extreme AI risks amid rapid progress
Bengio, Y., Hinton, G., Yao, A., Song, D., Abbeel, P., Darrell, T., Harari, Y. N., Zhang, Y.-Q., Xue, L., Shalev-Shwartz, S., et al. (2024) · Science · doi:10.1126/science.adn0117 · Cited by 349 · see citing works
Self-citationThe Critical Periphery in the Growth of Social Protests
Barberá, P., Wang, N., Bonneau, R., Jost, J. T., Nagler, J., Tucker, J., & González-Bailón, S. (2015) · PLOS ONE · doi:10.1371/journal.pone.0143611 · Cited by 348 · see citing works
Locally noisy autonomous agents improve global human coordination in network experiments
Shirado, H., & Christakis, N. A. (2017) · Nature · doi:10.1038/nature22332 · Cited by 288 · see citing works
Self-citationBail, C. A., Guay, B., Maloney, E., Combs, A., Hillygus, D. S., Merhout, F., Freelon, D., & Volfovsky, A. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1906420116 · Cited by 224 · see citing works
Artificial Intelligence and Democracy: A Conceptual Framework
Jungherr, A. M. (2023) · Social Media + Society · doi:10.1177/20563051231186353 · Cited by 217 · see citing works
Ong, J. C., & Cabañes, J. V. A. (2019) · Goldsmiths (University of London) · doi:10.7275/2cq4-5396 · Cited by 217 · see citing works
MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework
Hong, S., Zhuge, M., Chen, J., Zheng, X., Cheng, Y., Zhang, C., Wang, J., Wang, Z., Yau, S. K. S., Lin, Z., et al. (2023) · arXiv · doi:10.48550/arxiv.2308.00352 · Cited by 214 · see citing works
Anticipating and Addressing the Ethical Implications of Deepfakes in the Context of Elections
Diakopoulos, N. A., & Johnson, D. K. (2019) · New Media & Society · doi:10.1177/1461444820925811 · Cited by 210 · see citing works
Uncovering Coordinated Networks on Social Media: Methods and Case Studies
Pacheco, D., Hui, P.-M., Torres-Lugo, C., Truong, B. T., Flammini, A., & Menczer, F. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18075 · Cited by 140 · see citing works
Self-citationHow persuasive is AI-generated propaganda?
Goldstein, J. A., Chao, J., Grossman, S., Stamos, A., & Tomz, M. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae034 · Cited by 133 · see citing works
Brinkmann, L., Baumann, F., Bonnefon, J.-F., Derex, M., Müller, T. F., Nussberger, A.-M., Czaplicka, A., Acerbi, A., Griffiths, T. L., Henrich, J., et al. (2023) · Nature Human Behaviour · doi:10.1038/s41562-023-01742-2 · Cited by 132 · see citing works
Resilience to Online Censorship
Roberts, M. E. (2020) · Annual Review of Political Science · doi:10.1146/annurev-polisci-050718-032837 · Cited by 131 · see citing works
Eady, G., Paskhalis, T., Zilinsky, J., Bonneau, R., Nagler, J., & Tucker, J. A. (2023) · Nature Communications · doi:10.1038/s41467-022-35576-9 · Cited by 125 · see citing works
S3: Social-network Simulation System with Large Language Model-Empowered Agents
Gao, C., Lan, X., Lu, Z., Mao, J., Piao, J., Wang, H., Jin, D., & Li, Y. (2023) · doi:10.2139/ssrn.4607026 · Cited by 113 · see citing works
Confronting risks of mirror life
Adamala, K. P., Agashe, D., Belkaid, Y., Bittencourt, D. M. D. C., Cai, Y., Chang, M. W., Chen, I. A., Church, G. M., Cooper, V. S., Davis, M. M., et al. (2024) · Science · doi:10.1126/science.ads9158 · Cited by 91 · see citing works
Emergent social conventions and collective bias in LLM populations
Ashery, A. F., Aiello, L. M., & Baronchelli, A. (2025) · Science Advances · doi:10.1126/sciadv.adu9368 · Cited by 81 · cited 1× · see citing works
“This chorus erodes the independence essential to collective intelligence and democracy, already weakened by pervasive social influence operations on contemporary platforms.”
Self-citationMuennighoff, N., Yang, Z., Shi, W., Li, X. L., Fei-Fei, L., Hajishirzi, H., Zettlemoyer, L., Liang, P., Candes, E., & Hashimoto, T. (2025) · Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/2025.emnlp-main.1025 · Cited by 76 · see citing works
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71 · see citing works
Self-citationNetwork Engineering Using Autonomous Agents Increases Cooperation in Human Groups
Shirado, H., & Christakis, N. A. (2020) · iScience · doi:10.1016/j.isci.2020.101438 · Cited by 48 · see citing works
Self-citationInduction of social contagion for diverse outcomes in structured experiments in isolated villages
Airoldi, E. M., & Christakis, N. A. (2024) · Science · doi:10.1126/science.adi5147 · Cited by 46 · see citing works
Self-citationCalderón, C. A., Holgado, P. S., Gómez, J., Barbosa, M., Qi, H., Matilla-Molina, A., Amado, P., Guzmán, A., López-Matías, D., & Fernández-Villazala, T. (2024) · Humanities and Social Sciences Communications · doi:10.1057/s41599-024-03899-1 · Cited by 43 · see citing works
Addressing the harms of AI-generated inauthentic content
Menczer, F., Crandall, D., Ahn, Y.-Y., & Kapadia, A. (2023) · Nature Machine Intelligence · doi:10.1038/s42256-023-00690-w · Cited by 42 · see citing works
Self-citationLi, W., Hu, Y., Jiang, C., et al. (2023) · Applied Soft Computing · doi:10.1016/j.asoc.2023.110062 · Cited by 38 · see citing works
Generative propaganda: Evidence of AI’s impact from a state-backed disinformation campaign
Wack, M., Ehrett, C., Linvill, D. L., & Warren, P. L. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf083 · Cited by 36 · see citing works
Characteristics and Prevalence of Fake Social Media Profiles with AI-generated Faces
Yang, K.-C., Singh, D., & Menczer, F. (2024) · Journal of Online Trust and Safety · doi:10.54501/jots.v2i4.197 · Cited by 32 · see citing works
Self-citationBaronchelli, A. (2024) · Philosophical Transactions of the Royal Society B: Biological Sciences · doi:10.1098/rstb.2023.0028 · Cited by 29 · see citing works
Self-citationA language framework for modeling social media account behavior
Nwala, A. C., Flammini, A., & Menczer, F. (2023) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00410-9 · Cited by 21 · see citing works
Self-citationQuantifying the vulnerabilities of the online public square to adversarial manipulation tactics
Truong, B. T., Lou, X., Flammini, A., & Menczer, F. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae258 · Cited by 18 · see citing works
Self-citationWilliams, A. R., Burke-Moore, L., Chan, R. S.-Y., Enock, F. E., Nanni, F., Sippy, T., Chung, Y.-L., Gabašová, E., Hackenburg, K., & Bright, J. (2025) · PLoS ONE · doi:10.1371/journal.pone.0317421 · Cited by 17 · see citing works
TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks
Xu, F. F., Song, Y., Li, B., Tang, Y., Jain, K., Bao, M., Wang, Z. Z., Zhou, X., Guo, Z., Cao, M., et al. (2024) · arXiv · doi:10.48550/arxiv.2412.14161 · Cited by 17
Evolving AI Collectives to Enhance Human Diversity and Enable Self-RegulationPreprint
Lai, S., Potter, Y., Kim, J., et al. (2024) · arXiv.org · doi:10.48550/arxiv.2402.12590 · Cited by 13 · see citing works
Schipper, T. (2025) · Data & Policy · doi:10.1017/dap.2025.18 · Cited by 12 · see citing works
Training Language Models with Language Feedback
Scheurer, J., Campos, J. A., Chan, J. S., Chen, A., Cho, K., & Perez, E. (2022) · arXiv · doi:10.48550/arxiv.2204.14146 · Cited by 11 · see citing works
Individuals, Collectives, and Individuals in Collectives: The Ineliminable Role of Dependence
Hahn, U. (2023) · Perspectives on Psychological Science · doi:10.1177/17456916231198479 · Cited by 9 · see citing works
Adler, S., Hitzig, Z., Jain, S., Brewer, C., Chang, W., DiResta, R., Lazzarin, E., McGregor, S., Seltzer, W., Siddarth, D., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.07892 · Cited by 8 · see citing works
How will advanced AI systems impact democracy?Preprint
Summerfield, C., Argyle, L. P., Bakker, M. A., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2409.06729 · Cited by 5 · see citing works
Self-citationAI agents can coordinate beyond human scale
De Marzo, G., Castellano, C., & Garcia, D. (2024) · arXiv · doi:10.48550/arxiv.2409.02822 · Cited by 4 · see citing works
Self-citationPilditch, T. D., Hahn, U., & Lagnado, D. A. (2025) · Synthese · doi:10.1007/s11229-025-04969-w · Cited by 2 · see citing works
Scaling Trends for Data Poisoning in LLMsPreprint
Bowen, D., Murphy, B., Cai, W., et al. (2024) · arXiv (Cornell University) · doi:10.48550/arxiv.2408.02946 · Cited by 1 · see citing works
Biased AI Writing Assistants Shift Users’ Attitudes on Societal IssuesPreprint
Williams‐Ceci, S., Jakesch, M., Bhat, A., et al. (2025) · doi:10.31234/osf.io/mhjn6_v2 · Cited by 1 · see citing works
Community by DesignPreprint
Weyl, E. G., Thorburn, L., de Keulenaar, E. V., et al. (2025) · arXiv (Cornell University) · doi:10.48550/arxiv.2502.10834
Flagged first, then by citations
Research works printed but not linked: 2
- M. Wack, C. Ehrett, D. Linvill, P. Warren, Proc. Natl. Acad. Sci. U.S.A. Nexus 4, pgaf083 (2025).
- B. T. Truong, X. Lou, A. Flammini, F. Menczer, Proc. Natl. Acad. Sci. U.S.A. Nexus 3, 258 (2024).
Cited by
Still being checked.
- Authors
- Schroeder, D. T., Cha, M., Baronchelli, A., Bostrom, N., Christakis, N. A., Garcia, D., Goldenberg, A., Kyrychenko, Y., Leyton-Brown, K., Lutz, N., Marcus, G., Menczer, F., Pennycook, G., Rand, D. G., Ressa, M., Schweitzer, F., Song, D., Summerfield, C., Tang, A., Van Bavel, J. J., van der Linden, S., & Kunst, J. R.
- Affiliations
- SINTEF Digital
- Max Planck Institute for Security and Privacy (DE)
- City St George's, University of London (GB)
- Macrostrategy Research Initiative, London, England, UK.
- Yale University (US)
- Department of Politics and Public Administration, University of Konstanz, Konstanz, Germany.
- Harvard University (US)
- University of Cambridge (GB)
- Department of Computer Science, University of British Columbia, Vancouver, BC, Canada.
- University of Washington (US)
- New York University (US)
- Indiana University Bloomington (US)
- Cornell University (US)
- Columbia University (US)
- ETH Zurich (CH)
- University of California, Berkeley (US)
- University of Oxford (GB)
- Ministry of Foreign Affairs (TW)
- Department of Psychology, New York University, New York City, NY, USA.
- Department of Psychology, University of Cambridge, Cambridge, England, UK.
- BI Norwegian Business School (NO)
- ORCID
- Venue
- Science
- Published
- 2026-01-22
- Volume
- 391
- Issue
- 6783
- Pages
- 354-357
- ISSN
- 0036-8075 (ISSN-L)
- 1095-9203
- OpenAlex
- W7125492504
- Semantic Scholar
- b1c63b2cc33f9040c68a213d567d7670bcc16f5b
- Cited by
- 25
- References
- 52
- Citations by year
- 2026: 24 · 2025: 1
- FWCI
- 309.71
- Type
- journal article
- Language
- en
- Open access
- green
- nbn-resolving.de
- Locations
- Science (publishedVersion)
- PubMed (publishedVersion)
- KOPS (University of Konstanz) (submittedVersion)
- City Research Online (City University London) (submittedVersion)
- OSF Preprints (OSF Preprints) (submittedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Safety Research › Ethics and Social Impacts of AI
- Physical Sciences › Computer Science › Artificial Intelligence › Adversarial Robustness in Machine Learning
- Keywords
- agentic AI, large language models, information warfare, threats to democracy, AI security
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Claims read by deepseek-v4.1-flash under definition version 3 (extract-v3) on 2026-10-10.
Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-10-10.
Cited sources are checked by gpt-6-sol.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review