The DOI matches its Crossref record; 35 of the 46 references it cites were found in a registry or an index.
Political Astroturfing on Twitter: How to Coordinate a Disinformation Campaign
Open accessAbstract
Political astroturfing, a centrally coordinated disinformation campaign in which participants pretend to be ordinary citizens acting independently, has the potential to influence electoral outcomes and other forms of political behavior. Yet, it is hard to evaluate the scope and effectiveness of political astroturfing without “ground truth” information, such as the verified identity of its agents and instigators. In this paper, we study the South Korean National Information Service’s (NIS) disinformation campaign during the presidential election in 2012, taking advantage of a list of participating accounts published in court proceedings. Features that best distinguish these accounts from regular users in contemporaneously collected Twitter data are traces left by coordination among astroturfing agents, instead of the individual account characteristics typically used in related approaches such as social bot detection. We develop a methodology that exploits these distinct empirical patterns to identify additional likely astroturfing accounts and validate this detection strategy by analyzing their messages and current account status. However, an analysis relying on Twitter influence metrics shows that the known and suspect NIS accounts only had a limited impact on political social media discussions. By using the principal-agent framework to analyze one of the earliest revealed instances of political astroturfing, we improve on extant methodological approaches to detect disinformation campaigns and ground them more firmly in social science theory.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“For Keller et al. (2020), another key characteristic of astroturfing is its reliance on centralized coordination, especially for disseminating and sharing information”
From Beyond Verification| Inauthentic Authenticity: Astroturfing to Generate True Affect Using Fakery (2026) · cites this paper as Keller et al., 2020
In the citing paper
“Astroturfing is most frequently equated with political or commercial communications strategy”
From Beyond Verification| Inauthentic Authenticity: Astroturfing to Generate True Affect Using Fakery (2026) · cites this paper as Keller et al., 2020
In the citing paper
“Traditional social bots often follow pre-defined instructions to perform simplistic tasks, such as spamming (Yang et al., 2019), following others, and amplifying certain narratives (Keller et al., 2020).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Keller et al., 2020
Citations
Cites
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
Social Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767 · cited 1×
“…level have become more influential, political actors such as governments (King, Pan, & Roberts, 2017; Lukito et al., 2018), hyperpartisan media (Allcott & Gentzkow, 2017; Faris et al., 2017), far-right groups (Marwick & Lewis, 2017) or actors with economic motives (Silverman & Alexander, 2016)…”
BackgroundTHE LOGIC OF CONNECTIVE ACTION: Digital media and the personalization of contentious politics
Bennett, W. L., & Segerberg, A. (2012) · Information, Communication & Society · doi:10.1080/1369118x.2012.670661 · Cited by 3,116 · cited 3×
“…a “social movement grows when people with grievances meet, agree on a common agenda, and organize for political action” (Howard, 2006, p. 86; see also Bennett & Segerberg, 2013), an astroturfing campaign tries to appear like such an organic expression of public opinion, but is actually centrally…”
BackgroundDirect Clustering of a Data Matrix
Hartigan, J. (1972) · Journal of the American Statistical Association · doi:10.1080/01621459.1972.10481214 · Cited by 1,255 · cited 2×
“, they tweet on the same days – and groups them together (Hartigan, 1972).”
BackgroundWeaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate
Broniatowski, D. A., Jamison, A. M., Qi, S., AlKulaib, L., Chen, T., Benton, A., Quinn, S. C., & Dredze, M. (2018) · American Journal of Public Health · doi:10.2105/ajph.2018.304567 · Cited by 1,149 · cited 1×
“Tweets posted by known and suspect NIS accounts should reflect these goals and strategies, and therefore differ significantly from those posted by regular users, political users, and opinion leaders (Broniatowski et al., 2018).”
BackgroundKING, G., PAN, J., & ROBERTS, M. E. (2017) · American Political Science Review · doi:10.1017/s0003055417000144 · Cited by 1,028 · cited 3×
“In order to prevent this type of shirking, the principal may opt for constant and direct supervision by physically locating all agents in the same office – as done by the “Russian troll factories”(3) – or at least hire them to work during office hours instead of paying by post, as the eponymous “50 cent party” alleged…”
BackgroundMethodInfluentialThe Hybrid Media System: Politics and Power
Chadwick, A. (2013) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780199759477.001.0001 · Cited by 1,000 · cited 2×
“Social media also have the potential to set the news agenda (Chadwick, 2013; Neuman, Guggenheim, Jang, & Bae, 2014) as journalists pay close attention to social media buzz (Lukito et al.”
BackgroundOnline Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 981
Messing, S., & Westwood, S. J. (2014) · Communication Research · doi:10.1177/0093650212466406 · Cited by 902
THE POLITICAL EVOLUTION OF PRINCIPAL-AGENT MODELS
Miller, G. J. (2005) · Annual Review of Political Science · doi:10.1146/annurev.polisci.8.082103.104840 · Cited by 895 · cited 1×
“To better understand how the patterns of the two types of campaigns should differ, we turn to principal-agent theory (Miller, 2005; Ross, 1973).”
BackgroundTwitter use in election campaigns: A systematic literature review
Jungherr, A. M. (2015) · Journal of Information Technology & Politics · doi:10.1080/19331681.2015.1132401 · Cited by 877
Is the Sample Good Enough? Comparing Data from Twitter's Streaming API with Twitter's Firehose
Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14401 · Cited by 863
The Dynamics of Public Attention: Agenda-Setting Theory Meets Big Data
Neuman, W. R., Guggenheim, L., Jones-Jang, S. M., & Bae, S. Y. (2014) · Journal of Communication · doi:10.1111/jcom.12088 · Cited by 705
Stier, S., Bleier, A., Lietz, H., & Strohmaier, M. (2018) · Political Communication · doi:10.1080/10584609.2017.1334728 · Cited by 675
Self-citationDetecting Automation of Twitter Accounts: Are You a Human, Bot, or Cyborg?
Chu, Z., Gianvecchio, S., Wang, H., & Jajodia, S. (2012) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2012.75 · Cited by 634
Political Polarization on Twitter
Conover, M., Ratkiewicz, J., Francisco, M., Goncalves, B., Menczer, F., & Flammini, A. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14126 · Cited by 626 · cited 1×
“Retweeting is often used to amplify the impact of a campaign (Yang & Kim, 2017), and retweet patterns usually reflect political divisions among Twitter users (Conover et al., 2011).”
BackgroundCurated Flows: A Framework for Mapping Media Exposure in the Digital Age
Thorson, K., & Wells, C. (2016) · Communication Theory · doi:10.1111/comt.12087 · Cited by 570 · cited 2×
“Social media platforms such as Twitter and Facebook have become major venues for ordinary citizens to discuss politics, disseminate news, and organize collective action (Bennett & Segerberg, 2013; Lilleker & Koc-Michalska, 2017; Stier et al., 2018; Thorson & Wells, 2016; Vaccari, 2017).”
BackgroundBots, #Strongerin, and #Brexit: Computational Propaganda During the UK-EU Referendum
Howard, P. N., & Kollanyi, B. (2016) · doi:10.2139/ssrn.2798311 · Cited by 392 · cited 4×
“We argue that past research’s predominant focus on automated accounts, famously known as “social bots”, (see Howard & Kollanyi, 2016; Varol, Ferrara, Davis, Menczer, & Flammini, 2017) misses its target since reports on recent astroturfing campaigns suggest that they are often at least partially run…”
BackgroundInfluentialBadawy, A., Ferrara, E., & Lerman, K. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508646 · Cited by 380
Uncovering Large Groups of Active Malicious Accounts in Online Social Networks
Cao, Q., Yang, X., Yu, J., & Palow, C. (2014) · Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security · doi:10.1145/2660267.2660269 · Cited by 274
Zannettou, S., Caulfield, T., Cristofaro, E. D., Sirivianos, M., Stringhini, G., & Blackburn, J. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3316495 · Cited by 221
What Drives Political Participation? Motivations and Mobilization in a Digital Age
Lilleker, D. G., & Koç-Michalska, K. (2016) · Political Communication · doi:10.1080/10584609.2016.1225235 · Cited by 181
Media Manipulation and Disinformation Online
Marwick, A. (2017) · doi:10.69985/qpaf6919 · Cited by 160 · cited 1×
“…governments (King, Pan, & Roberts, 2017; Lukito et al., 2018), hyperpartisan media (Allcott & Gentzkow, 2017; Faris et al., 2017), far-right groups (Marwick & Lewis, 2017) or actors with economic motives (Silverman & Alexander, 2016) try to influence the public with false, misleading or…”
BackgroundDetecting Bots on Russian Political Twitter
Stukal, D., Sanovich, S., Bonneau, R., & Tucker, J. A. (2017) · Big Data · doi:10.1089/big.2017.0038 · Cited by 140 · cited 2×
“Social media platforms such as Twitter and Facebook have become major venues for ordinary citizens to discuss politics, disseminate news, and organize collective action (Bennett & Segerberg, 2013; Lilleker & Koc-Michalska, 2017; Stier et al., 2018; Thorson & Wells, 2016; Vaccari, 2017).”
BackgroundCandidate Networks, Citizen Clusters, and Political Expression
Bode, L., Hanna, A., Yang, J., & Shah, D. V. (2015) · The Annals of the American Academy of Political and Social Science · doi:10.1177/0002716214563923 · Cited by 110 · cited 1×
“The increasing roles of opinion leaders and ordinary citizens in the public opinion formation process can enable new actors to frame public discourses in their favor (Bode et al., 2015).”
BackgroundSelf-citationChanging Perspectives: Is It Sufficient to Detect Social Bots?
Grimme, C., Assenmacher, D., & Adam, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-91521-0_32 · Cited by 93 · cited 1×
“Grimme et al. (2018), for instance, find that most of the accounts involved in a troll attack during a German elections TV debate were less than one month old.”
BackgroundVaccari, C. (2016) · Political Communication · doi:10.1080/10584609.2016.1201558 · Cited by 87 · cited 2×
“Social media platforms such as Twitter and Facebook have become major venues for ordinary citizens to discuss politics, disseminate news, and organize collective action (Bennett & Segerberg, 2013; Lilleker & Koc-Michalska, 2017; Stier et al., 2018; Thorson & Wells, 2016; Vaccari, 2017).”
BackgroundAnalyzing the Political Landscape of 2012 Korean Presidential Election in Twitter
Song, M., Kim, M., & Jeong, Y. (2014) · IEEE Intelligent Systems · doi:10.1109/mis.2014.20 · Cited by 71
Of activists and gatekeepers: Temporal and structural properties of policy networks on Twitter
Stier, S., Schünemann, W. J., & Steiger, S. (2017) · New Media & Society · doi:10.1177/1461444817709282 · Cited by 70
Self-citationKeller, F., Schoch, D., Stier, S., & Yang, J. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14941 · Cited by 58
Self-citationGrassroots for Hire: Public Affairs Consultants in American Democracy
Walker, E. (2014) · Cambridge University Press eBooks · doi:10.1017/cbo9781139108829 · Cited by 55 · cited 6×
“Its participants also send out similar messages, but they are intrinsically motivated, and usually organized in a more decentralized fashion than agents of an astroturfing campaign (Bennett & Segerberg, 2013; Walker, 2014).”
BackgroundInfluentialAnalysis of Twitter for 2012 South Korea Presidential Election by Text Mining Techniques
Bae, J.-H., Son, J. Y., & Song, M. (2013) · doi:10.13088/jiis.2013.19.3.141 · Cited by 50 · cited 1×
“The third baseline is created using all 27,606 tweets posted by 58 opinion leaders as identified by Bae, Son, and Song (2013).10 10The list of opinion leaders includes major politicians, Twitter celebrities, popular podcasters, journalists, We use these datasets for three types of content…”
MethodYang, J., & Kim, Y. M. (2017) · Journal of Information Technology & Politics · doi:10.1080/19331681.2017.1338174 · Cited by 31 · cited 2×
“Retweeting is often used to amplify the impact of a campaign (Yang & Kim, 2017), and retweet patterns usually reflect political divisions among Twitter users (Conover et al., 2011).”
BackgroundMethodSelf-citationDirector of National Intelligence
Elrod, S. (2011) · doi:10.1002/0471686786.ebd0155 · Cited by 31
Political Communication and Information Technology
Howard, P. N. (2005) · Cambridge University Press eBooks · doi:10.1017/cbo9780511615986.003 · cited 3×
“86; see also Bennett & Segerberg, 2013), an astroturfing campaign tries to appear like such an organic expression of public opinion, but is actually centrally coordinated and organized – hence the analogy to the eponymous carpet “AstroTurf” which tries to mimic real grass (Howard, 2006; Walker, 2014).”
BackgroundMethod
Research works printed but not linked: 11
- 10.1017/CBO9781139198752 · DOI
- Conover M., 2011, Proceedings of the Fifth International AAAI Conference on Web and Social Media, 89
- Faris, R., Roberts, H., Etling, B., Bourassa, N., Zuckerman, E. & Benkler, Y. (2017). Partisanship, propaganda, and disinformation: Online media and the 2016 U.S. presidential election (Berkman Klein Center Research Publication 2017-6). Retrieved from https://ssrn.com/abstract=3019414
- Howard P. N., 2006, New media campaigns and the managed citizen
- Keller F., 2017, Proceedings of the Eleventh International AAAI Conference on Web and Social Media, 564
- Linvill D. R., 2018, Troll factories: The internet research agency and state-sponsored agenda building
- Morstatter F., 2013, Proceedings of the Seventh International AAAI Conference on Weblogs and Social Media, 400
- Ross S. A., 1973, The American Economic Review, 63, 134
- SeoulDistrictCourt, 2014, Case ID: 2013GoHap577, 2013GoHap1060
- SeoulHigherCourt, 2015, Case ID: 2014No2820
- Varol O., 2017, Proceedings of the Eleventh International AAAI Conference on Web and Social Media, 280
Cited by
Sorted by citations
Social Media and Fake News in the 2016 Election
Allcott, H., & Gentzkow, M. (2017) · Journal of Economic Perspectives · doi:10.1257/jep.31.2.211 · Cited by 6,767
Susceptibility to misinformation about COVID-19 around the world
Roozenbeek, J., Schneider, C. R., Dryhurst, S., Kerr, J., Freeman, A. L. J., Recchia, G., van der Bles, A. M., & van der Linden, S. (2020) · Royal Society Open Science · doi:10.1098/rsos.201199 · Cited by 1,447
Fake news detection: A hybrid CNN-RNN based deep learning approach
Nasir, J. A., Khan, O. S., & Varlamis, I. (2021) · International Journal of Information Management Data Insights · doi:10.1016/j.jjimei.2020.100007 · Cited by 568
Earl, J. (2022) · Science Advances · doi:10.1126/sciadv.abl8198 · Cited by 250
“These messages drew attention away from negative news, denigrated the opposition, and raised divisive issues ( 164 ).”
BackgroundThe False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184
“Overall it is also important to broaden the perspective and focus more on political digital astroturfing [46] for which bots are just one potential dimension [47].”
Background
Metadata
- Authors
- Keller, F. B., Schoch, D., Stier, S., & Yang, J.
- Affiliations
- Hong Kong University of Science and Technology (HK)
- University of Manchester (GB)
- GESIS - Leibniz Institute for the Social Sciences (DE)
- University of Illinois Urbana-Champaign (US)
- Corresponding author
- Stier, Sebastian
- Venue
- Political Communication
- Publisher
- Informa UK Limited
- Published
- 2019-10-26
- Volume
- 37
- Issue
- 2
- Pages
- 256-280
- ISSN
- 1058-4609 (ISSN-L)
- 1091-7675
- OpenAlex
- W2971881481
- Cited by
- 378
- References
- 46
- Citations by year
- 2026: 53 · 2025: 56 · 2024: 71 · 2023: 53 · 2022: 59 · 2021: 41 · 2020: 45
- FWCI
- 73.01
- Type
- journal article
- Language
- en
- Open access
- green
- hdl.handle.net
- Locations
- Political Communication (publishedVersion)
- Illinois Digital Environment for Access to Learning and Scholarship (University of Illinois at Urbana-Champaign) (submittedVersion)
- BORIS (University Library Bern) (submittedVersion)
- Research Explorer (The University of Manchester) (submittedVersion, other-oa)
- Rare & Special e-Zone (The Hong Kong University of Science and Technology) (submittedVersion)
- Funders
- National Research Foundation of Korea (NRF-2016S1A3A2925033)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Social Sciences › Communication › Social Media and Politics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- disinformation campaigns, Twitter, 2012 South Korean presidential election, social bots, principal-agent model, election interference
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review