Still out there: Modeling and Identifying Russian Troll Accounts on Twitter
Open accessThe DOI matches its Crossref record; 10 of the 10 references it cites were found in a registry or an index.
Abstract
There is evidence that Russia’s Internet Research Agency attempted to interfere with the 2016 U.S. election by running fake accounts on Twitter—often referred to as “Russian trolls”. In this work, we: 1) develop machine learning models that predict whether a Twitter account is a Russian troll within a set of 170K control accounts; and, 2) demonstrate that it is possible to use this model to find active accounts on Twitter still likely acting on behalf of the Russian state. Using both behavioral and linguistic features, we show that it is possible to distinguish between a troll and a non-troll with a precision of 78.5% and an AUC of 98.9%, under cross-validation. Applying the model to out-of-sample accounts still active today, we find that up to 2.6% of top journalists’ mentions are occupied by Russian trolls. These findings imply that the Russian trolls are very likely still active today. Additional analysis shows that they are not merely software-controlled bots, and manage their online identities in various complex ways. Finally, we argue that if it is possible to discover these accounts using externally-accessible data, then the platforms—with access to a variety of private internal signals—should succeed at similar or better rates.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
I tweet honestly, I tweet passionately: Twitter users, context collapse, and the imagined audience
Marwick, A. E., & boyd, D. (2011) · New Media & Society · doi:10.1177/1461444810365313 · Cited by 4,612 · cited 2×
“And on Twitter, users manage their self-presentation both via profiles and ongoing conversations [24].”
BackgroundManaging impressions online: Self-presentation processes in the online dating environment
Ellison, N., Heino, R., & Gibbs, J. (2006) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2006.00020.x · Cited by 1,616 · cited 2×
“Online dating site users attend to small details in others’ profiles and are careful when crafting their own profiles, since fewer cues meant the importance of the remaining ones were amplified [12].”
BackgroundSocial bots distort the 2016 U.S. Presidential election online discussion
Bessi, A., & Ferrara, E. (2016) · First Monday · doi:10.5210/fm.v21i11.7090 · Cited by 993 · cited 1×
“For instance, bots were responsible for millions of tweets right before the 2016 election (Bessi and Ferrara 2016).”
BotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909 · cited 4×
“Researchers at Indiana University provide a tool to the public called Botometer5 that will assess an account for bot characteristics (Davis et al. 2016).”
MethodResultInfluentialDisinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524 · cited 7×
“This was not the first time, as a disinformation cam-paign was coordinated through bots before the 2017 French presidential election (Ferrara 2017).”
BackgroundInfluentialSocial Network Profiles as Taste Performances
Liu, H. (2007) · Journal of Computer-Mediated Communication · doi:10.1111/j.1083-6101.2007.00395.x · Cited by 414
Acting the Part: Examining Information Operations Within #BlackLivesMatter Discourse
Arif, A., Stewart, L. G., & Starbird, K. (2018) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3274289 · Cited by 152
Russian Twitter accounts and the partisan polarization of vaccine discourse, 2015-2017
Walter, D., Ophir, Y., & Jamieson, K. H. (2020) · American Journal of Public Health · doi:10.2105/ajph.2019.305564 · Cited by 111 · cited 1×
“Additionally, Russian troll accounts used words such as maga (Make America Great Again), a term often used by conservatives, and keywords related to the Fukushima disaster, such as fukushima2015 [39].”
MethodWho Falls for Online Political Manipulation?
Badawy, A., Lerman, K., & Ferrara, E. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3316494 · Cited by 105
Confounds and consequences in geotagged twitter data
Pavalanathan, U., & Eisenstein, J. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1256 · Cited by 104 · cited 2×
“SAGE has been used successfully for the analysis of many types of language differences, including age and gender (Pavalanathan and Eisenstein 2015), and politics (Sim et al. 2013).”
Method
Cited by
Evaluating the Effectiveness of Deplatforming as a Moderation Strategy on Twitter
Jhaver, S., Boylston, C., Yang, D., & Bruckman, A. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3479525 · Cited by 169
Content-based features predict social media influence operations
Alizadeh, M., Shapiro, J. N., Buntain, C., & Tucker, J. A. (2020) · Science Advances · doi:10.1126/sciadv.abb5824 · Cited by 90
How to Find Orchestrated Trolls? A Case Study on Identifying Polarized Twitter Echo Chambers
Kratzke, N. (2023) · Computers · doi:10.3390/computers12030057 · Cited by 13
Metadata
- Authors
- Im, J., Chandrasekharan, E., Sargent, J., Lighthammer, P., Denby, T., Bhargava, A., Hemphill, L., Jurgens, D., & Gilbert, E.
- Venue
- 12th ACM Conference on Web Science
- Publisher
- ACM
- Published
- 2020-07-06
- Pages
- 1-10
- OpenAlex
- W3037924975
- Cited by
- 98
- References
- 10
- FWCI
- 40.04
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Still out there: Modeling and Identifying Russian Troll Accounts on Twitter. https://app.cochat.ai/papers/10.1145/3394231.3397889
- The paper
- Im, J., Chandrasekharan, E., Sargent, J., Lighthammer, P., Denby, T., Bhargava, A., Hemphill, L., Jurgens, D., & Gilbert, E. (2020). Still out there: Modeling and Identifying Russian Troll Accounts on Twitter. 12th ACM Conference on Web Science. https://doi.org/10.1145/3394231.3397889
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.