CoChat
Sign Up
Cited by 90Misinformation and Its ImpactsSpam and Phishing DetectionHate Speech and Cyberbullying Detection

Content-based features predict social media influence operations

Open access
Alizadeh, Meysam, Shapiro, Jacob N., Buntain, Cody, Tucker, Joshua A.·2020·Science Advances
DOI 10.1126/sciadv.abb5824PMC PMC7439640
Open this paper in CoChat — free account

Abstract

Coordinated political influence operations leave a distinct signature in content that machine learning can detect.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1126/sciadv.abb5824

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • National-level, covert information channeling is one of the few forms of repression where foreign state actors intervene to influence other countries' domestic politics.

    “National-level, covert information channeling (Table 2, cell 4) is one of the few forms of repression where foreign state actors intervene to influence other countries’ domestic politics (157).”

    From Physical and virtual repertoires of contention (2022) · cites this paper as [157]

    The Introduction (page unavailable) defines coordinated influence operations as campaigns that can target another state's politics through social-media content designed to appear indigenous; it also names foreign governments as possible actors. It does not characterize these operations as a form of repression or establish that they are one of the few forms involving foreign-state intervention. (LLM, conf=0.86)

    Experimental — accuracy not yet measured

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 21

  1. [2]Welles B. F., Jackson S. J., The battle for #Baltimore: Networked counterpublics and the contested framing of urban unrest. Int. J. Commun. 13, 21 (2019).
  2. [5]E. Walker, Grassroots for Hire: Public Affairs Consultants in American Democracy (Cambridge Univ. Press, 2014).
  3. [7]Broniatowski D. A., Jamison A. M., Qi S., AlKulaib L., Chen T., Benton A., Quinn S. C., Dredze M., Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate. Am. J. Public Health 108, 1378– 1384 (2018). 30138075 PMC6137759
  4. [9]Arif A., Stewart L. G., Starbird K., Acting the part: Examining information operations within #BlackLivesMatter discourse. Proc. ACM Hum. Comput. Interact. 2, 20 (2018).
  5. [10]Boyd R. L., Spangher A., Fourney A., Nushi B., Ranade G., Pennebaker J., Horvitz E., Characterizing the internet research agency’s social media operations during the 2016 U.S. presidential election using linguistic analyses. PsyArxiv 1 October 2018. 10.31234/osf.io/ajh2q. · DOI
  6. [11]Golovchenko Y., Buntain C., Eady G., Brown M. A., Tucker J. A., Cross-platform state propaganda: Russian trolls on twitter and youtube during the 2016 U.S. presidential election. Intl. J. Press Polit., (2020).
  7. [13]Bail C. A., Guay B., Malony E., Combs A., Hillygus D. S., Merhout F., Freelon D., Vlofovsky A., Assessing the Russian internet research agency’s impact on the political attitudes and behaviors of American twitter users in late 2017. Proc. Natl. Acad. Sci. U.S.A. 117, 243– 250 (2019). 31767743 PMC6955293
  8. [14]A. Badawy, E. Ferrara, K. Lerman, Analyzing the digital traces of political manipulation: The 2016 Russian interference Twitter campaign, in 2018 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) (IEEE, 2018), pp.58–265.
  9. [15]S. Zannettou, B. Bradlyn, E. Deristofaro, G. Stringhini, J. Blackburn, Characterizing the use of images by state-sponsored troll accounts on Twitter. arXiv: 1901.05997 (2019). · arXiv
  10. [17]Stukal D., Sanovich S., Bonneau R., Tucker J. A., Detecting bots on Russian political twitter. Big Data 5, 310– 324 (2017). 29235918
  11. [19]Varol O., Ferrara E., Menczer F., Flammini A., Early detection of promoted campaigns on social media. EPJ Data Sci. 6, 13 (2017).
  12. [20]D. Pacheco, P.-M. Hui, C. Torres-Lugo, B. T. Truong, A. Flammini, F. Menczer, Uncovering coordinated networks on social media. arXiv: 2001.05658 [cs.SI] (16 January 2020). · arXiv
  13. [21]B. Ghanem, D. Buscaldi, P. Rosso, TexTrolls: Identifying Russian trolls on Twitter from a textual perspective. arXiv: 1910.01340 [cs.CL] (3 October 2019). · arXiv
  14. [22]J. Im, E. Chandrasekharan, J. Sargent, P. Lighthammer, T. Denby, A. Bhargava, L. Hemphill, D. Jurgens, E. Gilbert, Still out there: Modeling and identifying Russian troll accounts on Twitter. arXiv: 1901.11162 [cs.SI] (31 January 2019). · arXiv
  15. [26]Aral S., Eckles D., Protecting elections from social media manipulation. Science 365, 858– 861 (2019). 31467206
  16. [28]Barberá P., Casas A., Nagler J., Egan P. J., Bonneau R., Jost J. T., Tucker J. A., Who leads? Who follows? Measuring issue attention and agenda setting by legislators and the mass public using social media data. Am. Pol. Sci. Rev. 113, 883– 901 (2019). PMC7672368 33303996
  17. [31]M. Cataldi, L. Di Caro, C. Schifanella, Emerging topic detection on twitter based on temporal and social terms evaluation, in Proceedings of the Tenth International Workshop on Multimedia Data Mining (ACM, 2010), p. 4.
  18. [32]J. Cheng, L. Adamic, P. A. Dow, J. M. Kleinberg, J. Leskovec, Can cascades be predicted?, in Proceedings of the 23rd International Conference on World Wide Web (ACM, 2014), pp. 925–936
  19. [34]H. Hosseinmardi, S. A. Mattson, R. I. Rafiq, R. Han, Q. Lv, S. Mishra, Analyzing labeled cyberbullying incidents on the instagram social network, in International Conference on Social Informatics (Springer, 2015), pp. 49–66.
  20. [38]O. Tsur, A. Rappoport, What’s in a hashtag?: Content based prediction of the spread of ideas in microblogging communities, in Proceedings of the Fifth ACM International Conference on Web Search and Data Mining (ACM, 2012), pp. 643–652.
  21. [40]Twitter, Retrospective Review Twitter, Inc. and the 2018 Midterm Elections in the United States (2019); https://cdn.cms-twdigitalassets.com/content/dam/blog-twitter/official/en_us/company/2019/2018-retrospective-review.pdf.

Cited by

Sorted by citations

Authors
Alizadeh, M., Shapiro, J. N., Buntain, C., & Tucker, J. A.
Affiliations
  • Princeton University (US)
  • New Jersey Institute of Technology (US)
  • New York University (US)
Corresponding author
Alizadeh, Meysam
ORCID
Venue
Science Advances
Publisher
American Association for the Advancement of Science (AAAS)
Published
2020-07-24
Volume
6
Issue
30
Pages
eabb5824-eabb5824
ISSN
2375-2548 (ISSN-L)
DOI
10.1126/sciadv.abb5824
PMC
PMC7439640
OpenAlex
W3043892524
Cited by
90
References
4
Citations by year
2026: 7 · 2025: 15 · 2024: 10 · 2023: 19 · 2022: 26 · 2021: 10 · 2020: 3
FWCI
19.40
Type
journal article
Language
en
Open access
Licence
Locations
  • Science Advances (publishedVersion, cc-by-nc)
  • PubMed Central (publishedVersion, cc-by-nc)
  • PubMed (publishedVersion)
Funders
  • Microsoft
  • Bertelsmann Foundation
Topics
  • Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
  • Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
  • Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
Keywords
influence operations, content-based features, machine learning classification, United States, Twitter, Reddit, user-generated content

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review