Better Safe Than Sorry: An Adversarial Approach to Improve Social Bot Detection
Abstract
The arm race between spambots and spambot-detectors is made of several cycles (or generations): a new wave of spambots is created (and new spam is spread), new spambot filters are derived and old spambots mutate (or evolve) to new species. Recently, with the diffusion of the adversarial learning approach, a new practice is emerging: to manipulate on purpose target samples in order to make stronger detection models. Here, we manipulate generations of Twitter social bots, to obtain - and study - their possible future evolutions, with the aim of eventually deriving more effective detection techniques. In detail, we propose and experiment with a novel genetic algorithm for the synthesis of online accounts. The algorithm allows to create synthetic evolved versions of current state-of-the-art social bots. Results demonstrate that synthetic bots really escape current detection techniques. However, they give all the needed elements to improve such techniques, making possible a proactive approach for the design of social bot detection systems.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In 2018, Facebook and Twitter acknowledged the importance of focusing on coordinated inauthentic behaviors.
“In 2018 - roughly 5 years after the emergence of the group approach to bot detection also Facebook 9 and Twitter 10 acknowledged the importance of focusing on 'coordinated inauthentic behaviors' .”
From A decade of social bot detection (2020) · cites this paper as [9]
In adversarial bot detection, scholars experiment with meaningful adversarial examples with which they extensively test the capabilities of current bot detectors.
“In the so-called adversarial bot detection , scholars experiment with meaningful adversarial examples with which they extensively test the capabilities of current bot detectors [9].”
From A decade of social bot detection (2020) · cites this paper as [9]
Initial efforts towards adversarial bot detection were driven by the creativity of some researchers and only covered few cases with limited applicability.
“The previous analysis highlights that initial efforts towards adversarial bot detection were driven by the creativity of some researchers and only covered few cases with limited applicability [9, 17].”
From A decade of social bot detection (2020) · cites this paper as [9]
Novel social bots emerge everyday and evade purge from platforms.
“In an arms race between abusive behaviors and countermeasures, novel social bots emerge everyday and evade purge from the platforms [15, 21].”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [15]
Cites
The Elements of Statistical Learning
Hastie, T., Friedman, J., & Tibshirani, R. (2001) · Springer Series in Statistics · doi:10.1007/978-0-387-21606-5 · Cited by 24,162 · see citing works
An introduction to ROC analysis
Fawcett, T. (2006) · Pattern Recognition Letters · doi:10.1016/j.patrec.2005.10.010 · Cited by 21,879 · see citing works
An Introduction to Genetic Algorithms
Mitchell, M. (1996) · doi:10.7551/mitpress/3927.001.0001 · Cited by 11,340 · see citing works
The Elements of Statistical Learning
Hastie, T. J., Tibshirani, R. J., & Friedman, J. H. (2009) · Springer series in statistics · doi:10.1007/b94608 · Cited by 4,111 · see citing works
Algorithms on strings, trees, and sequences computer science and computational biology
Gusfield, D. (1997) · Cited by 3,490 · see citing works
Algorithms on Strings, Trees and Sequences
Gusfield, D. (1997) · Cambridge University Press eBooks · doi:10.1017/cbo9780511574931 · Cited by 3,092 · see citing works
Adversarial machine learning at scale
Kurakin, A., Goodfellow, I., & Bengio, S. (2016) · arXiv · doi:10.48550/arxiv.1611.01236 · Cited by 2,325 · see citing works
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973 · see citing works
Online Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 928 · cited 1× · see citing works
“A stream of research focused on specific behavioral analytics tasks, such as detecting specific behavioral patterns [6, 27, 38], predicting future behaviors [30, 55], and detecting anomalous ones [7, 11, 13, 45, 53, 54].”
BackgroundFame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 461 · see citing works
Self-citationSeven Months with the Devils: A Long-Term Study of Content Polluters on Twitter
Lee, K., Eoff, B. D., & Caverlee, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14106 · Cited by 448 · see citing works
The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3055135 · Cited by 381 · cited 1× · see citing works
“g- [5, 8, 11, 13, 22, 48], and networkcharacteristics [5, 31, 47, 48, 51–53] of the accounts. However, until now, new detection systems have been developed only as a consequence of spambot evolutions [12]. In fact, no work has ever been done towards studying, and possibly anticipating, such evolutions. In other words,…”
BackgroundSelf-citationShekhar, S., Xiong, H. Y., & Zhou, X. S. (2017) · Encyclopedia of GIS · doi:10.1007/978-3-319-17885-1_100388 · Cited by 365 · see citing works
Empirical evaluation and new design for fighting evolving Twitter spammers
Yang, C., Harkreader, R., & Gu, G. (2013) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2013.2267732 · Cited by 346 · see citing works
Twitter spammer detection using data stream clustering
Miller, Z., Dickinson, B., Deitrick, W., Hu, W., & Wang, A. H. (2014) · Information Sciences · doi:10.1016/j.ins.2013.11.016 · Cited by 332 · see citing works
Evidence of complex contagion of information in social media: An experiment using Twitter bots
Mønsted, B., Sapiezynski, P., Ferrara, E., & Lehmann, S. (2017) · PLoS ONE · doi:10.1371/journal.pone.0184148 · Cited by 302 · see citing works
Uncovering social network Sybils in the wild
Yang, Z., Wilson, C., Wang, X., Gao, T., Zhao, B. Y., & Dai, Y. (2014) · ACM Transactions on Knowledge Discovery from Data · doi:10.1145/2556609 · Cited by 290 · see citing works
DeBot: Twitter Bot Detection via Warped Correlation
Chavoshi, N., Hamooni, H., & Mueen, A. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0096 · Cited by 260 · see citing works
The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race
Cresci, S., Pietro, R. D., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Technical University of Denmark, DTU Orbit (Technical University of Denmark, DTU) · doi:10.48550/arxiv.1701.03017 · Cited by 237 · see citing works
Self-citationHaustein, S., Bowman, T. D., Holmberg, K., Tsou, A., Sugimoto, C. R., & Larivière, V. (2016) · Journal of the Association for Information Science and Technology · doi:10.1002/asi.23456 · Cited by 231 · see citing works
Prediction of Human Activity by Discovering Temporal Sequence Patterns
Li, K., & Fu, Y. (2014) · IEEE Transactions on Pattern Analysis and Machine Intelligence · doi:10.1109/tpami.2013.2297321 · Cited by 212 · see citing works
Lokot, T. (2016) · doi:10.1080/21670811.2015.1081822 · Cited by 209 · see citing works
A generic statistical approach for spam detection in Online Social Networks
Ahmed, F., & Abulaish, M. (2013) · Computer Communications · doi:10.1016/j.comcom.2013.04.004 · Cited by 199 · see citing works
DNA-Inspired Online Behavioral Modeling and Its Application to Spambot Detection
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.29 · Cited by 194 · cited 7× · see citing works
“As such accounts put in place complex mechanisms to evade existing detection systems, scholars tried to keep pace by proposing powerful techniques based on profile- [5, 10, 54], posting- [5, 8, 11, 13, 22, 48], and networkcharacteristics [5, 31, 47, 48, 51–53] of the accounts.”
BackgroundMethodResultInfluentialSelf-citationColor Set Size problem with applications to string matching
Chi, L., & Hui, K. (1992) · Lecture notes in computer science · doi:10.1007/3-540-56024-6_19 · Cited by 168 · see citing works
Social fingerprinting: detection of spambot groups through DNA-inspired behavioral modeling
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2017.2681672 · Cited by 168 · see citing works
Self-citationSavage, S., Monroy‐Hernández, A., & Höllerer, T. (2016) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2818048.2819985 · Cited by 120 · see citing works
The dynamics of health behavior sentiments on a large online social network
Salathé, M., Vu, D. Q., Khandelwal, S., & Hunter, D. R. (2013) · EPJ Data Science · doi:10.1140/epjds16 · Cited by 116 · see citing works
Stochastic Hillclimbing as a Baseline Method for Evaluating Genetic Algorithms
Juels, A., & WATTENBERG, M. P. (1995) · Cited by 97 · see citing works
Restivo, A., Rocca, S. R. D., & Roversi, L. (2001) · Lecture notes in computer science · doi:10.1007/3-540-45446-2 · Cited by 96 · see citing works
Wang, B., Gong, N. Z., & Fu, H. (2017) · Proceedings · doi:10.1109/icdm.2017.56 · Cited by 92 · see citing works
Liu, S., Hooi, B., & Faloutsos, C. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133018 · Cited by 74 · see citing works
$FAKE: Evidence of Spam and Bot Activity in Stock Microblogs on Twitter
Cresci, S., Lillo, F., Regoli, D., Tardelli, S., & Tesconi, M. (2018) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v12i1.15073 · Cited by 74 · see citing works
Self-citationDetecting Social Bots by Jointly Modeling Deep Behavior and Content Information
Cai, C., Li, L., & Zeng, D. D. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133050 · Cited by 58 · see citing works
People are Strange when you're a Stranger: Impact and Influence of Bots on Social NetworksPreprint
Aiello, L. M., Deplano, M., Schifanella, R., et al. (2014) · arXiv (Cornell University) · doi:10.48550/arxiv.1407.8134 · Cited by 58 · see citing works
Is that social bot behaving unethically?
Salge, C. A. D. L., & Berente, N. (2017) · CACM · doi:10.1145/3126492 · Cited by 53 · see citing works
On the capability of evolved spambots to evade detection via genetic engineering
Cresci, S., Petrocchi, M., Spognardi, A., & Tognazzi, S. (2018) · Online Social Networks and Media · doi:10.1016/j.osnem.2018.10.005 · Cited by 53 · see citing works
Self-citationSen, I., Aggarwal, A., Mian, S., Singh, S., Kumaraguru, P., & Datta, A. (2018) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3201064.3201105 · Cited by 50 · cited 1× · see citing works
“The latest social bots are capable of sharing (credible) fake news, inflating the popularity of OSN users, and reshaping political debates [40, 41].”
BackgroundMitchell, J. (1998) · Routledge eBooks · doi:10.4324/9780203360538-1 · Cited by 47 · see citing works
Uncovering Fake Likers in Online Social Networks
Badri Satya, P. R., Lee, K., Lee, D., Tran, T., & Zhang, J. (. (2016) · Proceedings of the 25th ACM International on Conference on Information and Knowledge Management · doi:10.1145/2983323.2983695 · Cited by 46 · see citing works
Yu, R., He, X., & Liu, Y. (2014) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2623330.2623719 · Cited by 45 · see citing works
Strength in Numbers: Robust Tamper Detection in Crowd Computations
Viswanath, B., Bashir, M. A., Zafar, M. B., Bouget, S., Guha, S., Gummadi, K. P., Kate, A., & Mislove, A. (2015) · Proceedings of the 2015 ACM on Conference on Online Social Networks · doi:10.1145/2817946.2817964 · Cited by 43 · see citing works
Fu, R., Chang, B., & Gao, P. Q. (2005) · doi:10.1109/ivesc.2004.1414180 · Cited by 43 · see citing works
Avvenuti, M., Bellomo, S., Cresci, S., Polla, M. N. L., & Tesconi, M. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3051155 · Cited by 42 · see citing works
Self-citationND-Sync: Detecting Synchronized Fraud Activities
Γιάτσογλου, Μ., Chatzakou, D., Shah, N., Beutel, A., Faloutsos, C., & Vakali, A. I. (2015) · Lecture notes in computer science · doi:10.1007/978-3-319-18032-8_16 · Cited by 36 · see citing works
Spectrum-based Deep Neural Networks for Fraud Detection
Yuan, S., Wu, X., Li, J., & Lu, A. (2017) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3132847.3133139 · Cited by 34 · see citing works
Introduction to genetic algorithms
Goodman, E. D. (2010) · Genetic and Evolutionary Computation Conference Companion (GECCO Companion) · doi:10.1145/1830761.1830893 · Cited by 34 · see citing works
Social Spammer and Spam Message Co-Detection in Microblogging with Social Context Regularization
Wu, F., Shu, J., Huang, Y., & Yuan, Z. (2015) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2806416.2806560 · Cited by 33 · see citing works
Linear Time Algorithms for Generalizations of the Longest Common Substring Problem
Arnold, M., & Ohlebusch, E. (2009) · Algorithmica · doi:10.1007/s00453-009-9369-1 · Cited by 33 · see citing works
Antisocial Behavior on the Web
Kumar, S., Cheng, J., & Leskovec, J. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3051106 · Cited by 32 · see citing works
Zafarani, R., & Liu, H. (2015) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/2806416.2806535 · Cited by 29 · see citing works
Cresci, S., Petrocchi, M., Spognardi, A., & Tognazzi, S. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3184558.3191595 · Cited by 28 · see citing works
Self-citation(2017) · doi:10.1109/dsaa.2017.57 · Cited by 21 · see citing works
Random walk based context-aware activity recommendation for location based social networks
Bagci, H., & Karagöz, P. (2015) · IEEE International Conference on Data Science and Advanced Analytics (DSAA) · doi:10.1109/dsaa.2015.7344852 · Cited by 18 · see citing works
An Internet research project draws conservative ire
Mervis, J. D. (2014) · Science · doi:10.1126/science.346.6210.686 · Cited by 15 · see citing works
The Devil’s Triangle: Ethical Considerations on Developing Bot Detection Methods
Thieltges, A., Schmidt, F., & Hegelich, S. (2016) · National Conference on Artificial Intelligence · Cited by 15 · see citing works
Social media data analysis for revealing collective behaviors
Zhou, A., Qian, W., & Ma, H. (2012) · ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/2339530.2339746 · Cited by 14 · see citing works
Combinatorial Pattern Matching
Lewenstein, M., & Valiente, G. (2006) · Lecture notes in computer science · doi:10.1007/11780441 · Cited by 14 · see citing works
Arming the public with AI to counter social bots.
Yang, K.-C., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Cited by 12 · see citing works
A Typology of Socialbots (Abbrev.)
Maus, G. (2017) · ACM Web Science Conference (WebSci) · doi:10.1145/3091478.3098860 · Cited by 8 · see citing works
Jeong, J., & Moon, S. B. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3051115 · Cited by 6 · see citing works
Hardening Classifiers against Evasion: the Good, the Bad, and the UglyPreprint
Liang, T., Li, B., Hajaj, C., et al. (2017) · arXiv (Cornell University) · Cited by 6 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A decade of social bot detection
Cresci, S. (2020) · Communications of the ACM · doi:10.1145/3409116 · Cited by 323 · see citing works
“The previous analysis highlights that initial efforts towards adversarial bot detection were driven by the creativity of some researchers and only covered few cases with limited applicability [9, 17].”
BackgroundSelf-citationDetection of Novel Social Bots by Ensembles of Specialized Classifiers
Sayyadiharikandeh, M., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412698 · Cited by 159 · see citing works
Unsupervised detection of coordinated fake-follower campaigns on social media
Zouzou, Y., & Varol, O. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-024-00499-6 · Cited by 19 · see citing works
Investigating coordinated account creation using burst detection and network analysis
Bellutta, D., & Carley, K. M. (2023) · Journal of Big Data · doi:10.1186/s40537-023-00695-7 · Cited by 15 · see citing works
- Authors
- Cresci, S., Petrocchi, M., Spognardi, A., & Tognazzi, S.
- Affiliations
- Institute of Informatics and Telematics (IT)
- Sapienza University of Rome (IT)
- IMT School for Advanced Studies Lucca (IT)
- Venue
- Proceedings of the 10th ACM Conference on Web Science
- Publisher
- ACM
- Published
- 2019-06-26
- Pages
- 47-56
- OpenAlex
- W2962783931
- Semantic Scholar
- cec8f4d556c569b37090984a9dd25eae4dd4ce49
- Cited by
- 52
- References
- 62
- Citations by year
- 2026: 3 · 2025: 6 · 2024: 3 · 2023: 6 · 2022: 6 · 2021: 15 · 2020: 12 · 2019: 1
- FWCI
- 6.77
- Type
- conference paper
- Language
- en
- Open access
- closed
- Locations
- ACM Web Science Conference (WebSci) (publishedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
- Funders
- IMT School for Advanced Studies Lucca (TOFFEe Integrated Activities Projects)
- MIUR (Dipartimenti di Eccellenza 2018-2022 Sapienza Dept. Computer Science)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- social bot detection, adversarial learning, genetic algorithm, Twitter bots, proactive detection
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
