The DOI matches its Crossref record; 57 of the 71 references it cites were found in a registry or an index.
Online Human-Bot Interactions: Detection, Estimation, and Characterization
Open accessAbstract
Increasing evidence suggests that a growing amount of social media content is generated by autonomous entities known as social bots. In this work we present a framework to detect such entities on Twitter. We leverage more than a thousand features extracted from public data and meta-data about users: friends, tweet content and sentiment, network patterns, and activity time series. We benchmark the classification framework by using a publicly available dataset of Twitter bots. This training data is enriched by a manually annotated collection of active Twitter users that include both humans and bots of varying sophistication. Our models yield high accuracy and agreement with each other and can detect bots of different nature. Our estimates suggest that between 9% and 15% of active Twitter accounts are bots. Characterizing ties among accounts, we observe that simple bots tend to interact with bots that exhibit more human-like behaviors. Analysis of content flows reveals retweet and mention strategies adopted by bots to interact with different target groups. Using clustering analysis, we characterize several subclasses of accounts, including spammers, self promoters, and accounts that post content from connected applications.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“A decade of research on social bot detection 13-15 has established that automated accounts exhibit distinctive temporal, linguistic, and network signatures 14,31 .”
From The Moltbook Illusion: Separating Human Influence from Emergent Behavior in AI Agent Societies (2026) · cites this paper as [14]
In the citing paper
“Various characteristics, ranging from account metadata to social network structure, are then considered during the process (Ferrara et al., 2016;Varol et al., 2017).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Varol et al., 2017
In the citing paper
“These features are shown to be effective in identifying social bots and are described in detail in the literature [42, 43, 48].”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [43]
Citations
Cites
Scikit-learn: Machine Learning in Python
Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Müller, A., Nothman, J., Louppe, G., et al. (2012) · arXiv · doi:10.48550/arxiv.1201.0490 · Cited by 71,016 · cited 1×
“We benchmarked our system using several off-the-shelf algorithms provided in the scikit-learn library (Pedregosa et al. 2011).”
Method(2008) · Journal of Machine Learning Research · Cited by 33,051
Twitter mood predicts the stock market
Bollen, J., Mao, H., & Zeng, X. (2011) · Journal of Computational Science · doi:10.1016/j.jocs.2010.12.007 · Cited by 5,206 · cited 1×
“Sentiment extracted from social media conversations has been used to forecast offline events including financial market fluctuations (Bollen, Mao, and Zeng 2011), and is known to affect information spreading (Mitchell et al. 2013; Ferrara and Yang 2015).”
Recognizing contextual polarity in phrase-level sentiment analysis
Wilson, T., Wiebe, J., & Hoffmann, P. (2005) · doi:10.3115/1220575.1220619 · Cited by 3,388 · cited 1×
“…extraction techniques to generate various sentiment features, including arousal , valence and dominance scores (Warriner, Kuperman, and Brysbaert 2013), happiness score (Kloumann et al. 2012), polarization and strength (Wilson, Wiebe, and Hoffmann 2005), and emoti-con score (Agarwal et al. 2011).”
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973
Self-citationNorms of valence, arousal, and dominance for 13,915 English lemmas
Warriner, A. B., Kuperman, V., & Brysbaert, M. (2013) · Behavior Research Methods · doi:10.3758/s13428-012-0314-x · Cited by 1,959
Sentiment Analysis of Twitter Data
Agarwal, A., Xie, B., Vovsha, I., Rambow, O., & Passonneau, R. J. (2011) · Cited by 1,565
Social bots distort the 2016 U.S. Presidential election online discussion
Bessi, A., & Ferrara, E. (2016) · First Monday · doi:10.5210/fm.v21i11.7090 · Cited by 993 · cited 7×
“Features extracted from user meta-data have been used to classify users and patterns before (Mislove et al. 2011; Ferrara et al. 2016a).”
InfluentialSelf-citationBotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909 · cited 2×
“This system leverages more than one thousand features to evaluate the extent to which a Twitter account exhibits similarity to the known characteristics of social bots (Davis et al. 2016).”
BackgroundMethodSelf-citationIs the Sample Good Enough? Comparing Data from Twitter's Streaming API with Twitter's Firehose
Morstatter, F., Pfeffer, J., Liu, H., & Carley, K. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v7i1.14401 · Cited by 863 · cited 9×
“For each of these accounts, we collected their tweets through the Twitter Search API.”
MethodInfluential
Cited by
Sorted by citations
Fake news on Twitter during the 2016 U.S. presidential election
Grinberg, N., Joseph, K., Friedland, L., Swire-Thompson, B., & Lazer, D. (2019) · Science · doi:10.1126/science.aau2706 · Cited by 1,803
“In order to further guarantee that the sample did not include bots, we identified accounts judged by BorOrNot as bot-like (35).”
BackgroundMethodThe future of social media in marketing
Appel, G., Grewal, L., Hadi, R., & Stephen, A. T. (2020) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-019-00695-1 · Cited by 1,893
“For example, experts estimate that up to 15% of active Twitter accounts are bots (Varol et al.”
BackgroundTucker, J. A., Guess, A., Barbera, P., Vaccari, C., Siegel, A., Sanovich, S., Stukal, D., & Nyhan, B. (2018) · doi:10.2139/ssrn.3144139 · Cited by 1,222
The spread of low-credibility content by social bots
Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2018) · Nature Communications · doi:10.1038/s41467-018-06930-7 · Cited by 1,125
Self-citationInfluence of fake news in Twitter during the 2016 US presidential election
Bovet, A., & Makse, H. A. (2019) · CUNY Academic Works (City University of New York) · Cited by 882
Fake news game confers psychological resistance against online misinformation
Roozenbeek, J., & van der Linden, S. (2019) · Palgrave Communications · doi:10.1057/s41599-019-0279-9 · Cited by 708
“%) are bots (Varol et al., 2017).”
Influence of fake news in Twitter during the 2016 US presidential election
Bovet, A., & Makse, H. A. (2019) · Nature Communications · doi:10.1038/s41467-018-07761-2 · Cited by 701
Rapp, A., Curti, L., & Boldi, A. (2021) · International Journal of Human-Computer Studies · doi:10.1016/j.ijhcs.2021.102630 · Cited by 733
The spread of low-credibility content by social bots
Shao, C., Ciampaglia, G. L., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2018) · RePEc: Research Papers in Economics · Cited by 574
Self-citationDeep neural networks for bot detection
Kudugunta, S., & Ferrara, E. (2018) · Information Sciences · doi:10.1016/j.ins.2018.08.019 · Cited by 506
Self-citationBots increase exposure to negative and inflammatory content in online social systems
Stella, M., Ferrara, E., & De Domenico, M. (2018) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1803470115 · Cited by 492
“However, digital systems are not populated only by humans, but also by software-controlled agents, better known as bots, programmed to pursue specific tasks, from sending automated messages to assuming specific social or antisocial behaviors (19, 20).”
Self-citationDisinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524
“Second, to train our models we used a mixture of various types of bots provided by multiple studies (Varol, et al., 2017; Cresci, et al., 2017).”
BackgroundMethodInfluentialSelf-citationSocial media data for conservation science: A methodological overview
Toivonen, T., Heikinheimo, V., Fink, C., Hausmann, A., Hiippala, T., Järv, O., Tenkanen, H., & Minin, E. D. (2019) · Biological Conservation · doi:10.1016/j.biocon.2019.01.023 · Cited by 504
“Hence, cleaning and filtering the data are two of the most crucial steps in any social media data analysis workflow (Di Minin et al., 2018; Varol et al., 2017).”
MethodArming the public with artificial intelligence to counter social bots
Yang, K., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.115 · Cited by 419
“A recent report 2 revealed that reality television stars, professional athletes, comedians, TED speakers, pastors, and fashion models all purchased fake followers on Twitter at least once.”
BackgroundMethodInfluentialSelf-citationCanhoto, A. I., & Clear, F. (2019) · Business Horizons · doi:10.1016/j.bushor.2019.11.003 · Cited by 446
Badawy, A., Ferrara, E., & Lerman, K. (2018) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2018.8508646 · Cited by 380
“…at scale, by cloning or emulating the activity of human users while operating at much higher pace (e.g., automatically producing content following a scripted agenda) [26, 34, 38, 54] – however, it should be noted that bots have been also used, in some instances, for positive interventions [40, 47].”
BackgroundInfluentialSelf-citation(2020) · doi:10.1609/aaai.v34i01.5460 · Cited by 361
“To obtain the varol-icwsm dataset, the authors manually labeled accounts sampled from different Botometer score deciles (Varol et al. 2017).”
BackgroundMethodResultInfluentialSelf-citationPolitical Astroturfing on Twitter: How to Coordinate a Disinformation Campaign
Keller, F. B., Schoch, D., Stier, S., & Yang, J. (2020) · Political Communication · doi:10.1080/10584609.2019.1661888 · Cited by 378
Anatomy of an online misinformation network
Shao, C., Hui, P.-M., Wang, L., Jiang, X., Flammini, A., Menczer, F., & Ciampaglia, G. L. (2018) · PLoS ONE · doi:10.1371/journal.pone.0196087 · Cited by 282
“nalyze the accounts in the core of the network to identify those users who play an important role in the diusion of misinformation. The use of Botometer, a state-of-the-art social bot detection tool [12], reveals a higher presence of social bots in the main core. We also consider a host of centrality measures (in-stre…”
BackgroundSelf-citationChatbots, Humbots, and the Quest for Artificial General Intelligence
Grudin, J., & Jacques, R. (2019) · Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3290605.3300439 · Cited by 275
“[36] studied Twitterbots in 2016 and concluded that between 9% and 15% of Twitter posters were bots, which meant there were 3045 million of these simple bots.”
BackgroundXBlock-ETH: Extracting and Exploring Blockchain Data From Ethereum
Zheng, P., Zheng, Z., Wu, J., & Dai, H. (2020) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2020.2990458 · Cited by 252
The spread of fake news by social botsPreprint
Shao, C., Ciampaglia, G. L., Varol, O., Flammini, A., & Menczer, F. (2017) · arXiv (Cornell University) · Cited by 249
Self-citationDetection of Bots in Social Media: A Systematic Review
Orabi, M., Mouheb, D., Aghbari, Z. A., & Kamel, I. (2020) · Information Processing & Management · doi:10.1016/j.ipm.2020.102250 · Cited by 232
Social Bots in Election Campaigns: Theoretical, Empirical, and Methodological Implications
Keller, T. R., & Klinger, U. (2018) · Political Communication · doi:10.1080/10584609.2018.1526238 · Cited by 216
“Concluding, anomalous account usage patterns suggest the possible existence of a black-market for reusable political disinformation bots ” (Ferrara, 2017, p. 1).”
BackgroundMethodInfluentialA review on social spam detection: Challenges, open issues, and future directions
Rao, S., Verma, A. K., & Bhatia, T. (2021) · Expert Systems with Applications · doi:10.1016/j.eswa.2021.115742 · Cited by 237
Zannettou, S., Caulfield, T., Cristofaro, E. D., Sirivianos, M., Stringhini, G., & Blackburn, J. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3316495 · Cited by 221
“political agenda and that they exhibit various behaviors, e.g., trying to hide their identity and promoting topics through hashtags and retweets. Finally, a large body of work focuses on social bots [1,4,9,8,28] and their role in spreading disinformation, highlighting that they can manipulate the public’s opinion at l…”
BackgroundTwiBot-20: A Comprehensive Twitter Bot Detection Benchmark
Feng, S., Wan, H., Wang, N., Li, J., & Luo, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482019 · Cited by 199
“Property Semantic Neighbor varol-icwsm [27] pronbots [31] ✓ celebrity [31] ✓ gilani-17 [13] ✓ cresci-rtbust [19] ✓ cresci-stock [7] ✓ midterm-18 [32] ✓ botwiki [32] ✓ verified [32] ✓ PAN-19 3 ✓ caverlee [15] ✓ ✓ cresci-17 [6] ✓ ✓ TwiBot-20 ✓ ✓ ✓ are also encouraged to download and use the full…”
BackgroundRoß, B., Pilz, L., Cabrera, B., Brachten, F., Neubaum, G., & Stieglitz, S. (2019) · European Journal of Information Systems · doi:10.1080/0960085x.2018.1560920 · Cited by 197
“Studies found that between 9% and 15% of Twitter accounts are actually bots (Chu, Gianvecchio, Wang, & Jajodia, 2012; Varol, Ferrara, Davis, Menczer, & Flammini, 2017).”
BackgroundInfluential(2020) · Sabanci University · Cited by 197
Self-citationCombating disinformation in a social media age
Shu, K., Bhattacharjee, A., Alatawi, F., Nazer, T. H., Ding, K., Karami, M., & Liu, H. (2020) · WIREs Data Mining and Knowledge Discovery · doi:10.1002/widm.1385 · Cited by 198
“Words, phrases (Varol et al., 2017), and topics (Morstatter et al., 2016) of social media posts can be a strong indicator of bot activity.”
BackgroundThe False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184
“We thus assume for our simulation that the Twitter population has 15% bots [4] and 85% non-bots.”
BackgroundMethodInfluentialLimited individual attention and online virality of low-quality information
Qiu, X., Oliveira, D. F. M., Shirazi, A. S., Flammini, A., & Menczer, F. (2017) · Nature Human Behaviour · doi:10.1038/s41562-017-0132 · Cited by 207
“Currently, bot accounts controlled by software make up a significant portion of online profiles [60], and many of them flood social media with high volumes of low-quality information to manipulate public discourse [21, 5].”
BackgroundSelf-citationRetractedGolovchenko, Y., Buntain, C., Eady, G., Brown, M. A., & Tucker, J. A. (2020) · The International Journal of Press/Politics · doi:10.1177/1940161220912682 · Cited by 179
Canhoto, A. I. (2020) · Journal of Business Research · doi:10.1016/j.jbusres.2020.10.012 · Cited by 214
Zannettou, S., Caulfield, T., Cristofaro, E. D., Kourtelris, N., Leontiadis, I., Sirivianos, M., Stringhini, G., & Blackburn, J. (2017) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3131365.3131390 · Cited by 209
“We find that 80% of the users of both platforms share only URLs from mainstream news, while, 13% of Twitter users – which are likely bots [31] – exclusively post URLs to alternative news.”
BackgroundConversational Agents: Goals, Technologies, Vision and Challenges
Allouch, M., Azaria, A., & Azoulay, R. (2021) · Sensors · doi:10.3390/s21248448 · Cited by 168
“[199] used a publicly available dataset of Twitter accounts and manually labeled all users either as humans or influence chatbots.”
MethodGraph Convolutional Networks with Markov Random Field Reasoning for Social Spammer Detection
Wu, Y., Lian, D., Xu, Y., Wu, L., & Chen, E. (2020) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v34i01.5455 · Cited by 169
“A recent study (Varol et al. 2017) estimated that between 9% and 15% of active Twitter accounts are fake.”
BackgroundTwitter Bot Detection Using Bidirectional Long Short-Term Memory Neural Networks and Word Embeddings
Wei, F., & Nguyen, U. T. (2019) · 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa48467.2019.00021 · Cited by 168
“[10] proposed a supervised machine learning system that extracts more than a thousand features in six different classes: users, friends meta-data, tweet content, sentiment, network patterns and activity time series.”
BackgroundDetection of Novel Social Bots by Ensembles of Specialized Classifiers
Sayyadiharikandeh, M., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412698 · Cited by 159
“posed in this paper, extract various features from an account’s profile, social network, and content [16, 23, 29, 41, 43, 48, 49].”
BackgroundMethodSelf-citationBotometer 101: social bot practicum for computational social scientists
Yang, K.-C., Ferrara, E., & Menczer, F. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00177-5 · Cited by 159
“1 3 Botometer considers over 1000 features that can be categorized into six classes: user profile, friends, network, temporal, content and language, and sentiment [40].”
BackgroundMethodInfluentialSelf-citationDetection of spam-posting accounts on Twitter
Inuwa-Dutse, I., Liptrott, M., & Korkontzelos, I. (2018) · Neurocomputing · doi:10.1016/j.neucom.2018.07.044 · Cited by 159
“Specifically on Twitter, for every 21 tweets, one is spam and about 15% of active users are autonomous agents, i.e. social bots [8] .”
BackgroundMethodInfluentialThe Bias That Divides Us: The Science and Politics of Myside Thinking
Stanovich, K. E. (2021) · doi:10.7551/mitpress/13367.001.0001 · Cited by 152
A Neural Network-Based Ensemble Approach for Spam Detection in Twitter
Madisetty, S., & Desarkar, M. S. (2018) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2018.2878852 · Cited by 157
“A recent study shows that nearly 15% of the Twitter accounts are bots [3].”
BackgroundHajli, N., Saeed, U., Tajvidi, M., & Shirazi, F. (2021) · British Journal of Management · doi:10.1111/1467-8551.12554 · Cited by 145
“As mentioned by Varol et al. (2017) mentioned bots tend to retweet each other, meaning that simple bots frequently mention sophisticated bots and sophisticated bots retweet but do not mention humans due to their inabilities to involve in expressive discourse with individuals.”
BackgroundE-Cigarette Surveillance With Social Media Data: Social Bots, Emerging Topics, and Trends
Allem, J.-P., Ferrara, E., Uppu, S. P., Cruz, T. B., & Unger, J. B. (2017) · JMIR Public Health and Surveillance · doi:10.2196/publichealth.8641 · Cited by 140
“Their adoption has been documented in a variety of domains, including political astroturfing [26], stock market manipulation [27], spread of misinformation [28], promotional content [29], and in sentiment classification [24].”
BackgroundMethodSelf-citationUncovering Coordinated Networks on Social Media: Methods and Case Studies
Pacheco, D., Hui, P.-M., Torres-Lugo, C., Truong, B. T., Flammini, A., & Menczer, F. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18075 · Cited by 140
“Most machine learning tools to combat online abuse target the detection of social bots, and mainly use methods that focus on individual accounts (Davis et al. 2016; Varol et al. 2017; Yang et al. 2019; 2020; Sayyadiharikandeh et al. 2020).”
BackgroundMethodInfluentialSelf-citationZannettou, S., Caulfield, T., Setzer, W., Sirivianos, M., Stringhini, G., & Blackburn, J. (2019) · ACM Web Science Conference (WebSci) · doi:10.1145/3292522.3326016 · Cited by 135
“while Dutt et al. [13] focus on the Facebook platform and analyze ads shared by Russian trolls in order to find the cues that make them effective. Finally, a large body of work focuses on social bots [5,10,17,16,46] and their role in spreading political disinformation, highlighting that they can manipulate the public’s…”
BackgroundTampering with Twitter’s Sample API
Pfeffer, J., Mayer, K., & Morstatter, F. (2018) · EPJ Data Science · doi:10.1140/epjds/s13688-018-0178-0 · Cited by 135
Skjuve, M. B., Haugstveit, I. M., Følstad, A., & Brandtzæg, P. B. (2019) · Human Technology · doi:10.17011/ht/urn.201902201607 · Cited by 143
“In recent years, the distinction between human and automated conversational agents in social media services such as Twitter has become blurred, where a large proportion of automated accounts pose as human conversational agents (Varol, Ferrara, Davis, Menczer, & Flammini, 2017).”
BackgroundCharting the Landscape of Online Cryptocurrency Manipulation
Nizzoli, L., Tardelli, S., Avvenuti, M., Cresci, S., Tesconi, M., & Ferrara, E. (2020) · IEEE Access · doi:10.1109/access.2020.3003370 · Cited by 132
Self-citationEarly detection of promoted campaigns on social media
Varol, O., Ferrara, E., Menczer, F., & Flammini, A. (2017) · EPJ Data Science · doi:10.1140/epjds/s13688-017-0111-y · Cited by 133
Self-citationBovet, A., Morone, F., & Makse, H. A. (2018) · Scientific Reports · doi:10.1038/s41598-018-26951-y · Cited by 135
The responsibility of social media in times of societal and political manipulation
Reisach, U. (2020) · European Journal of Operational Research · doi:10.1016/j.ejor.2020.09.020 · Cited by 177
Vo, N., & Lee, K. (2018) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3209978.3210037 · Cited by 137
“Since fake news can be viewed as misinformation, work about detecting content polluters [26], social bots [47] and malicious campaigns [48] are also related to our work.”
BackgroundOf Bots and Humans (on Twitter)
Gilani, Z., Farahbakhsh, R., Gareth Tyson, A., et al. (2017) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3110025.3110090 · Cited by 131
“We initially experimented with BotOrNot [7], a state-of-the-art bot detection tool (to the best of our knowledge, this is the only available online tool).”
MethodInfluentialSocial media usage patterns during natural hazards
Niles, M. T., Emery, B. F., Reagan, A. J., Dodds, P. S., & Danforth, C. M. (2019) · PLoS ONE · doi:10.1371/journal.pone.0210484 · Cited by 132
“Indeed, human raters visiting the account page associated with a handle struggle to agree on whether it reflects robotic activity [32].”
BackgroundPolitical communication on social media: A tale of hyperactive users and bias in recommender systems
Papakyriakopoulos, O., Serrano, J. C. M., & Hegelich, S. (2019) · Online Social Networks and Media · doi:10.1016/j.osnem.2019.100058 · Cited by 144
Machine learning-based social media bot detection: a comprehensive literature review
Aljabri, M., Zagrouba, R., Shaahid, A., Alnasser, F., Saleh, A., & Alomari, D. M. (2023) · Social Network Analysis and Mining · doi:10.1007/s13278-022-01020-5 · Cited by 132
“Only 5 studies (Wu et al. 2020; Davis et al. 2016; Inuwa-Dutse et al. 2018; Sayyadiharikandeh et al. 2020; Varol et al. 2017) made use of the network-based features.”
BackgroundMethodInfluentialCharacterizing the 2016 Russian IRA influence campaign
Badawy, A., Addawood, A., Lerman, K., & Ferrara, E. (2019) · Social Network Analysis and Mining · doi:10.1007/s13278-019-0578-6 · Cited by 109
“The number of users that have bot scores above 0.5, and can therefore be safely considered bots according to prior work Varol et al. (2017b), stands at 8,424 accounts.”
BackgroundResultInfluentialSelf-citationContrastive Attributed Network Anomaly Detection with Data Augmentation
Xu, Z., Huang, X. X., Zhao, Y., Dong, Y., & Li, J. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-05936-0_35 · Cited by 125
“Another abnormal account type in social or e-commerce networks is created in large quantities to spam certain posts [38].”
BackgroundThe role of bot squads in the political propaganda on Twitter
Caldarelli, G., Nicola, R. D., Vigna, F. D., Petrocchi, M., & Saracco, F. (2020) · Communications Physics · doi:10.1038/s42005-020-0340-4 · Cited by 142
“Estimates conducted on Twitter report that, on average, social bots account for 9% to 15% of total active platform users [11].”
BackgroundDetecting Social Bots on Twitter: A Literature Review
Alothali, E., Zaki, N., Mohamed, E. A., & Alashwal, H. (2018) · 2018 International Conference on Innovations in Information Technology (IIT) · doi:10.1109/innovations.2018.8605995 · Cited by 102
“In 2017, they extended their work to improve the accuracy of the evaluation through the use of a new training data [4].”
BackgroundInfluentialEarly Warnings of Cyber Threats in Online Discussions
Sapienza, A., Bessi, A., Damodaran, S., Shakarian, P., Lerman, K., & Ferrara, E. (2017) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw.2017.94 · Cited by 104
Self-citationContagion dynamics of extremist propaganda in social networks
Ferrara, E. (2017) · Information Sciences · doi:10.1016/j.ins.2017.07.030 · Cited by 103
“.SI] 7 Jun 2017 May 2017 Social media bots (non-human automated accounts) have been used to coordinate attacks to successfully manipulate the stock market, causing losses in the billions of dollars [35,23,24,62]. Some governments and non-state actors have been active on social media to spread their propaganda. In some…”
BackgroundSelf-citationPredictive Analysis on Twitter: Techniques and Applications
Kursuncu, U., Gaur, M., Lokala, U., Thirunarayan, K., Sheth, A., & Arpinar, I. B. (2018) · Lecture notes in social networks · doi:10.1007/978-3-319-94105-9_4 · Cited by 99
“Sentiment has also been used in spam detection works [119, 120] as a feature to detect bots on Twitter.”
MethodUyheng, J., & Carley, K. M. (2020) · Journal of Computational Social Science · doi:10.1007/s42001-020-00087-4 · Cited by 105
“They may also aim to break up a community or inorganically link two communities together [4, 83].”
BackgroundResultA novel framework for detecting social bots with deep neural networks and active learning
Wu, Y., Fang, Y., Shang, S., Jin, J., Wei, L., & Wang, H. (2020) · Knowledge-Based Systems · doi:10.1016/j.knosys.2020.106525 · Cited by 95
Unreliable Users Detection in Social Media: Deep Learning Techniques for Automatic Detection
Sansonetti, G., Gasparetti, F., D’Aniello, G., & Micarelli, A. (2020) · IEEE Access · doi:10.1109/access.2020.3040604 · Cited by 105
“For that reason, we decided to select the features that are commonly considered in the literature (see, for instance, [52]) for representing tweets and related text content in social networks.”
BackgroundAuditing Elon Musk's Impact on Hate Speech and Bots
Hickey, D., Schmitz, M., Fessler, D., Smaldino, P. E., Muric, G., & Burghardt, K. (2023) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v17i1.22222 · Cited by 100
“Social bots have a significant presence on Twitter, estimated to be 9–15% of accounts (Varol et al. 2017).”
BackgroundCoordinated Behavior on Social Media in 2019 UK General Election
Nizzoli, L., Tardelli, S., Avvenuti, M., Cresci, S., & Tesconi, M. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v15i1.18074 · Cited by 95
“Many bot detection techniques have been proposed (Chavoshi, Hamooni, and Mueen 2016; Varol et al. 2017; Cresci et al. 2018; Mazza et al. 2019), but their effectiveness towards tracking IOs and CIB is still debated 10 .”
Detection of malicious social bots: A survey and a refined taxonomy
Latah, M. (2020) · Expert Systems with Applications · doi:10.1016/j.eswa.2020.113383 · Cited by 94
Why Botter: How Pro-Government Bots Fight Opposition in Russia
Стукал, Д., Sanovich, S., Bonneau, R., & Tucker, J. A. (2022) · American Political Science Review · doi:10.1017/s0003055421001507 · Cited by 99
“Disinformation campaigns involving coordinated human activity (King, Pan, and Roberts 2017; Sobolev 2018; Varol et al. 2017), denial-of-service attacks (Lutscher et al.”
BackgroundChanging Perspectives: Is It Sufficient to Detect Social Bots?
Grimme, C., Assenmacher, D., & Adam, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-91521-0_32 · Cited by 93
DeeProBot: a hybrid deep neural network model for social bot detection based on user profile data
Hayawi, K., Mathew, S. S., Venugopal, N., Masud, M. M., & Ho, P. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00869-w · Cited by 98
“As per Varol et al. (2017), up to 15% of Twitter users are social bots.”
BackgroundRuffo, G., Semeraro, A., Giachanou, A., & Rosso, P. (2022) · Computer Science Review · doi:10.1016/j.cosrev.2022.100531 · Cited by 130
“[266] was employed in several works, for instance by Shao et al.”
MethodInfluentialContent-based features predict social media influence operations
Alizadeh, M., Shapiro, J. N., Buntain, C., & Tucker, J. A. (2020) · Science Advances · doi:10.1126/sciadv.abb5824 · Cited by 90
Shi, W., Liu, D., Yang, J., Zhang, J., Wen, S., & Su, J. Y. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17228701 · Cited by 96
Bot, or not? Comparing three methods for detecting social bots in five political discourses
Martini, F., Samula, P., Keller, T. R., & Klinger, U. (2021) · Big Data & Society · doi:10.1177/20539517211033566 · Cited by 91
“Botometer has to date been used in various studies: Botometer ’ s developers estimated that the bot population on Twitter ranges between 9% and 15% (Varol et al., 2017).”
BackgroundMethodInfluentialTackling misinformation: What researchers could do with social media data
Pasquetto, I. V., Swire‐Thompson, B., & Amazeen, M. A. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-49 · Cited by 99
“Inauthentic, coordinated, malicious accounts, whether human-controlled or automated (Ferrara et al., 2016; Shao et al., 2018; Varol et al., 2017; Yang et al., 2019), domestic or state-sponsored (Zannettou et al., 2019), or anywhere in between, have been weaponized to influence public conversations…”
Identifying and Analyzing Cryptocurrency Manipulations in Social Media
Mirtaheri, M., Abu-El-Haija, S., Morstatter, F., Steeg, G. V., & Galstyan, A. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3059286 · Cited by 129
“2) Botometer Score [7].”
Ciampaglia, G. L. (2017) · Journal of Computational Social Science · doi:10.1007/s42001-017-0005-6 · Cited by 84
“Social bots, which so far have been responsible for spreading large amounts of misinformation [57–60], could be employed to this end, since initial evidence shows that they can be employed to spread positive messages [61].”
BackgroundComputational Social Science and the Study of Political Communication
Theocharis, Y., & Jungherr, A. M. (2020) · Political Communication · doi:10.1080/10584609.2020.1833121 · Cited by 106
“While methods of labeling social media accounts as bots abound (Varol et al., 2017), findings have been mixed (Keller et al., 2020).”
MethodLeveraging Analysis of User Behavior to Identify Malicious Activities in Large-Scale Social Networks
Al‐Qurishi, M., Hossain, M. S., Alrubaian, M., Rahman, S. M. M., & Alamri, A. M. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2753202 · Cited by 83
“This comes in the form of a CAPTCHA, where the user is asked to feed in a string of characters or resolve a given logical challenge [8], [15], [25], [38].”
BackgroundHow State and Protester Violence Affect Protest Dynamics
STEINERT-THRELKELD, Z., Chan, A., & Joo, J. (2021) · The Journal of Politics · doi:10.1086/715600 · Cited by 81
“We submit every user to the Botometer service and remove tweets with a complete automation probability ≥ .4, the threshold that produces the most accurate classification of bots (Varol et al. 2017).”
MethodExposure to social engagement metrics increases vulnerability to misinformation
Avram, M., Micallef, N., Patil, S., & Menczer, F. (2020) · Harvard Kennedy School Misinformation Review · doi:10.37016/mr-2020-033 · Cited by 83
“…& Flammini, 2017; Yang et al., 2019; Yang, Varol, Hui, & Menczer, 2020; Hui et al., 2019), time series analysis to detect promoted campaigns (Varol, Ferrara, Menczer, & Flammini, 2017), and natural language processing for flagging factually incorrect content (Prez-Rosas, Kleinberg, Lefevre,…”
BackgroundSelf-citationContrast Pattern-Based Classification for Bot Detection on Twitter
Loyola‐González, O., Monroy, R., Rodríguez‐Ruiz, J., López‐Cuevas, A., & Mata-Sánchez, J. I. (2019) · IEEE Access · doi:10.1109/access.2019.2904220 · Cited by 80
“There are currently about 48 million bot accounts in Twitter, amounting approximately to 15% of all active client accounts [8].”
BackgroundA new sociology of humans and machines
Tsvetkova, M., Yasseri, T., Pescetelli, N., & Werner, T. (2024) · Nature Human Behaviour · doi:10.1038/s41562-024-02001-8 · Cited by 122
“Bots and bot detection methods have co-evolved, resulting in increasingly more sophisticated imitation or detection strategies 25,193–198 , but detection is inherently limited due to the overlap between covert autonomous bots, managed user accounts, hacked accounts, cyborgs, sock puppets and…”
Understanding public perception of coronavirus disease 2019 (COVID-19) social distancing on Twitter
Saleh, S. N., Lehmann, C. U., McDonald, S., et al. (2020) · Infection Control and Hospital Epidemiology · doi:10.1017/ice.2020.406 · Cited by 80
Understanding the growth of the Fediverse through the lens of Mastodon
Cava, L. L., Greco, S., & Tagarelli, A. (2021) · Applied Network Science · doi:10.1007/s41109-021-00392-5 · Cited by 80
DeepSBD: A Deep Neural Network Model With Attention Mechanism for SocialBot Detection
Fazil, M., Sah, A. K., & Abulaish, M. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3102498 · Cited by 91
A Deep Learning Approach for Robust Detection of Bots in Twitter Using Transformers
Martin-Gutierrez, D., Hernández-Peñaloza, G., Belmonte-Hernández, A., Lozano-Díez, A., & Álvarez, F. (2021) · IEEE Access · doi:10.1109/access.2021.3068659 · Cited by 79
“Additionally, [38] presented a model for Twitter bot detection based on a large number of metadata from the account to perform the classification.”
MethodThe power of emotions: Leveraging user generated content for customer experience management
Sykora, M. D., Elayan, S., Hodgkinson, I. R., Jackson, T., & West, A. A. (2022) · Journal of Business Research · doi:10.1016/j.jbusres.2022.02.048 · Cited by 106
Lugosi, P., & Quinton, S. (2018) · Journal of Marketing Management · doi:10.1080/0267257x.2018.1431303 · Cited by 88
Lynn, T., Rosati, P., Santos, G. L., & Endo, P. T. (2020) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph17228557 · Cited by 86
“[57] estimated that between 9% and 15% of active Twitter accounts were bots.”
BackgroundResultInfluentialAre You a Cyborg, Bot or Human?—A Survey on Detecting Fake News Spreaders
Shahid, W., Li, Y., Staples, D., Gilkar, G. A., Hakak, S., & Ghorbani, A. A. (2022) · IEEE Access · doi:10.1109/access.2022.3157724 · Cited by 79
A big data approach to examining social bots on Twitter
Liu, X. (2019) · Journal of Services Marketing · doi:10.1108/jsm-02-2018-0049 · Cited by 73
Information flow reveals prediction limits in online social activity
Bagrow, J. P., Liu, X., & Mitchell, L. (2019) · RePEc: Research Papers in Economics · Cited by 72
Social Bots and Their Coordination During Online Campaigns: A Survey
Khaund, T., Kirdemir, B., Agarwal, N., Liu, H., & Morstatter, F. (2021) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2021.3103515 · Cited by 78
Ferrara, E. (2019) · Communications of the ACM · doi:10.1145/3299768 · Cited by 92
“Recent advancements in Artificial Intelligence (especially Artificial Neural Networks) fuel bots that can generate human-like natural language and interact with human users in near real time [16, 35].”
BackgroundMethodSelf-citationAnatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71
“Various characteristics, ranging from account metadata to social network structure, are then considered during the process (Ferrara et al., 2016; Varol et al., 2017).”
BackgroundMethodSelf-citation
Metadata
- Authors
- Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A.
- Affiliations
- Indiana University (US)
- University of Southern California (US)
- Venue
- Proceedings of the International AAAI Conference on Web and Social Media
- Published
- 2017-05-03
- Volume
- 11
- Issue
- 1
- Pages
- 280-289
- ISSN
- 2162-3449 (ISSN-L)
- 2334-0770
- OpenAlex
- W2595521492
- Semantic Scholar
- 6a025e1e4d7269b20dadc91e14f28a93124fc145
- Cited by
- 981
- References
- 71
- Citations by year
- 2026: 90 · 2025: 81 · 2024: 84 · 2023: 105 · 2022: 96 · 2021: 103 · 2020: 164 · 2019: 126 · 2018: 104 · 2017: 28
- FWCI
- 75.86
- Type
- conference paper
- Language
- en
- Open access
- diamond
- doi.org
- Locations
- Proceedings of the International AAAI Conference on Web and Social Media (publishedVersion)
- Funders
- National Science Foundation (1101743)
- James S. McDonnell Foundation
- Defense Advanced Research Projects Agency (W911NF-12-1-0037)
- Office of Naval Research (N15A-020-0053)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- social bots, bot detection, Twitter, human-bot interaction, botnets, spammer account, self-promotion, user activity patterns, social media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
