A decade of social bot detection

In the paper

“Contrarily, all techniques based on the analysis of the textual content of posted messages, such as those works that exclusively employ natural language processing techniques, are supervised detectors that analyze individual accounts.”

In the paper

“Instead, the number of new supervised detectors has been constantly increasing since 2012, despite their severe generalization issues [11].”

What the sources say

• Not checked yetcites [11] [11] Juan Echeverrìa, Emiliano De Cristofaro, Nicolas Kourtellis, Ilias Leontiadis, Gianluca Stringhini, and Shi Zhou. 2018. LOBO: Evaluation of generalization deficiencies in Twitter bot classifiers. In The 34th Annual Computer Security Applications Conference (ACSAC'18) . ACM, 137-146. -

In the paper

“In most cases, a real ground truth is lacking and the labels are simply given by human operators that manually analyze the data.”

In the paper

“The adoption of unsupervised machine learning started in 2013 with the rise of group approaches, and now appears to be stationary.”

In the paper

“Nowadays, new studies on the characterization, detection and impact estimation of bots are published at an impressive rate, as shown in Figure 2.”

In the paper

“In the early days, the vast majority of attempts at bot detection featured two distinctive characteristics: (i) they were based on supervised machine learning, and (ii) on the analysis of individual accounts.”

In the paper

“So far, we highlighted that a shift took place from individual to group detectors in an effort to contrast social bot evolution.”

In the paper

“Among the most pressing challenges along this line of research is the problem of scalability of group-based detectors and the intrinsic fuzziness of 'inauthentic coordinaton'.”

In the paper

“Similarly, computational means to discriminate between authentic and inauthentic coordination are yet to be proposed and evaluated.”

In the paper

“In fact, the scalable and generalizable detection of coordination is still a largely open challenge, with only few contributions proposed so far [12, 25].”

What the sources say

• Not checked yetcites [12] [12] Mohd Fazil and Muhammad Abulaish. 2020. A socialbots analysis-driven graph-based approach for identifying coordinated campaigns in Twitter. Journal of Intelligent & Fuzzy Systems 38 (2020), 2961-2977. Issue 3. - source
• Not checked yetcites [25] [25] Diogo Pacheco, Pik-Mai Hui, Christopher Torres-Lugo, Bao Tran Truong, Alessandro Flammini, and Filippo Menczer. 2020. Uncovering Coordinated Networks on Social Media. arXiv preprint arXiv:2001.05658 (2020). -

In the paper

“At the same time, some scientific communities tend to favor and stick to a specific approach.”

In the paper

“Unsupervised detectors emerged in 2012-13 and shifted the target to groups of misbehaving accounts.”

In the paper

“These challenges currently stand as largely unsolved, and call for the highest effort of our scientific community.”

In the paper

“Here, challenges include the limited availability of data itself, missing or ambiguous ground truth and the obsolescence of existing datasets that hardly cope with the rapid evolution of malicious accounts.”

In the paper

“A notable example of this kind is represented by the GAN trained in [33] for creating adversarial examples of social bots that improved the training of downstream detectors.”

What the sources say

• Not checked yetcites [33] [33] Bin Wu, Le Liu, Yanqing Yang, Kangfeng Zheng, and Xiujuan Wang. 2020. Using Improved Conditional Generative Adversarial Networks to Detect Social Bots on Twitter. IEEE Access 8 (2020), 36664-36680. - source

In the paper

“For instance, works published within the natural language processing community, quite naturally focus on textual content, thus resulting in a multitude of supervised classifiers that analyze accounts individually and that yield binary labels.”

In the paper

“The previous analysis highlights that initial efforts towards adversarial bot detection were driven by the creativity of some researchers and only covered few cases with limited applicability [9, 17].”

What the sources say

• Not checked yetcites [9] [9] Stefano Cresci, Marinella Petrocchi, Angelo Spognardi, and Stefano Tognazzi. 2019. Better Safe Than Sorry: An Adversarial Approach to Improve Social Bot Detection. In The 11th International Conference on Web Science (WebSci'19) . ACM. - source
• Not checked yetcites [17] [17] Christian Grimme, Dennis Assenmacher, and Lena Adam. 2018. Changing perspectives: Is it sufficient to detect social bots?. In The 10th International Conference on Social Computing and Social Media (SCSM'18) . - source

In the paper

“Despite the high hopes placed on adversarial approaches for detecting deception and automation, this research direction is still in its infancy and, probably due to its recency, is still lagging behind more traditional approaches.”

In the paper

“This paradigm has never been applied to the task of social bot detection, but it was tested with promising results for related tasks, such as that of fake news generation/detection [39].”

What the sources say

• Not checked yetcites [39] [39] Rowan Zellers, Ari Holtzman, Hannah Rashkin, Yonatan Bisk, Ali Farhadi, Franziska Roesner, and Yejin Choi. 2019. Defending against neural fake news. In The 33rd Conference on Neural Information Processing Systems (NeurIPS'19) . 9051-9062. -

In the paper

“One aspect that is often overlooked when evaluating bot detectors is their generalizability - that is, their capacity of achieving good detection results also for types of bots that have not been originally considered.”

In the paper

“In contrast, the complex networks community favors graph-based approaches.”

In the paper

“Today, we know that a minority of social bots - automated social media accounts mimicking humans - played a central role in spreading divisive messages and disinformation, possibly contributing to Trump's victory [16, 19].”

What the sources say

• Not checked yetcites [16] [16] Yuriy Gorodnichenko, Tho Pham, and Oleksandr Talavera. 2018. Social Media, Sentiment and Public Opinions: Evidence from #Brexit and #USElection . Working Paper 24631. National Bureau of Economic Research. -
• Not checked yetcites [19] [19] Nir Grinberg, Kenneth Joseph, Lisa Friedland, Briony Swire-Thompson, and David Lazer. 2019. Fake news on Twitter during the 2016 US presidential election. Science 363, 6425 (2019), 374-378. - source

In the paper

“and 11% of all Facebook accounts in 2019 [38]”

What the sources say

• Not checked yetcites [38] [38] Mattia Zago, Pantaleone Nespoli, Dimitrios Papamartzivanos, Manuel Gil Perez, Felix Gomez Marmol, Georgios Kambourakis, and Gregorio Martinez Perez. 2019. Screening Out Social Bots Interference: Are There Any Silver Bullets? IEEE Communications Magazine 57, 8 (2019), 98-104. - source

In the paper

“The average presence of bots was estimated to be in the region of 15% of all active Twitter accounts in 2017 [31]”

What the sources say

• Not checked yetcites [31] [31] Onur Varol, Emilio Ferrara, Clayton A Davis, Filippo Menczer, and Alessandro Flammini. 2017. Online human-bot interactions: Detection, estimation, and characterization. In The 11th International Conference on Web and Social Media (ICWSM'17) . AAAI. -

In the paper

“We forecast that from 2021 there will be more than 1 new paper published per day on social bots, which poses a heavy burden on those trying to keep pace with the evolution of this thriving field.”

In the paper

“Yet, we still lack a precise and well-agreed definition of what a social bot is.”

In the paper

“Social bots are actively used for both beneficial and nefarious purposes [13].”

What the sources say

• Not checked yetcites [13] [13] Emilio Ferrara, Onur Varol, Clayton Davis, Filippo Menczer, and Alessandro Flammini. 2016. The rise of social bots. Commun. ACM 59, 7 (2016), 96-104. - source

In the paper

“Other studies specifically focused on political activity, concluding that bots played a role in strategic information operations orchestrated ahead of numerous worldwide events, as shown in Figure 1.”

In the paper

“Similar results were obtained about the presence of bots in online cryptocurrency discussions [24] and as part of the 'infodemics' about the COVID-19 pandemic [14].”

What the sources say

• Not checked yetcites [24] [24] Leonardo Nizzoli, Serena Tardelli, Marco Avvenuti, Stefano Cresci, Maurizio Tesconi, and Emilio Ferrara. 2020. Charting the landscape of online cryptocurrency manipulation. IEEE Access (2020). https://doi.org/10.1109/ACCESS.2020.3003370 - source
• Not checked yetcites [14] [14] Riccardo Gallotti, Francesco Valle, Nicola Castaldo, Pierluigi Sacco, and Manlio De Domenico. 2020. Assessing the risks of 'infodemics' in response to COVID-19 epidemics. arXiv preprint arXiv:2004.03997 (2020). - source

In the paper

“A 2019 study reported that 71% of Twitter users mentioning trending US stocks, are likely to be bots [8].”

What the sources say

• Not checked yetcites [8] [8] Stefano Cresci, Fabrizio Lillo, Daniele Regoli, Serena Tardelli, and Maurizio Tesconi. 2019. Cashtag piggybacking: Uncovering spam and bot activity in stock microblogs on Twitter. ACM Transactions on the Web (TWEB) 13, 2 (2019), 11. - source

In the paper

“These cyborgs exist and operate halfway between the traditional concepts of bots and humans, resulting in weakened distinctions and overlapping behaviors between the two.”

In the paper

“In fact, by 2018, scholars found more than 40,000 public bot repositories [1].”

What the sources say

• Not checked yetcites [1] [1] Dennis Assenmacher, Lena Adam, Lena Frischlich, Heike Trautmann, and Christian Grimme. 2019. Openbots. arXiv preprint arXiv:1902.06691 (2019). -

In the paper

“Bence Kollanyi reported an exponential growth that led in 2016 to more than 4,000 GitHub repositories containing code for deploying Twitter bots [22].”

What the sources say

• Not checked yetcites [22] [22] Bence Kollanyi. 2016. Where Do Bots Come From? An Analysis of Bot Codes Shared on GitHub. International Journal of Communication 10 (2016), 20. -

In the paper

“The looming picture is one where social bots are among the weapons of choice for deceiving and manipulating crowds.”

In the paper

“Moreover, hundreds of tech-savvy social media users that participated in a crowdsourcing experiment were able to tell apart newer bots from legitimate users only 24% of the times. The same users were instead able of spotting older bots 91% of the times [5].”

What the sources say

• Not checked yetcites [5] [5] Stefano Cresci, Roberto Di Pietro, Marinella Petrocchi, Angelo Spognardi, and Maurizio Tesconi. 2017. The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race. In The 26th International World Wide Web Conference Companion (WWW'17 Companion) . IW3C2. - source

In the paper

“Interestingly, the plateau hit by unsupervised approaches co-occurred with the rise of adversarial ones, which might take their place in the coming years.”

In the paper

“For example, Cresci et al. developed a set of supervised machine learning classifiers for detecting so-called fake followers, a type of automated accounts commonly used to artificially boost the popularity of the public characters that buy them [4].”

What the sources say

• Not checked yetcites [4] [4] Stefano Cresci, Roberto Di Pietro, Marinella Petrocchi, Angelo Spognardi, and Maurizio Tesconi. 2015. Fame for sale: Efficient detection of fake Twitter followers. Decision Support Systems 80 (2015), 56-71. - source

In the paper

“Fake followers can be bought for as low as 12$ per 1,000 followers in the surface Web.”

What the sources say

• Not checked yetcites [5] [5] Stefano Cresci, Roberto Di Pietro, Marinella Petrocchi, Angelo Spognardi, and Maurizio Tesconi. 2017. The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race. In The 26th International World Wide Web Conference Companion (WWW'17 Companion) . IW3C2. - source

In the paper

“Indeed, the possibility for malicious accounts to leverage deepfake texts, profile pictures, and videos is worrying, and worthy of increased attention [10].”

What the sources say

• Not checked yetcites [10] [10] Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cedeño, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. 2020. A survey on computational propaganda detection. In The 29th International Joint Conference on Artificial Intelligence (IJCAI'20) . -

In the paper

“By leveraging more than 1,200 features of an account, Botometer evaluates possible bots based on their profile characteristics, social network structure, the content they produce, their sentiment expressions, and the timings of their actions [35].”

What the sources say

• Not checked yetcites [35] [35] Kai-Cheng Yang, Onur Varol, Clayton A Davis, Emilio Ferrara, Alessandro Flammini, and Filippo Menczer. 2019. Arming the public with artificial intelligence to counter social bots. Human Behavior and Emerging Technologies 1, 1 (2019), 48-61. - source

In the paper

“Conversely, the social bots studied by Chao Yang et al. appeared as more popular and credible, given the relatively large number of their social connections. In addition, they were no longer spamming the same messages over and over again.”

What the sources say

• Not checked yetcites [34] [34] Chao Yang, Robert Harkreader, and Guofei Gu. 2013. Empirical evaluation and new design for fighting evolving twitter spammers. IEEE Transactions on Information Forensics and Security 8, 8 (2013), 1280-1293. - source

In the paper

“Furthermore, these approaches typically output binary classifications. In many cases however, malicious accounts feature a mixture of automated and human-driven behaviors that cannot be accounted for with simple binary labels.”

In the paper

“In addition, by analyzing large groups of accounts, detectors also have access to more data for fueling powerful - yet data-hungry - AI algorithms.”

In the paper

“Initial success at social bot detection forced bot developers to put in place sophisticated countermeasures. Because of this, newer bots often feature advanced characteristics that make them way harder to detect with respect to older ones.”

In the paper

“Within this global picture, dichotomous classifications - such as human vs bot, fake vs real, coordinated vs not coordinated - might represent oversimplifications, unable to grasp the complexity of these phenomena and unlikely to yield accurate and actionable results.”

In the paper

“The first wave of social bots that populated OSNs until around 2011 was made of rather simplistic bots. Accounts with very low reputation due to few social connections and posted messages, and featuring clear signs of automation as shown in panel A of Figure 4.”

What the sources say

• Not checked yetcites [34] [34] Chao Yang, Robert Harkreader, and Guofei Gu. 2013. Empirical evaluation and new design for fighting evolving twitter spammers. IEEE Transactions on Information Forensics and Security 8, 8 (2013), 1280-1293. - source

In the paper

“We surveyed more than 230 papers that proposed a bot detection technique and we manually classified each detector along two orthogonal dimensions.”

In the paper

“The social bot pandemic gradually became much more difficult to stop.”

In the paper

“These results are backed by the same platforms where information operations took place - namely, Facebook 2 , Twitter 3 and Reddit 4 - that banned tens of thousands accounts involved in coordinated activities since 2016.”

What the sources say

• Not checked yetcites [2] [2] Nikan Chavoshi, Hossein Hamooni, and Abdullah Mueen. 2016. DeBot: Twitter bot detection via warped correlation. In The 16th International Conference on Data Mining (ICDM'16) . IEEE, 817-822. - source
• Not checked yetcites [3] [3] Zi Chu, Steven Gianvecchio, Haining Wang, and Sushil Jajodia. 2012. Detecting automation of Twitter accounts: Are you a human, bot, or cyborg? IEEE Transactions on Dependable and Secure Computing 9, 6 (2012), 811-824. - source
• Not checked yetcites [4] [4] Stefano Cresci, Roberto Di Pietro, Marinella Petrocchi, Angelo Spognardi, and Maurizio Tesconi. 2015. Fame for sale: Efficient detection of fake Twitter followers. Decision Support Systems 80 (2015), 56-71. - source

In the paper

“Results showed that only 5% of newer bots are removed from social platforms, whereas older ones are removed 60% of the times [5].”

What the sources say

• Not checked yetcites [5] [5] Stefano Cresci, Roberto Di Pietro, Marinella Petrocchi, Angelo Spognardi, and Maurizio Tesconi. 2017. The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race. In The 26th International World Wide Web Conference Companion (WWW'17 Companion) . IW3C2. - source

In the paper

“The existence of botnets does not necessarily imply that accounts are explicitly connected in the social network, but rather that they are maneuvered by a single entity and that time they share common goals.”

In the paper

“The adversarial framework sketched in Figure 6 is general enough to be applied to a wide set of deception detection tasks, comprising the detection of social bots, cyborgs, trolls and mis/disinformation.”

In the paper

“As such, efforts at adversarial detection can only be successful if the scientific community decides to rise to the many open challenges.”

In the paper

“Generative adversarial networks (GANs) are a powerful machine learning framework where two competing deep learning networks are jointly trained in a game-theoretic setting [15].”

What the sources say

• Not checked yetcites [15] [15] Ian Goodfellow, Patrick McDaniel, and Nicolas Papernot. 2018. Making machine learning robust against adversarial inputs. Commun. ACM 61, 7 (2018). - source

In the paper

“In addition, this paradigm accounts for adversaries by design , thus providing higher guarantees for deception detection, which violates the stationarity and neutrality assumptions.”

In the paper

“Inauthentic coordination is an important piece of the deception puzzle, since it is exploited by bad actors for obtaining visibility and impact.”

In the paper

“All tasks related to the detection of online deception, manipulation and automation are intrinsically adversarial .”

In the paper

“Finding good adversarial examples can help scholars understand the weaknesses of existing bot detection systems.”

In the paper

“Adversarial machine learning is a paradigm specifically devised for application in those scenarios presenting adversaries motivated in fooling learned models [15].”

What the sources say

• Not checked yetcites [15] [15] Ian Goodfellow, Patrick McDaniel, and Nicolas Papernot. 2018. Making machine learning robust against adversarial inputs. Commun. ACM 61, 7 (2018). - source

In the paper

“Furthermore, in contrast with existing adversarial approaches for bot detection, it is grounded on an established and successful machine learning framework, rather than on ad-hoc solutions lacking broad applicability.”

In the paper

“Combining these two evaluation dimensions, thus extensively exploring the generalizability space, would allow a much more reliable assessment of the detection capabilities of present and future techniques, thus avoiding overestimates of detection performance.”

In the paper

“As a result, bot hunters need not wait anymore for new bot mischiefs in order to adapt their techniques, but instead they can proactively (instead of reactively) test them, in an effort that could quickly make them more robust.”

In the paper

“Early detection of vulnerabilities can in turn contribute to the development of more robust detection systems.”

In the paper

“Now, thanks to the rise of AI-enabled deception techniques such as deepfakes , the most sophisticated of these malicious actors are bound to become indistinguishable from one another, and likely also from legitimate accounts.”

In the paper

“Interestingly, the same analysis that anticipated worldwide interest in inauthentic coordination, is now suggesting that adversarial approaches might give us an edge in the long-lasting fight against online deception.”

In the paper

“The exponentially growing body of work on social bot detection shown in Figure 2, somehow reassures us that much effort is bound to be devoted in the coming years to the fight of this crucial issue.”

In the paper

“Notably, the task of social bot detection is neither stationary nor neutral .”

In the paper

“Secondly, the foreseen increase in publications inevitably implies that more bot detectors will be proposed.”