The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.
The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Open accessAbstract
Recent studies in social media spam and automation provide anecdotal argumentation of the rise of a new generation of spambots, so-called social spambots. Here, for the first time, we extensively study this novel phenomenon on Twitter and we provide quantitative evidence that a paradigm-shift exists in spambot design. First, we measure current Twitter's capabilities of detecting the new social spambots. Later, we assess the human performance in discriminating between genuine accounts, social spambots, and traditional spambots. Then, we benchmark several state-of-the-art techniques proposed by the academic literature. Results show that neither Twitter, nor humans, nor cutting-edge applications are currently capable of accurately detecting the new social spambots. Our results call for new approaches capable of turning the tide in the fight against this raising phenomenon. We conclude by reviewing the latest literature on spambots detection and we highlight an emerging common research trend based on the analysis of collective behaviors. Insights derived from both our extensive experimental campaign and survey shed light on the most promising directions of research and lay the foundations for the arms race against the novel social spambots. Finally, to foster research on this novel phenomenon, we make publicly available to the scientific community all the datasets used in this study.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper provides quantitative evidence that a new generation of Twitter “social spambots” mimics genuine users closely enough to evade Twitter, human annotators, and several state-of-the-art detection tools. It argues for a paradigm shift toward group-level/collective-behavior detection and releases annotated datasets.
Key findings
- Twitter’s own suspensions barely affect social spambots: survival rates were 95.2%, 96.1%, and 99.6% for social spambots #1–#3, close to genuine accounts at 96.5%; by contrast, traditional spambots #2 were 1.0% alive / 99.0% suspended, and fake followers were 25.4% alive / 73.5% suspended.
- Human annotators failed on social spambots: accuracy 0.2355 with 1,065 false negatives, versus 0.9136 on traditional spambots and 0.9201 on genuine accounts; Fleiss’ kappa was 0.186 for social spambots, 0.007 for traditional spambots, and 0.410 for genuine accounts.
- Established detection tools also failed: BotOrNot? achieved F-Measure 0.288 and MCC 0.174 on test set #1; Yang et al. achieved F-Measure 0.261 and MCC 0.043; Miller et al. performed worst.
- Group-level approaches performed much better: Ahmed et al. modified with fastgreedy community detection reached MCC 0.886 on test set #1 and 0.847 on test set #2; Cresci et al.’s digital DNA reached MCC 0.952 and 0.867, respectively.
Methods
- The study used Twitter datasets: 3,474 genuine accounts / 8,377,522 tweets; social spambots #1 (991 accounts / 1,610,176 tweets), #2 (3,457 / 428,542), #3 (464 / 1,418,626); traditional spambots #1–#4; fake followers (3,351 / 196,027); and two mixed test sets of 1,982 and 928 accounts.
- Twitter monitoring used API error codes 63 (suspended) and 50 (deleted) to measure account survival. Crowdsourcing used CrowdFlower with 4,428 accounts, at least 3 contributors per account, 247 trusted contributors from 42 countries, 13,284 answers, majority voting, and a 70% test-question accuracy threshold.
- The authors reproduced and benchmarked BotOrNot?, Yang et al.’s supervised classifier, Miller et al.’s stream clustering, Ahmed et al.’s graph clustering (modified with fastgreedy), and Cresci et al.’s digital DNA, evaluating precision, recall, specificity, accuracy, F-Measure, and MCC on the test sets.
Limitations
- Stated: BotOrNot?’s authors note its performance against evolved spambots may be worse than originally reported; Viswanath et al.’s technique cannot directly spot individual tampering accounts; and Table 7 is explicitly not complete but illustrative of an emerging trend.
- Stated/evident: 142 accounts (3.2%) were deleted, suspended, or protected during the crowdsourcing campaign, leaving 4,286 instead of 4,428; contributors were tech-savvy Twitter users and filtered by >70% test accuracy, so human-performance results may not generalize to all users.
- Evident: the datasets come from specific Twitter campaigns and years and rely on manual annotation; the social-spambot ground truth used group comparison, and the paper does not establish performance on other platforms or future spambot generations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
39 Own result · 21 Own interpretation · 16 Not checked yet · 10 No basis given · 5 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, we measured a slightly higher agreement for the classification of social spambots than for traditional ones, with κ = 0 . 186.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown, social spambots feature very small differences with respect to genuine accounts ( ∼ ± 3%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 2, genuine accounts feature a very high survival rate (96.5%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, we found such automated accounts to be similar to genuine ones in every way.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some of these differences are not even statistically significant, according to a chi-square test.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The deceitful activity was carried out by spamming URLs pointing to the advertised products.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instead, they showed very little agreement while classifying traditional spambots, as represented by κ = 0 . 007.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So far, traditional spambots #1 have largely managed to evade suspension, despite dating back to 2009.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, we uncovered a third group of social bots, social spambots #3 , which advertise products on sale on Amazon.com .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, among the no longer available accounts, the vast majority have been deleted by the original owner, rather than suspended by Twitter.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
- Probably not a substantial claim. 13
- It may state two separate claims as one. 21
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, we collected 13,284 answers given by 247 trusted contributors from 42 different countries.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of the 247 participating contributors, 60 of them ( ∼ 24%) evaluated our campaign, leading to a convincing aggregated score of 3.7/5, as shown in detail in Table 5.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Instead, traditional spambots #2 and fake followers show massive differences: ∼ ± 96% and ∼ ± 72%, respectively.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“So far, we have demonstrated that neither Twitter nor human operators are currently capable of identifying novel social spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Amazingly, we also found tens of human accounts who tried to engage in conversation with some of the spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Those accounts also represented credible sources of information since they all had thousands of followers and friends, the majority of which were genuine users 2 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All the 3 groups of social spambots feature very high survival rates, respectively 95.2%, 96.1%, and 99.6%.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in Table 6, overall, the human annotators obtained an accuracy of less than 0.24 on the social spambots, with more than 1,000 False Negatives (FN), meaning that contributors classified more than 1,000 accounts as genuine, when they actually belonged to the dataset of the last generation of spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The vast majority of tweets were harmless messages, occasionally interspersed by tweets mentioning a specific genuine (human) account and suggesting him to buy the VIP version of the app from a Web store.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, only 8.6% of the bots have been suspended, while 88.9% of them are still alive.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whereas the genuine accounts feature distributions that almost uniformly span across the possible range of values, social spambots have anomalous distributions.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As shown in Table 7, the established works benchmarked in Section 4 largely failed to detect the new wave of social spambots, one notable exception being the unsupervised approach proposed in [2].”
What the sources say
- Not checked yetcites [2] F. Ahmed and M. Abulaish. A generic statistical approach for spam detection in online social networks. Computer Communications, 36(10):1120--1129, 2013. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in Table 7, our modified implementation proved effective in detecting social spambots, with an MCC = 0.886 for test set #1 and MCC = 0.847 for test set #2 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This interesting result shows that, overall, the human contributors were able to correctly detect traditional spambots, as shown by the 0.91 accuracy, but also that contributors rarely agreed on the class.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We only recruited contributors who were tech-savvy and Twitter users themselves, in order to be reasonably sure about their knowledge of Twitter and its dynamics.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“This seems to suggest that Twitter's spambot detection mechanisms are still unable to accurately identify such accounts, while recent solutions proposed by Academia have succeeded in this task [44].”
What the sources say
- Not checked yetcites [44] C. Yang, R. Harkreader, and G. Gu. Empirical evaluation and new design for fighting evolving Twitter spammers. IEEE Trans. Inform. Forens. Sec., 8(8):1280--1293, 2013. source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results show that neither Twitter, nor humans, nor cutting-edge applications are currently capable of accurately detecting the new social spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results imply that humans generally failed in classifying social spambots (accuracy = 0 . 2355) and, furthermore, that they also were more in agreement on this mistake than they were when (correctly) classifying traditional spambots.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
In the paper
“BotOrNot? is a publicly-available service 7 to evaluate the similarity of a Twitter account with the known characteristics of social spambots [15].”
What the sources say
- Not checked yetcites [15] C. A. Davis, O. Varol, E. Ferrara, A. Flammini, and F. Menczer. BotOrNot: A system to evaluate social bots. In WWW companion. ACM, 2016. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our internal manual annotation has been carried out by comparing every account to all the others, in order to highlight possible similarities and common behaviors.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the political candidate was able to reach many more accounts in addition to his direct followers and managed to alter Twitter engagement metrics during the electoral campaign.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More worryingly, they also suggest that humans might not be able to detect social spambots in the wild, and to distinguish them from genuine accounts.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Recent studies in social media spam and automation provide anecdotal argumentation of the rise of a new generation of spambots, so-called social spambots .”
- Also stated in: § 1. INTRODUCTION · p. 1 · hedged
“Recent investigations anecdotally highlight how new waves of social spambots are rising [16, 48].”
What the sources say
- Not checked yetcites [16] E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. The rise of social bots. Commun. ACM, 59(7):96--104, 2016. source
- Not checked yetcites [48] J. Zhang, R. Zhang, Y. Zhang, and G. Yan. The rise of social botnets: Attacks and countermeasures. arXiv preprint arXiv:1603.02714, 2016.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § 1. INTRODUCTION · p. 1 · hedged
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, we required each account to be classified by at least 3 different contributors, with the final class decided by majority voting.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Low values of both Precision and Recall mean incomplete and unreliable spambot detection.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“However, the technique cannot directly spot the tampering accounts, and, thus, detection and removal of the single accounts must be performed using a separate methodology.”
What the sources say
- Not checked yetcites [40] B. Viswanath, M. A. Bashir, M. B. Zafar, S. Bouget, S. Guha, K. P. Gummadi, A. Kate, and A. Mislove. Strength in numbers: Robust tamper detection in crowd computations. In COSN. ACM, 2015. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, a further property - and advantage - of this recent research wave is that it proposes ad-hoc detection algorithms, rather than adopting generic and off-the-shelf machine learning algorithms.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can observe a paradigm-shift for research and development of spambot detection systems, which may exploit the new concepts to achieve better resilience and robustness and to withstand the next evolution of social media spambots.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The promising outcome of these novel approaches clearly indicates that this is a favorable research avenue.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite being based on different key concepts, all these systems propose novel algorithmic solutions and investigate groups of accounts as a whole, marking a significant difference with the previous literature.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrated the need for novel analytic tools capable of turning the tide in the arms race against such sophisticated spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
“API queries to a suspended account result in Twitter responding with the error code 63. API queries to accounts that have been deleted by their original owner result in Twitter responding with the error code 50.”
The paper relies on this without support.
The internal manual verification correctly certifies the automated nature of the accounts.
“All the accounts collected in this process have then undergone an internal manual verification phase to certify their automated nature.”
The paper relies on this without support.
The genuine accounts that answered the crowdsensing question are human-operated.
“All the replies to our questions were manually verified and all the 3,474 accounts that answered were certified as humans.”
The paper relies on this without support.
The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
“The level of agreement can be also interpreted as a proxy for the difficulty of a task.”
The paper relies on this without support.
Social spambots have a common goal, such as increasing someone's reputation score.
“since they do have a common goal (e.g., increasing someone's reputation score).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“these models can be evaded by malicious actors if they change their behaviors over time [40, 92]”
From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [40]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“we examined the behavioral similarities between accounts during annotation as described in the literature [12] and noticed significant similarities in the content-sharing behavior of anomalous accounts”
From Unsupervised detection of coordinated fake-follower campaigns on social media (2024) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“malicious actors continuously evolve their behaviors to evade platform detection [40, 58]”
From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [40]
In the citing paper
“Fake followers can be bought for as low as 12$ per 1,000 followers in the surface Web.”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Time went by and new studies acknowledged the rise of a third wave of bots that spread through online social networks from 2016 onwards [5, 13], as shown in panel C of Figure 4.”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“As such, botnets leave behind more traces of their automation and coordination than those left behind by sophisticated single bots [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Moreover, hundreds of tech-savvy social media users that participated in a crowdsourcing experiment were able to tell apart newer bots from legitimate users only 24% of the times. The same users were instead able of spotting older bots 91% of the times [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“The generality and ease of deployment of this detector are however counterbalanced by a reduced bot detection accuracy [5, 17].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Results showed that only 5% of newer bots are removed from social platforms, whereas older ones are removed 60% of the times [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“They also report that some bot classes, like social spambots, are hard to distinguish from human accounts.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“Cresci et al. [11] highlight that it is hard for human annotators to assign one label to one account to describe its bot class.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“The performance decay of current detection systems in the wild was reported by Cresci et al. [11], who showed that Twitter, human annotators, and state-of-the-art bot detection systems failed at discriminating between some new social spambots and genuine accounts.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [10, 7, 12, 13].”
From Influence of augmented humans in online interactions during voting events (2019) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| social spambots #2 | 3,457 | 3,322 (96.1%) | 1 (0.1%) | 134 (3.8%) |”
Reported 0.1 · recomputed 0.028927
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| social spambots #2 | 3,457 | 3,322 (96.1%) | 1 (0.1%) | 134 (3.8%) |”
Reported 3.8 · recomputed 3.876193
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Keywords · p. 2
“| social spambots #1 | retweeters of an Italian political candidate | 991 | 1,610,176 | 2012 | 3.1, 3.2 |”
§ 2. DATASETS · p. 3
“| social spambots #1 | 994 | 946 (95.2%) | 2 (0.2%) | 46 (4.6%) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| traditional spambots #1 | - 7.6% *** | - 0.8% | +8.7% *** |”
Reported 8.7 · recomputed 8.5
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| genuine accounts | 3,474 | 3,353 (96.5%) | 115 (3.3%) | 6 (0.1%) |”
Reported 0.1 · recomputed 0.172712
Citations
Cites
Finding community structure in very large networks
Clauset, A., Newman, M. E. J., & Moore, C. (2004) · Physical Review E · doi:10.1103/physreve.70.066111 · Cited by 7,390
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973
Gwet, K. L. (2014) · Medical Entomology and Zoology · Cited by 1,095
Density-Based Clustering over an Evolving Data Stream with Noise
Cao, F., Estert, M., Qian, W., & Zhou, A. (2006) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972764.29 · Cited by 1,033
BotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909
Detecting spammers on social networks
Stringhini, G., Kruegel, C., & Vigna, G. (2010) · Proceedings of the 26th Annual Computer Security Applications Conference · doi:10.1145/1920261.1920263 · Cited by 791
Graph Clustering Via a Discrete Uncoupling Process
van Dongen, S. (2008) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/040608635 · Cited by 745
Magno, G., Rodrigues, T., & Almeida, V. A. F. (2010) · Cited by 649
Uncovering social spammers: social honeypots + machine learning
Lee, K., Caverlee, J., & Webb, S. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835522 · Cited by 619
Fame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 475 · cited 1×
“Fake followers are another kind of malicious accounts that recently gained interest both from platform administrators and from the scientific world [13].”
BackgroundSelf-citation
Research works printed but not linked: 1
- M. Jiang, P. Cui, A. Beutel, C. Faloutsos, and S. Yang. Catching synchronized behaviors in large networks: A graph mining approach. ACM Trans. Knowl. Discov. Data, 10(4):35:1--35:27, 2016.
Cited by
Sorted by citations
Tucker, J. A., Guess, A., Barbera, P., Vaccari, C., Siegel, A., Sanovich, S., Stukal, D., & Nyhan, B. (2018) · doi:10.2139/ssrn.3144139 · Cited by 1,222
Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate
Broniatowski, D. A., Jamison, A. M., Qi, S., AlKulaib, L., Chen, T., Benton, A., Quinn, S. C., & Dredze, M. (2018) · American Journal of Public Health · doi:10.2105/ajph.2018.304567 · Cited by 1,149
Arming the public with artificial intelligence to counter social bots
Yang, K., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.115 · Cited by 419
“While supervised methods have proven to be effective in many cases, they do not perform well at detecting coordinated social bots that post human-generated content (Cresci et al., 2017; Grimme et al., 2018; Chen and Subramanian, 2018).”
BackgroundSocial Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373
Measuring objective and subjective well-being: dimensions and data sources
Voukelatou, V., Gabrielli, L., Miliou, I., Cresci, S., Sharma, R. R., Tesconi, M., & Pappalardo, L. (2020) · International Journal of Data Science and Analytics · doi:10.1007/s41060-020-00224-2 · Cited by 459
Self-citationHarnessing Artificial Intelligence Capabilities to Improve Cybersecurity
Zeadally, S., Adi, E., Baig, Z. A., & Khan, I. A. (2020) · IEEE Access · doi:10.1109/access.2020.2968045 · Cited by 359
“In contrast, malicious bots also emerged to cheat in online games [49], post spams [50], [51] and spread malware [52].”
Facts and Where to Find Them: Empirical Research on Internet Platforms and Content Moderation
Keller, D., & Leerssen, P. (2020) · Data Archiving and Networked Services (DANS) · doi:10.1017/9781108890960
A decade of social bot detection
Cresci, S. (2020) · Communications of the ACM · doi:10.1145/3409116 · Cited by 323
“th the recent rise of the adversarial ones. they share common goals. As such, botnets leave behind more traces of their automation and coordination than those left behind by sophisticated single bots [5]. Devising techniques for spotting suspiciously coordinated and synchronized behaviors is thus likely to yield bette…”
BackgroundInfluentialSelf-citationMisinformation, Disinformation, and Online Propaganda
Guess, A. M., & Lyons, B. A. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.003 · Cited by 284
Bots and Misinformation Spread on Social Media: Implications for COVID-19
Himelein-Wachowiak, M., Giorgi, S., Devoto, A., Rahman, M. M., Ungar, L., Schwartz, H. A., Epstein, D. H., Leggio, L., & Curtis, B. L. (2021) · Journal of Medical Internet Research · doi:10.2196/26933 · Cited by 238
“In light of findings that humans are largely unable to distinguish social bots from genuine (human) accounts [33], it is likely that humans unknowingly contribute to the spread of misinformation as well.”
InfluentialTwiBot-20: A Comprehensive Twitter Bot Detection Benchmark
Feng, S., Wan, H., Wang, N., Li, J., & Luo, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3482019 · Cited by 199
“Regarding dataset size, the number of users included in each dataset also range from 693 users in cresci-rtbust [19] to 50,538 in midterm-18 [32], and one of the most widely used dataset cresci-17 [6] contains 2,764 human users and 7,049 bots.”
BackgroundMethodInfluential(2020) · Sabanci University · Cited by 197
TweepFake: about detecting deepfake tweets
Fagni, T., Falchi, F., Gambini, M., Martella, A., & Tesconi, M. (2021) · PLOS ONE · doi:10.1371/journal.pone.0251415 · Cited by 185
“[19] proved that a human is not reliable on this labeling task.”
BackgroundSelf-citationThe State of the Art in Enhancing Trust in Machine Learning Models with the Use of Visualizations
Chatzimparmpas, A., Martins, R. M., Jusufi, I., Kucher, K., Fabrice Rossi, A., & Kerren, A. (2020) · Computer Graphics Forum · doi:10.1111/cgf.14034 · Cited by 203
“Squares [RAL ∗ 17] is a performance visualization for multi-class classification problems.”
BackgroundMethodInfluentialThe False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184
“At least researchers should be aware of the data used as training data for the classifier that might only cover certain types of bots [15, 16, 18].”
BackgroundMethodResultInfluentialA survey on deep learning for cybersecurity: Progress, challenges, and opportunities
Macas, M., Wu, C., & Fuertes, W. (2022) · Computer Networks · doi:10.1016/j.comnet.2022.109032 · Cited by 202
RTbust: Exploiting Temporal Patterns for Botnet Detection on Twitter
Mazza, M., Cresci, S., Avvenuti, M., Quattrociocchi, W., & Tesconi, M. (2019) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3292522.3326015 · Cited by 163
Self-citationTwitter Bot Detection Using Bidirectional Long Short-Term Memory Neural Networks and Word Embeddings
Wei, F., & Nguyen, U. T. (2019) · 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa48467.2019.00021 · Cited by 168
“We prepared two test sets following [56].”
Detection of Novel Social Bots by Ensembles of Specialized Classifiers
Sayyadiharikandeh, M., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412698 · Cited by 159
Botometer 101: social bot practicum for computational social scientists
Yang, K.-C., Ferrara, E., & Menczer, F. (2022) · Journal of Computational Social Science · doi:10.1007/s42001-022-00177-5 · Cited by 159
“These accounts might come from a dif‐ ferent context, use different languages other than English [52, 53], or show novel behavioral patterns [34, 45, 54].”
BackgroundDetecting Bots on Russian Political Twitter
Stukal, D., Sanovich, S., Bonneau, R., & Tucker, J. A. (2017) · Big Data · doi:10.1089/big.2017.0038 · Cited by 140
“26 However, it was only in recent years that both the scope 27 and sophistication 28 of bot activity on Twitter increased to such an extent that it began to draw serious attention from political actors, including state actors.”
A Hybrid Approach for Detecting Automated Spammers in Twitter
Fazil, M., & Abulaish, M. (2018) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2018.2825958 · Cited by 137
“The experimental proof of the existence of social spambots and the challenges arising due to their presence are discussed in [24].”
BackgroundCharting the Landscape of Online Cryptocurrency Manipulation
Nizzoli, L., Tardelli, S., Avvenuti, M., Cresci, S., Tesconi, M., & Ferrara, E. (2020) · IEEE Access · doi:10.1109/access.2020.3003370 · Cited by 132
Self-citationYuan, X., Schuchard, R., & Crooks, A. T. (2019) · Social Media + Society · doi:10.1177/2056305119865465 · Cited by 154
“This is not surprising as the emerging field of social bot research is still working toward establishing norms (Cresci, Di Pietro, Petrocchi, Spognardi, & Tesconi, 2017; Ferrara et al., 2016) and a multitude of different methodologies must be employed to account for the vast array of different…”
BackgroundDetect Me If You Can: Spam Bot Detection Using Inductive Representation Learning
Alhosseini, S. A., Tareaf, R. B., Najafi, P., & Meinel, C. (2019) · International World Wide Web Conference (WWW) · doi:10.1145/3308560.3316504 · Cited by 133
Of Bots and Humans (on Twitter)
Gilani, Z., Farahbakhsh, R., Gareth Tyson, A., et al. (2017) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1145/3110025.3110090 · Cited by 131
The Future of False Information Detection on Social Media: New Perspectives and Trends
Guo, B., Ding, Y., Yao, L., Liang, Y., & Yu, Z. (2021) · ACM Computing Surveys · doi:10.1145/3393880 · Cited by 122
“Cresci et al. [26] also leverage characteristics of group accounts to detect malicious bots.”
BackgroundMachine learning-based social media bot detection: a comprehensive literature review
Aljabri, M., Zagrouba, R., Shaahid, A., Alnasser, F., Saleh, A., & Alomari, D. M. (2023) · Social Network Analysis and Mining · doi:10.1007/s13278-022-01020-5 · Cited by 132
“The authors recommend considering the three types of bot class as in (Cresci et al. 2017) dataset.”
InfluentialCashtag Piggybacking: Uncovering Spam and Bot Activity in Stock Microblogs on Twitter
Cresci, S., Lillo, F., Regoli, D., Tardelli, S., & Tesconi, M. (2019) · ACM Transactions on the Web · doi:10.1145/3313184 · Cited by 105
Self-citationWho Falls for Online Political Manipulation?
Badawy, A., Lerman, K., & Ferrara, E. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3316494 · Cited by 105
“…Kollanyi et al. 2016; Marwick and Lewis 2017; Shorey and Howard 2016; Wang et al. 2016; Woolley 2016; Woolley and Howard 2016] and all over the world, including in South America [Forelle et al. 2015; Suárez-Serrato et al. 2016], the U.K. [Howard and Kollanyi 2016], and Italy [Cresci et al. 2017].”
BackgroundThe role of bot squads in the political propaganda on Twitter
Caldarelli, G., Nicola, R. D., Vigna, F. D., Petrocchi, M., & Saracco, F. (2020) · Communications Physics · doi:10.1038/s42005-020-0340-4 · Cited by 142
“, [12]: exactly what is revealed by the entropy-based filtering.”
BackgroundSelf-citationSiegel, A. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.005 · Cited by 123
Demystifying Social Bots: On the Intelligence of Automated Social Media Actors
Assenmacher, D., Clever, L., Frischlich, L., Quandt, T., Trautmann, H., & Grimme, C. (2020) · Social Media + Society · doi:10.1177/2056305120939264 · Cited by 100
Heidari, M., Jones, J. H., & Uzuner, Ö. (2020) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw51313.2020.00071 · Cited by 99
“The data set for this research is the Cresci 2017 data set [1], which is a labeled data set of different types of bots and genuine (human) users.”
BackgroundMethodInfluentialDisinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524
Twitter spam account detection based on clustering and classification methods
Adewole, K. S., Han, T., Wu, W., Song, H. H., & Sangaiah, A. K. (2018) · The Journal of Supercomputing · doi:10.1007/s11227-018-2641-x · Cited by 95
DeeProBot: a hybrid deep neural network model for social bot detection based on user profile data
Hayawi, K., Mathew, S. S., Venugopal, N., Masud, M. M., & Ho, P. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00869-w · Cited by 98
Ruffo, G., Semeraro, A., Giachanou, A., & Rosso, P. (2022) · Computer Science Review · doi:10.1016/j.cosrev.2022.100531 · Cited by 129
Bot, or not? Comparing three methods for detecting social bots in five political discourses
Martini, F., Samula, P., Keller, T. R., & Klinger, U. (2021) · Big Data & Society · doi:10.1177/20539517211033566 · Cited by 91
“Studies have already shown that human coders are by no means better able to identify automated accounts (Cresci et al., 2017a, 2017b), at least in manual content analysis, while sophisticated digital forensics is a different story.”
BackgroundFeng, S., Wan, H., Wang, N., Li, J., & Luo, M. (2021) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/3459637.3481949 · Cited by 102
“• cresci-17 [5] is a public dataset with 4 components: genuine accounts, social spambots, traditional spambots and fake followers.”
Leveraging Analysis of User Behavior to Identify Malicious Activities in Large-Scale Social Networks
Al‐Qurishi, M., Hossain, M. S., Alrubaian, M., Rahman, S. M. M., & Alamri, A. M. (2017) · IEEE Transactions on Industrial Informatics · doi:10.1109/tii.2017.2753202 · Cited by 83
Contrast Pattern-Based Classification for Bot Detection on Twitter
Loyola‐González, O., Monroy, R., Rodríguez‐Ruiz, J., López‐Cuevas, A., & Mata-Sánchez, J. I. (2019) · IEEE Access · doi:10.1109/access.2019.2904220 · Cited by 80
“Account Information: an account information feature is extracted from the Twitter account that has been allegedly used to post a message [14].”
BackgroundMethodInfluentialDetecting malicious activity in Twitter using deep learning techniques
Ilias, L., & Roussaki, I. G. (2021) · Applied Soft Computing · doi:10.1016/j.asoc.2021.107360 · Cited by 87
DeepSBD: A Deep Neural Network Model With Attention Mechanism for SocialBot Detection
Fazil, M., Sah, A. K., & Abulaish, M. (2021) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2021.3102498 · Cited by 91
A Deep Learning Approach for Robust Detection of Bots in Twitter Using Transformers
Martin-Gutierrez, D., Hernández-Peñaloza, G., Belmonte-Hernández, A., Lozano-Díez, A., & Álvarez, F. (2021) · IEEE Access · doi:10.1109/access.2021.3068659 · Cited by 79
“In [11], authors annotated more than 8000 accounts and proposed a classifier which achieved a considerable level of accuracy for such set of samples.”
BackgroundDetecting computer-generated disinformation
Stiff, H., & Johansson, F. (2022) · International Journal of Data Science and Analytics · doi:10.1007/s41060-021-00299-5 · Cited by 79
“Early machine learning-based approaches to detecting automation of social media accounts were often based on relatively simple measurements related to posting behavior, posted content, and account properties of individual accounts [6], but such approaches are not working as well today, due to newer generations of bots…”
BackgroundLingam, G., Rout, R. R., & Somayajulu, D. V. L. N. (2019) · Applied Intelligence · doi:10.1007/s10489-019-01488-3 · Cited by 73
Unsupervised Social Bot Detection via Structural Information Theory
Peng, H. M., Zhang, J., Huang, X., Hao, Z., Li, A., Yu, Z., & Yu, P. S. (2024) · ACM Transactions on Information Systems · doi:10.1145/3660522 · Cited by 82
An Empirical Study of Machine learning Algorithms for Social Media Bot Detection
Heidari, M., Jones, J. H., & Uzuner, Ö. (2021) · IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS) · doi:10.1109/iemtronics52119.2021.9422605 · Cited by 76
“In the study conducted by Cresci et al. [13], the authors identified multiple types of spam bots, including promoter bots, URL spam bots, and fake followers.”
BackgroundMethodInfluentialAre You a Cyborg, Bot or Human?—A Survey on Detecting Fake News Spreaders
Shahid, W., Li, Y., Staples, D., Gilkar, G. A., Hakak, S., & Ghorbani, A. A. (2022) · IEEE Access · doi:10.1109/access.2022.3157724 · Cited by 79
“The automated nature of bots makes it easy to achieve a large scale impact when spreading misinformation [15].”
BackgroundAmplifying influence through coordinated behaviour in social networks
Weber, D., & Neumann, F. (2021) · Social Network Analysis and Mining · doi:10.1007/s13278-021-00815-2 · Cited by 82
“This is especially difficult given inau-thentic behaviour is hard for humans to judge by eye (Cresci et al. 2017; Benkler et al. 2018; Jamieson 2020).”
BackgroundDetection of Malicious Social Bots Using Learning Automata With URL Features in Twitter Network
Rout, R. R., Lingam, G., & Somayajulu, D. V. L. N. (2020) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2020.2992223 · Cited by 64
“In this section, the performance of our proposed LA-MSBD algorithm is evaluated by considering two Twitter data sets, namely, Social Honeypot 1 data set [22] and The Fake Project 2 data set [21].”
BackgroundMethodInfluentialThe limited reach of fake news on Twitter during 2019 European elections
Cinelli, M., Cresci, S., Galeazzi, A., Quattrociocchi, W., & Tesconi, M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0234689 · Cited by 63
Self-citationA Large-scale Behavioural Analysis of Bots and Humans on Twitter
Gilani, Z., Farahbakhsh, R., Gareth Tyson, A., & Crowcroft, J. (2019) · ACM Transactions on the Web · doi:10.1145/3298789 · Cited by 62
“tested known bots on Twitter to see if they are perceived differently than humans [14].”
BackgroundLOBO: Evaluation of Generalization Deficiencies in Twitter Bot Classifiers
Echeverra, J., De Cristofaro, E., Kourtellis, N., Leontiadis, I., Stringhini, G., & Zhou, S. (2018) · Proceedings of the 34th Annual Computer Security Applications Conference · doi:10.1145/3274694.3274738 · Cited by 61
Deep Learning Based Social Bot Detection on Twitter
Arın, E., & Kutlu, M. (2023) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2023.3254429 · Cited by 66
Misinformation and Its Correction
Wittenberg, C., & Berinsky, A. J. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.009 · Cited by 64
BERT Model for Fake News Detection Based on Social Bot Activities in the COVID-19 Pandemic
Heidari, M., Zad, S., Hajibabaee, P., Malekzadeh, M., HekmatiAthar, S., Uzuner, Ö., & Jones, J. H. (2021) · IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon53757.2021.9666618 · Cited by 58
“Cresci authors bought Fake followers of Twitter accounts from different websites [46].”
BackgroundMethodCognitive Network Science for Understanding Online Social Cognitions: A Brief Review
Stella, M. (2021) · Topics in Cognitive Science · doi:10.1111/tops.12551 · Cited by 56
“…of thinking about specific news, with relevance for understanding about how social bots (Ferrara et al., 2016, Stella et al., 2018), fake accounts (Cresci et al., 2017) and trolls (Monakhov, 2020) influenced other users; (v) profile online users in terms of their digital footprint based on the…”
BackgroundBotMoE: Twitter Bot Detection with Community-Aware Mixtures of Modal-Specific Experts
Liu, Y., Tan, Z., Wang, H., Feng, S., Zheng, Q., & Luo, M. (2023) · International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR) · doi:10.1145/3539618.3591646 · Cited by 61
“However, in part due to the awareness of these feature-based detectors, bot operators began to tamper with user metadata [6, 8], leaving featurebased approaches less effective.”
BackgroundBetter Safe Than Sorry: An Adversarial Approach to Improve Social Bot Detection
Cresci, S., Petrocchi, M., Spognardi, A., & Tognazzi, S. (2019) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3292522.3326030 · Cited by 52
“g- [5, 8, 11, 13, 22, 48], and networkcharacteristics [5, 31, 47, 48, 51–53] of the accounts. However, until now, new detection systems have been developed only as a consequence of spambot evolutions [12]. In fact, no work has ever been done towards studying, and possibly anticipating, such evolutions. In other words,…”
BackgroundSelf-citationBots, Elections, and Social Media: A Brief Overview
Ferrara, E. (2020) · Lecture notes in social networks · doi:10.1007/978-3-030-42699-6_6 · Cited by 66
“Other recent surveys propose complementary or alternative taxonomies that are worth considering as well [71,20,20,86].”
BackgroundСтукал, Д., Sanovich, S., Tucker, J. A., & Bonneau, R. (2019) · SAGE Open · doi:10.1177/2158244019827715 · Cited by 59
Influence of augmented humans in online interactions during voting events
Stella, M., Cristoforetti, M., & De Domenico, M. (2019) · PLOS ONE · doi:10.1371/journal.pone.0214210 · Cited by 51
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [7, 10, 12, 13].”
BackgroundGANBOT: a GAN-based framework for social bot detection
Najari, S., Salehi, M., & Farahbakhsh, R. (2021) · Social Network Analysis and Mining · doi:10.1007/s13278-021-00800-9 · Cited by 51
“They can influence and bias users’ opinion via spreading malicious manipulated contents (Ferrara et al. 2016; Halawa et al. 2019) such as spam or fake review contents (Cresci et al. 2017; Aghakhani et al. 2018) on social media.”
BackgroundMethodBot detection using a single post on social media
Mohammad, S., Khan, M. U. S., Ali, M., Liu, L., Shardlow, M., & Nawaz, R. (2019) · World Conference on Smart Trends in Systems, Security and Sustainability (WorldS4) · doi:10.1109/worlds4.2019.8903989 · Cited by 48
“We have used a publicly available dataset [11] for our experiments.”
MethodMeasuring Bot and Human Behavioral Dynamics
Pozzana, I., & Ferrara, E. (2020) · Frontiers in Physics · doi:10.3389/fphy.2020.00125 · Cited by 49
Ephemeral Astroturfing Attacks: The Case of Fake Twitter Trends
Elmas, T., Overdorf, R., Özkalay, A. F., & Aberer, K. (2021) · IEEE European Symposium on Security and Privacy · doi:10.1109/eurosp51992.2021.00035 · Cited by 50
Facts and Where to Find Them: Empirical Research on Internet Platforms and Content Moderation
Keller, D., & Leerssen, P. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.011 · Cited by 39
Systematic Literature Review of Social Media Bots Detection Systems
Ellaky, Z., Benabbou, F., & Ouahabi, S. (2023) · Journal of King Saud University - Computer and Information Sciences · doi:10.1016/j.jksuci.2023.04.004 · Cited by 38
“They used four known datasets ICWSM17 (Varol et al., 2017), RAID11 (Liu et al., 2017), WWW17 (Yang et al., 2013), and Cresci-17 (Cresci et al., 2017).”
InfluentialDemocratic Transparency in the Platform Society
Gorwa, R., & Ash, T. G. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.013 · Cited by 71
Bots and Computational Propaganda: Automation for Communication and Control
Woolley, S. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.006 · Cited by 41
“Spambots, conversely, are built to generate noise (Cresci et al. 2017 ).”
BackgroundInfluentialA Survey of Attacks Against Twitter Spam Detectors in an Adversarial Environment
Imam, N. H., & Vassilakis, V. G. (2019) · Robotics · doi:10.3390/robotics8030050 · Cited by 36
“A recent study noted that the increase in OSN spammers, who distribute unsolicited spam and advertise untrustworthy products, affects the public’s perception of companies, which can eventually lead to people developing biased opinions [16].”
BackgroundBots in Social and Interaction Networks
Mendoza, M., Tesconi, M., & Cresci, S. (2020) · ACM Transactions on Information Systems · doi:10.1145/3419369 · Cited by 36
“By taking into account patterns of coordination, this annotation process offers higher guarantees with regards to the correct labeling of sophisticated bots that, if analyzed individually, are likely to be mistaken for legitimate accounts [13].”
BackgroundMethodResultInfluentialSelf-citationAn unsupervised method for social network spammer detection based on user information interests
Koggalahewa, D. N., Xu, Y., & Foo, E. (2022) · Journal Of Big Data · doi:10.1186/s40537-021-00552-5 · Cited by 40
“The Social Honeypot [18], HSpam14 [19] and The Fake Project [20] datasets are used to evaluate our approach.”
BackgroundMethodAn Attention-Based Graph Neural Network for Spam Bot Detection in Social Networks
Zhao, C., Xin, Y., Li, X., Zhu, H., Yang, Y., & Chen, Y. (2020) · Applied Sciences · doi:10.3390/app10228160 · Cited by 37
“The behavioral features of a single user may seem legitimate, but when multiple such users appear as a group, they will show anomaly features [8].”
BackgroundBots, disinformation, and the first impeachment of U.S. President Donald Trump
Rossetti, M., & Zaman, T. R. (2023) · PLoS ONE · doi:10.1371/journal.pone.0283971 · Cited by 34
“For this reason, other methods have been developed which focus on analyzing collective behaviors and similarities among groups of accounts [23, 29, 30, 33, 34], or including such analysis in a hybrid “tiered” approach [35].”
BackgroundMethodBot stamina: examining the influence and staying power of bots in online social networks
Schuchard, R., Crooks, A. T., Stefanidis, A., & Croitoru, A. (2019) · Applied Network Science · doi:10.1007/s41109-019-0164-x · Cited by 39
“Relying upon a single bot detection platform helped validate this study’s applied network analysis methods, but a sole source detection algorithm is not sufficient for overcoming known specific limitations that currently challenge all open-source bot detection results (Subrahmanian et al. 2016; Cresci et al. 2017).”
BackgroundMethodInfluentialLingam, G., Rout, R. R., Somayajulu, D. V. L. N., & Ghosh, S. K. (2020) · IEEE Systems Journal · doi:10.1109/jsyst.2020.3034416 · Cited by 33
“…Q -value function; 3) to develop a SIU-ICD algorithm to minimize the disseminating of spam-content through influential communities in Twitter network; 4) experimentation using Social Honeypot dataset [16] and The Fake Project dataset [17] demonstrates the efficacy of our proposed algorithms.”
MethodGrossi, V., Rapisarda, B., Giannotti, F., et al. (2018) · International Journal of Data Science and Analytics · doi:10.1007/s41060-018-0126-x · Cited by 32
“The ability of combiningdata sensedboth byopportunistic and participatory methods is expected to open new avenues for learning systems, capable of recognizing and mapping human behaviour at global and local scale [15,16].”
BackgroundDemocratic Creative Destruction? The Effect of a Changing Media Landscape on Democracy
Nielsen, R. K., & Fletcher, R. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.008 · Cited by 35
Lingam, G., Rout, R. R., Somayajulu, D. V. L. N., & Das, S. K. (2020) · ACM Asia Conference on Computer and Communications Security (ASIA CCS) · doi:10.1145/3320269.3384770 · Cited by 32
Smart, B., Watt, J., Benedetti, S., Mitchell, L., & Roughan, M. (2022) · Lecture notes in computer science · doi:10.1007/978-3-031-19097-1_3 · Cited by 42
“[9] measured Twitter’s current capabilities of detecting social spambots.”
Real-time Detection of Content Polluters in Partially Observable Twitter Networks
Nasim, M., Nguyen, A., Lothian, N., Cope, R. C., & Mitchell, L. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3184558.3191574 · Cited by 32
“The importance of detecting bots in online social media has produced an active research area on this topic [9, 21].”
BackgroundDon’t @ Me: Experimentally Reducing Partisan Incivility on Twitter
Munger, K. (2021) · Journal of Experimental Political Science · doi:10.1017/xps.2020.14 · Cited by 31
“Increasingly sophisticated “social spam bots” have been developed that are difficult for human coders to detect (Cresci et al., 2017).”
BackgroundShehnepoor, S., Togneri, R. B., Liu, W., & Bennamoun, M. (2021) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2021.3123876 · Cited by 35
Screening Out Social Bots Interference: Are There Any Silver Bullets?
Zago, M., Nespoli, P., Papamartzivanos, D., Perez, M. G., Marmol, F. G., Kambourakis, G., & Perez, G. M. (2019) · IEEE Communications Magazine · doi:10.1109/mcom.2019.1800520 · Cited by 30
Emotional Bots: Content-based Spammer Detection on Social Media
Andriotis, P., & Takasu, A. (2018) · IEEE International Workshop on Information Forensics and Security · doi:10.1109/wifs.2018.8630760 · Cited by 34
“The final dataset ( WWW17 ) [7] consists of various accounts (14,398 in total).”
BackgroundHays, C., Schutzman, Z., Raghavan, M., Walk, E., & Zimmer, P. (2023) · International World Wide Web Conference (WWW) · doi:10.1145/3543507.3583214 · Cited by 29
“…during the 2018 US elections. stock-2018 [13] are accounts with high volumes of tweets that tagged stock microblogs with cashtags. genuine-accounts-cresci [11] is purportedly a random sample of human Twitter users, confirmed to be genuine by their response to a nat-ural language question.”
BackgroundMethodInfluentialKouvela, M., Dimitriadis, I., & Vakali, A. I. (2020) · International Conference on Management of Emergent Digital EcoSystems (MEDES) · doi:10.1145/3415958.3433075 · Cited by 31
“The same observation is reached in [50] and in [16], which proceed to a more fine-grained classification of bots, providing separate datasets for each type of bots (celebrities, porn bots,etc.).”
BackgroundCresci, S., Petrocchi, M., Spognardi, A., & Tognazzi, S. (2018) · International World Wide Web Conference (WWW) · doi:10.1145/3184558.3191595 · Cited by 28
“Most notably, spambot evolution still goes on: recent investigations highlight that new waves of social spambots are rising [1, 3, 6].”
BackgroundSelf-citationBots Acting Like Humans: Understanding and Preventing Harm
Daniel, F., Cappiello, C., & Benatallah, B. (2019) · IEEE Internet Computing · doi:10.1109/mic.2019.2893137 · Cited by 28
Characterizing Social Bots Spreading Financial Disinformation
Tardelli, S., Avvenuti, M., Tesconi, M., & Cresci, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-49570-1_26 · Cited by 29
Self-citationRostami, R. R. (2020) · Webology · doi:10.14704/web/v17i1/a204 · Cited by 26
T-Bot: AI-based social media bot detection model for trend-centric twitter network
Gera, S., & Sinha, A. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00897-6 · Cited by 25
“The rapid rise in the number of users participating in OSN conversations has inclined the researchers towards detecting social media bots (Cresci et al. (2017), Perna and Tagarelli (2018), Varol et al. (2017), Morstatter et al. (2016), Zhang and Gupta (2018b)).”
BackgroundFine-Tuned Understanding: Enhancing Social Bot Detection With Transformer-Based Classification
Sallah, A., Alaoui, E. A. A., Agoujil, S., Wani, M. A., Hammad, M. A., Maleh, Y., & El‐Latif, A. A. A. (2024) · IEEE Access · doi:10.1109/access.2024.3440657 · Cited by 28
Comparative Media Regulation in the United States and Europe
Fukuyama, F., & Grotto, A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.010 · Cited by 28
Application of the Benford’s law to Social bots and Information Operations activities
Madahali, L., & Hall, M. (2020) · International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA) · doi:10.1109/cybersa49311.2020.9139709 · Cited by 26
“These accounts are from Crecsi 2017 dataset [37].”
BackgroundMethodRandom Forest Twitter Bot Classifier
Schnebly, J., & Sengupta, S. (2019) · IEEE Annual Computing and Communication Workshop and Conference (CCWC) · doi:10.1109/ccwc.2019.8666593 · Cited by 24
Analyzing Twitter Users' Behavior Before and After Contact by the Russia's Internet Research Agency
Dutta, U., Hanscom, R., Zhang, J. S., Han, R. Y., Lehman, T. S., Lv, Q., & Mishra, S. (2021) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3449164 · Cited by 24
Metadata
- Authors
- Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M.
- Affiliations
- University of Pisa (IT)
- Nokia Bell Labs & University of Padua, Paris, France
- National Research Council (IT)
- Technical University of Denmark (DK)
- Venue
- Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion
- Publisher
- ACM Press
- Published
- 2017
- Pages
- 963-972
- OpenAlex
- W4289253852
- Cited by
- 381
- References
- 50
- Citations by year
- 2026: 31 · 2025: 42 · 2024: 48 · 2023: 48 · 2022: 48 · 2021: 46 · 2020: 62 · 2019: 37 · 2018: 15 · 2017: 4
- FWCI
- 24.78
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
- Funders
- EU (654024, 675320)
- Regione Toscana (SmartNews: Social sensing for Breaking News)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- Twitter, spam social media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
