The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.
The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Open accessAbstract
Recent studies in social media spam and automation provide anecdotal argumentation of the rise of a new generation of spambots, so-called social spambots. Here, for the first time, we extensively study this novel phenomenon on Twitter and we provide quantitative evidence that a paradigm-shift exists in spambot design. First, we measure current Twitter's capabilities of detecting the new social spambots. Later, we assess the human performance in discriminating between genuine accounts, social spambots, and traditional spambots. Then, we benchmark several state-of-the-art techniques proposed by the academic literature. Results show that neither Twitter, nor humans, nor cutting-edge applications are currently capable of accurately detecting the new social spambots. Our results call for new approaches capable of turning the tide in the fight against this raising phenomenon. We conclude by reviewing the latest literature on spambots detection and we highlight an emerging common research trend based on the analysis of collective behaviors. Insights derived from both our extensive experimental campaign and survey shed light on the most promising directions of research and lay the foundations for the arms race against the novel social spambots. Finally, to foster research on this novel phenomenon, we make publicly available to the scientific community all the datasets used in this study.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper provides quantitative evidence that a new generation of Twitter “social spambots” mimics genuine users closely enough to evade Twitter, human annotators, and several state-of-the-art detection tools. It argues for a paradigm shift toward group-level/collective-behavior detection and releases annotated datasets.
Key findings
- Twitter’s own suspensions barely affect social spambots: survival rates were 95.2%, 96.1%, and 99.6% for social spambots #1–#3, close to genuine accounts at 96.5%; by contrast, traditional spambots #2 were 1.0% alive / 99.0% suspended, and fake followers were 25.4% alive / 73.5% suspended.
- Human annotators failed on social spambots: accuracy 0.2355 with 1,065 false negatives, versus 0.9136 on traditional spambots and 0.9201 on genuine accounts; Fleiss’ kappa was 0.186 for social spambots, 0.007 for traditional spambots, and 0.410 for genuine accounts.
- Established detection tools also failed: BotOrNot? achieved F-Measure 0.288 and MCC 0.174 on test set #1; Yang et al. achieved F-Measure 0.261 and MCC 0.043; Miller et al. performed worst.
- Group-level approaches performed much better: Ahmed et al. modified with fastgreedy community detection reached MCC 0.886 on test set #1 and 0.847 on test set #2; Cresci et al.’s digital DNA reached MCC 0.952 and 0.867, respectively.
Methods
- The study used Twitter datasets: 3,474 genuine accounts / 8,377,522 tweets; social spambots #1 (991 accounts / 1,610,176 tweets), #2 (3,457 / 428,542), #3 (464 / 1,418,626); traditional spambots #1–#4; fake followers (3,351 / 196,027); and two mixed test sets of 1,982 and 928 accounts.
- Twitter monitoring used API error codes 63 (suspended) and 50 (deleted) to measure account survival. Crowdsourcing used CrowdFlower with 4,428 accounts, at least 3 contributors per account, 247 trusted contributors from 42 countries, 13,284 answers, majority voting, and a 70% test-question accuracy threshold.
- The authors reproduced and benchmarked BotOrNot?, Yang et al.’s supervised classifier, Miller et al.’s stream clustering, Ahmed et al.’s graph clustering (modified with fastgreedy), and Cresci et al.’s digital DNA, evaluating precision, recall, specificity, accuracy, F-Measure, and MCC on the test sets.
Limitations
- Stated: BotOrNot?’s authors note its performance against evolved spambots may be worse than originally reported; Viswanath et al.’s technique cannot directly spot individual tampering accounts; and Table 7 is explicitly not complete but illustrative of an emerging trend.
- Stated/evident: 142 accounts (3.2%) were deleted, suspended, or protected during the crowdsourcing campaign, leaving 4,286 instead of 4,428; contributors were tech-savvy Twitter users and filtered by >70% test accuracy, so human-performance results may not generalize to all users.
- Evident: the datasets come from specific Twitter campaigns and years and rely on manual annotation; the social-spambot ground truth used group comparison, and the paper does not establish performance on other platforms or future spambot generations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
39 Own result · 21 Own interpretation · 16 Not checked yet · 10 No basis given · 5 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, we measured a slightly higher agreement for the classification of social spambots than for traditional ones, with κ = 0 . 186.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown, social spambots feature very small differences with respect to genuine accounts ( ∼ ± 3%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 2, genuine accounts feature a very high survival rate (96.5%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, we found such automated accounts to be similar to genuine ones in every way.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some of these differences are not even statistically significant, according to a chi-square test.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The deceitful activity was carried out by spamming URLs pointing to the advertised products.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instead, they showed very little agreement while classifying traditional spambots, as represented by κ = 0 . 007.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So far, traditional spambots #1 have largely managed to evade suspension, despite dating back to 2009.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, we uncovered a third group of social bots, social spambots #3 , which advertise products on sale on Amazon.com .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, among the no longer available accounts, the vast majority have been deleted by the original owner, rather than suspended by Twitter.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
- Probably not a substantial claim. 13
- It may state two separate claims as one. 21
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, we collected 13,284 answers given by 247 trusted contributors from 42 different countries.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of the 247 participating contributors, 60 of them ( ∼ 24%) evaluated our campaign, leading to a convincing aggregated score of 3.7/5, as shown in detail in Table 5.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Instead, traditional spambots #2 and fake followers show massive differences: ∼ ± 96% and ∼ ± 72%, respectively.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“So far, we have demonstrated that neither Twitter nor human operators are currently capable of identifying novel social spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Amazingly, we also found tens of human accounts who tried to engage in conversation with some of the spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Those accounts also represented credible sources of information since they all had thousands of followers and friends, the majority of which were genuine users 2 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All the 3 groups of social spambots feature very high survival rates, respectively 95.2%, 96.1%, and 99.6%.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in Table 6, overall, the human annotators obtained an accuracy of less than 0.24 on the social spambots, with more than 1,000 False Negatives (FN), meaning that contributors classified more than 1,000 accounts as genuine, when they actually belonged to the dataset of the last generation of spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The vast majority of tweets were harmless messages, occasionally interspersed by tweets mentioning a specific genuine (human) account and suggesting him to buy the VIP version of the app from a Web store.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, only 8.6% of the bots have been suspended, while 88.9% of them are still alive.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whereas the genuine accounts feature distributions that almost uniformly span across the possible range of values, social spambots have anomalous distributions.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As shown in Table 7, the established works benchmarked in Section 4 largely failed to detect the new wave of social spambots, one notable exception being the unsupervised approach proposed in [2].”
What the sources say
- Not checked yetcites [2] F. Ahmed and M. Abulaish. A generic statistical approach for spam detection in online social networks. Computer Communications, 36(10):1120--1129, 2013. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in Table 7, our modified implementation proved effective in detecting social spambots, with an MCC = 0.886 for test set #1 and MCC = 0.847 for test set #2 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This interesting result shows that, overall, the human contributors were able to correctly detect traditional spambots, as shown by the 0.91 accuracy, but also that contributors rarely agreed on the class.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We only recruited contributors who were tech-savvy and Twitter users themselves, in order to be reasonably sure about their knowledge of Twitter and its dynamics.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“This seems to suggest that Twitter's spambot detection mechanisms are still unable to accurately identify such accounts, while recent solutions proposed by Academia have succeeded in this task [44].”
What the sources say
- Not checked yetcites [44] C. Yang, R. Harkreader, and G. Gu. Empirical evaluation and new design for fighting evolving Twitter spammers. IEEE Trans. Inform. Forens. Sec., 8(8):1280--1293, 2013. source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results show that neither Twitter, nor humans, nor cutting-edge applications are currently capable of accurately detecting the new social spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results imply that humans generally failed in classifying social spambots (accuracy = 0 . 2355) and, furthermore, that they also were more in agreement on this mistake than they were when (correctly) classifying traditional spambots.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
In the paper
“BotOrNot? is a publicly-available service 7 to evaluate the similarity of a Twitter account with the known characteristics of social spambots [15].”
What the sources say
- Not checked yetcites [15] C. A. Davis, O. Varol, E. Ferrara, A. Flammini, and F. Menczer. BotOrNot: A system to evaluate social bots. In WWW companion. ACM, 2016. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our internal manual annotation has been carried out by comparing every account to all the others, in order to highlight possible similarities and common behaviors.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the political candidate was able to reach many more accounts in addition to his direct followers and managed to alter Twitter engagement metrics during the electoral campaign.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More worryingly, they also suggest that humans might not be able to detect social spambots in the wild, and to distinguish them from genuine accounts.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Recent studies in social media spam and automation provide anecdotal argumentation of the rise of a new generation of spambots, so-called social spambots .”
- Also stated in: § 1. INTRODUCTION · p. 1 · hedged
“Recent investigations anecdotally highlight how new waves of social spambots are rising [16, 48].”
What the sources say
- Not checked yetcites [16] E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. The rise of social bots. Commun. ACM, 59(7):96--104, 2016. source
- Not checked yetcites [48] J. Zhang, R. Zhang, Y. Zhang, and G. Yan. The rise of social botnets: Attacks and countermeasures. arXiv preprint arXiv:1603.02714, 2016.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § 1. INTRODUCTION · p. 1 · hedged
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, we required each account to be classified by at least 3 different contributors, with the final class decided by majority voting.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Low values of both Precision and Recall mean incomplete and unreliable spambot detection.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“However, the technique cannot directly spot the tampering accounts, and, thus, detection and removal of the single accounts must be performed using a separate methodology.”
What the sources say
- Not checked yetcites [40] B. Viswanath, M. A. Bashir, M. B. Zafar, S. Bouget, S. Guha, K. P. Gummadi, A. Kate, and A. Mislove. Strength in numbers: Robust tamper detection in crowd computations. In COSN. ACM, 2015. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, a further property - and advantage - of this recent research wave is that it proposes ad-hoc detection algorithms, rather than adopting generic and off-the-shelf machine learning algorithms.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can observe a paradigm-shift for research and development of spambot detection systems, which may exploit the new concepts to achieve better resilience and robustness and to withstand the next evolution of social media spambots.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The promising outcome of these novel approaches clearly indicates that this is a favorable research avenue.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite being based on different key concepts, all these systems propose novel algorithmic solutions and investigate groups of accounts as a whole, marking a significant difference with the previous literature.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrated the need for novel analytic tools capable of turning the tide in the arms race against such sophisticated spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
“API queries to a suspended account result in Twitter responding with the error code 63. API queries to accounts that have been deleted by their original owner result in Twitter responding with the error code 50.”
The paper relies on this without support.
The internal manual verification correctly certifies the automated nature of the accounts.
“All the accounts collected in this process have then undergone an internal manual verification phase to certify their automated nature.”
The paper relies on this without support.
The genuine accounts that answered the crowdsensing question are human-operated.
“All the replies to our questions were manually verified and all the 3,474 accounts that answered were certified as humans.”
The paper relies on this without support.
The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
“The level of agreement can be also interpreted as a proxy for the difficulty of a task.”
The paper relies on this without support.
Social spambots have a common goal, such as increasing someone's reputation score.
“since they do have a common goal (e.g., increasing someone's reputation score).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“these models can be evaded by malicious actors if they change their behaviors over time [40, 92]”
From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [40]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“we examined the behavioral similarities between accounts during annotation as described in the literature [12] and noticed significant similarities in the content-sharing behavior of anomalous accounts”
From Unsupervised detection of coordinated fake-follower campaigns on social media (2024) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“malicious actors continuously evolve their behaviors to evade platform detection [40, 58]”
From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [40]
In the citing paper
“Fake followers can be bought for as low as 12$ per 1,000 followers in the surface Web.”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Time went by and new studies acknowledged the rise of a third wave of bots that spread through online social networks from 2016 onwards [5, 13], as shown in panel C of Figure 4.”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“As such, botnets leave behind more traces of their automation and coordination than those left behind by sophisticated single bots [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Moreover, hundreds of tech-savvy social media users that participated in a crowdsourcing experiment were able to tell apart newer bots from legitimate users only 24% of the times. The same users were instead able of spotting older bots 91% of the times [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“The generality and ease of deployment of this detector are however counterbalanced by a reduced bot detection accuracy [5, 17].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Results showed that only 5% of newer bots are removed from social platforms, whereas older ones are removed 60% of the times [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“They also report that some bot classes, like social spambots, are hard to distinguish from human accounts.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“Cresci et al. [11] highlight that it is hard for human annotators to assign one label to one account to describe its bot class.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“The performance decay of current detection systems in the wild was reported by Cresci et al. [11], who showed that Twitter, human annotators, and state-of-the-art bot detection systems failed at discriminating between some new social spambots and genuine accounts.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [10, 7, 12, 13].”
From Influence of augmented humans in online interactions during voting events (2019) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| social spambots #2 | 3,457 | 3,322 (96.1%) | 1 (0.1%) | 134 (3.8%) |”
Reported 0.1 · recomputed 0.028927
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| social spambots #2 | 3,457 | 3,322 (96.1%) | 1 (0.1%) | 134 (3.8%) |”
Reported 3.8 · recomputed 3.876193
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Keywords · p. 2
“| social spambots #1 | retweeters of an Italian political candidate | 991 | 1,610,176 | 2012 | 3.1, 3.2 |”
§ 2. DATASETS · p. 3
“| social spambots #1 | 994 | 946 (95.2%) | 2 (0.2%) | 46 (4.6%) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| traditional spambots #1 | - 7.6% *** | - 0.8% | +8.7% *** |”
Reported 8.7 · recomputed 8.5
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| genuine accounts | 3,474 | 3,353 (96.5%) | 115 (3.3%) | 6 (0.1%) |”
Reported 0.1 · recomputed 0.172712
Citations
Cites
Finding community structure in very large networks
Clauset, A., Newman, M. E. J., & Moore, C. (2004) · Physical Review E · doi:10.1103/physreve.70.066111 · Cited by 7,390
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973
Gwet, K. L. (2014) · Medical Entomology and Zoology · Cited by 1,095
Density-Based Clustering over an Evolving Data Stream with Noise
Cao, F., Estert, M., Qian, W., & Zhou, A. (2006) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972764.29 · Cited by 1,033
BotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909
Detecting spammers on social networks
Stringhini, G., Kruegel, C., & Vigna, G. (2010) · Proceedings of the 26th Annual Computer Security Applications Conference · doi:10.1145/1920261.1920263 · Cited by 791
Graph Clustering Via a Discrete Uncoupling Process
van Dongen, S. (2008) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/040608635 · Cited by 745
Magno, G., Rodrigues, T., & Almeida, V. A. F. (2010) · Cited by 649
Uncovering social spammers: social honeypots + machine learning
Lee, K., Caverlee, J., & Webb, S. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835522 · Cited by 619
Fame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 475 · cited 1×
“Fake followers are another kind of malicious accounts that recently gained interest both from platform administrators and from the scientific world [13].”
BackgroundSelf-citation
Research works printed but not linked: 1
- M. Jiang, P. Cui, A. Beutel, C. Faloutsos, and S. Yang. Catching synchronized behaviors in large networks: A graph mining approach. ACM Trans. Knowl. Discov. Data, 10(4):35:1--35:27, 2016.
Cited by
Sorted by citations
Items 101–200 of 382
Bot Datasets on Twitter: Analysis and Challenges
Samper-Escalante, L. D., Loyola‐González, O., Monroy, R., & Medina‐Pérez, M. A. (2021) · Applied Sciences · doi:10.3390/app11094105 · Cited by 26
Online Political Advertising in the United States
Fowler, E. F., Franz, M. M., & Ridout, T. N. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.007 · Cited by 26
Conclusion: The Challenges and Opportunities for Social Media Research
Persily, N., & Tucker, J. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.014 · Cited by 24
Alothali, E., Hayawi, K., & Alashwal, H. (2021) · Social Network Analysis and Mining · doi:10.1007/s13278-021-00786-4 · Cited by 25
Alothali, E., Salih, M., Hayawi, K., & Alashwal, H. (2022) · Applied Sciences · doi:10.3390/app12168117 · Cited by 24
Ellaky, Z., Benabbou, F., Matrane, Y., & Qaqa, S. (2024) · IEEE Access · doi:10.1109/access.2024.3430859 · Cited by 33
“The ‘‘Cresci-2017’’ [34] dataset is a widely used dataset in the field of social media bot detection.”
MethodTwitter Bot Detection with Reduced Feature Set
Abreu, J. V. F., Ralha, C. G., & Gondim, J. J. C. (2020) · IEEE International Conference on Intelligence and Security Informatics (ISI) · doi:10.1109/isi49825.2020.9280525 · Cited by 25
“The dataset provided by [20] is used including several files in the csv format containing information related to user profiles and tweets.”
BackgroundMethodInfluentialCai, Z., Tan, Z., Lei, Z., Zhu, Z., Wang, H., Zheng, Q., & Luo, M. (2024) · ACM International Conference on Web Search and Data Mining (WSDM) · doi:10.1145/3616855.3635843 · Cited by 31
“The percentage of blue points among all points is 99.59% on Cresci-2017 and 95.77% on TwiBot-20, which suggests that LM and GNN/MLP produce consistent prediction on datasets with and without graph structure.”
InfluentialYe, J., & Skiena, S. (2019) · ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD) · doi:10.1145/3292500.3330709 · Cited by 20
MGTAB: A Multi-Relational Graph-Based Twitter Account Detection Benchmark
Shi, S., Qiao, K., Liu, Z., Yang, J., Chen, C., Chen, J., & Yan, B. (2025) · Neurocomputing · doi:10.1016/j.neucom.2025.130490 · Cited by 60
(2017) · doi:10.1109/dsaa.2017.57 · Cited by 21
“Such studies offer fresh insights into the characteristics and fundamental properties of online behaviors and actions [1].”
BackgroundSelf-citationA language framework for modeling social media account behavior
Nwala, A. C., Flammini, A., & Menczer, F. (2023) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00410-9 · Cited by 31
Beyond Text: Multimodal Credibility Assessment Approaches for Online User-Generated Content
Choudhary, M., Chouhan, S. S., & Rathore, S. S. (2024) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3673236 · Cited by 20
“The analysis revealed that most multimodal UGC research focuses on identifying fake news.”
Social Botomics: A Systematic Ensemble ML Approach for Explainable and Multi-Class Bot Detection
Dimitriadis, I., Georgiou, K., & Vakali, A. I. (2021) · Applied Sciences · doi:10.3390/app11219857 · Cited by 22
“As bot behaviors evolve over time, the produced methodologies are sometimes unable to stay up to date and are rendered useless in detection campaigns [6], while it has been pointed out that focusing on the evasion strategies of bots rather than simply detecting their presence is the next logical step towards the evolu…”
BackgroundTrollHunter [Evader]: Automated Detection [Evasion] of Twitter Trolls During the COVID-19 Pandemic
Jachim, P., Sharevski, F., & Treebridge, P. (2020) · New Security Paradigms Workshop · doi:10.1145/3442167.3442169 · Cited by 21
“the dissemination of trolling content on social media platforms, often using social bots to amplify by creating misperceptions of consensus on a polarised topic [19].”
BackgroundContent Based Bot Detection using Bot Language Model and BERT Embeddings
Kumar, S., Garg, S., Vats, Y., & Parihar, A. S. (2021) · International Conference on Computer, Communication and Signal Processing (ICCCSP) · doi:10.1109/icccsp52374.2021.9465506 · Cited by 21
Friendship Preference: Scalable and Robust Category of Features for Social Bot Detection
Moghaddam, S. H., & Abbaspour, M. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3159007 · Cited by 23
“Cresci et al. also presented a labeled dataset of fake and human accounts that we name Cresci2017 [49], [50].”
BackgroundTwitter Bot Detection Using Diverse Content Features and Applying Machine Learning Algorithms
Alarfaj, F. K., Ahmad, H., Khan, H. U., Alomair, A. M., Almusallam, N., & Ahmed, M. (2023) · Sustainability · doi:10.3390/su15086662 · Cited by 19
Detection and impact estimation of social bots in the Chilean Twitter network
Mendoza, M., Providel, E., dos Santos, M. L. B., & Valenzuela, S. (2024) · Scientific Reports · doi:10.1038/s41598-024-57227-3 · Cited by 22
Over-Sampling Strategy in Feature Space for Graphs based Class-imbalanced Bot Detection
Shi, S., Qiao, K., Chen, C., Yang, J., Chen, J., & Yan, B. (2024) · International World Wide Web Conference (WWW) · doi:10.1145/3589335.3651544 · Cited by 28
“Among all bot detection datasets, only TwiBot-22, MGTAB, TwiBot-20, and Crecsi-15 contain graph structures.”
Social Media and Microblogs Credibility: Identification, Theory Driven Framework, and Recommendation
Qureshi, K. A., Malick, R. A. S., & Sabih, M. (2021) · IEEE Access · doi:10.1109/access.2021.3114417 · Cited by 23
Unsupervised detection of coordinated fake-follower campaigns on social media
Zouzou, Y., & Varol, O. (2024) · EPJ Data Science · doi:10.1140/epjds/s13688-024-00499-6 · Cited by 25
Using Social Media to Detect Fake News Information Related to Product Marketing: The FakeAds Corpus
Alnazzawi, N., Alsaedi, N., Alharbi, F., & Alaswad, N. (2022) · Data · doi:10.3390/data7040044 · Cited by 18
“The previous studies on fake news detection in social media focused on different topics, including: bot detection [5,18,19], predicting spammer behavior and detecting these spammers [20–25], tweets related to natural disasters that were fake, spam and legitimate [26] clickbait detection [27].”
BackgroundOn pushing DeepFake Tweet Detection capabilities to the limits
Gambini, M., Fagni, T., Falchi, F., & Tesconi, M. (2022) · 14th ACM Web Science Conference 2022 · doi:10.1145/3501247.3531560 · Cited by 19
Self-citationTwitter Buzzer Detection for Indonesian Presidential Election
Suciati, A., Wibisono, A., & Mursanto, P. (2019) · International Conference on Informatics and Computational Sciences (ICICoS) · doi:10.1109/icicos48119.2019.8982529 · Cited by 18
“The data that were used was from [10] The deep learning model was DeBD which uses CNNLSTM algorithm to detect social bots.”
MethodAutomated Identification of Social Media Bots Using Deepfake Text Detection
Saravani, S. M., Ray, I., & Ray, I. (2021) · Lecture Notes in Computer Science · doi:10.1007/978-3-030-92571-0_7 · Cited by 18
“Cresci’s dataset [8], also used in [14,18], is among the few such datasets; however, an important factor in choosing the dataset for us was the quality of the text and how similar the bot-generated Tweets are to human-generated ones.”
BackgroundNew Automation for Social Bots: From Trivial Behavior to AI-Powered Communication
Grimme, C., Pohl, J., Cresci, S., Lüling, R., & Preuss, M. (2022) · Lecture Notes in Computer Science · doi:10.1007/978-3-031-18253-2_6 · Cited by 19
Self-citationBotTriNet: A Unified and Efficient Embedding for Social Bots Detection via Metric Learning
Wu, J., Ye, X., & Man, Y. (2023) · International Symposium on Digital Forensics and Security (ISDFS) · doi:10.1109/isdfs58141.2023.10131839 · Cited by 17
“[17] published a real-world dataset of Twitter”
CGNN: A Compatibility-Aware Graph Neural Network for Social Media Bot Detection
Huang, H., Tian, H., Zheng, X., Zhang, X., Zeng, D. D., & Wang, F. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3396413 · Cited by 23
“Nevertheless, since social bots continue to evolve, certain features have gradually become ineffective [9], [49].”
BackgroundSharevski, F., Jachim, P., & Florek, K. (2020) · International Conference on Availability, Reliability and Security (ARES) · doi:10.1145/3407023.3407025 · Cited by 22
“The idea was to "manipulate the perception of public opinion and sow discord between people debating health issue topics" in order to perpetuate a latent state of disagreement among the American public [11].”
BackgroundMethodMalicious Bot Detection in Online Social Networks: Arming Handcrafted Features with Deep Learning
Mou, G., & Lee, K. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-60975-7_17 · Cited by 22
“Researchers have proposed approaches to detect specific types of malicious bots [6, 10, 14, 18, 30].”
BackgroundA Comparative Study of Bot Detection Techniques With an Application in Twitter Covid-19 Discourse
Antenore, M., Camacho, J. M., & Panizzi, E. (2022) · Social Science Computer Review · doi:10.1177/08944393211073733 · Cited by 21
“In this way, the researchers were able to cope, at least temporally, with the paradigm shift of bots (Cresci, Di Pietro, Petrocchi, Spognardi, Tesconi, 2017a) and the effort of bot developers to evade detection techniques (Cresci, 2020).”
An Evolutionary Computation Approach for Twitter Bot Detection
Rovito, L., Bonin, L., Manzoni, L., & Lorenzo, A. D. (2022) · Applied Sciences · doi:10.3390/app12125915 · Cited by 18
Shah, A., Varshney, S., & Mehrotra, M. (2024) · SN Computer Science · doi:10.1007/s42979-024-02839-9 · Cited by 16
“There were two datasets used: the Twitter dataset [42, 43] and the Instagram dataset [32].”
Bow-tie structures of twitter discursive communities
Mattei, M., Pratelli, M., Caldarelli, G., Petrocchi, M., & Saracco, F. (2022) · Scientific Reports · doi:10.1038/s41598-022-16603-7 · Cited by 24
Self-citationMultimodal Detection of Bots on X (Twitter) Using Transformers
Ilias, L., Kazelidis, I. M., & Askounis, D. (2024) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2024.3435138 · Cited by 17
“…(1) exploit only the user description field and create images according to the type and content of the tweets posted by each user, (2) create images consisting of three channels and train several pretrained models comparing their performances, (3) employ TwHIN-BERT which is a new multi-lingual…”
BackgroundMethodIdentification of credulous users on Twitter
Balestrucci, A., Nicola, R. D., Inverso, O., & Trubiani, C. (2019) · ACM Symposium on Applied Computing · doi:10.1145/3297280.3297486 · Cited by 16
INFOSHIELD: Generalizable Information-Theoretic Human-Trafficking Detection
Lee, M.-C., Vajiac, C., Kulshrestha, A., Lévy, S., Park, N., Jones, C., Reihaneh Rabbany, K., & Faloutsos, C. (2021) · IEEE International Conference on Data Engineering (ICDE) · doi:10.1109/icde51399.2021.00101 · Cited by 17
“The Twitter datasets are publicly available (see [4]).”
BackgroundMethodInfluentialFake accounts detection system based on bidirectional gated recurrent unit neural network
Benabbou, F., Boukhouima, H., & Sael, N. (2022) · International Journal of Power Electronics and Drive Systems/International Journal of Electrical and Computer Engineering · doi:10.11591/ijece.v12i3.pp3129-3137 · Cited by 18
MulBot: Unsupervised Bot Detection Based on Multivariate Time Series
Mannocci, L., Cresci, S., Monreale, A., Vakali, A. I., & Tesconi, M. (2022) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata55660.2022.10020363 · Cited by 21
Self-citationSchuchard, R., & Crooks, A. T. (2021) · PLoS ONE · doi:10.1371/journal.pone.0244309 · Cited by 14
“The constantly evolving sophistication of social bots has proven challenging for even the most promising detection algorithms developed to date [25].”
BackgroundEnhancing Sybil Detection via Social-Activity Networks: A Random Walk Approach
Zhang, X., Xie, H. L., Yi, P., & Lui, J. C. S. (2022) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2022.3151701 · Cited by 15
“First, starting from 991 public fake accounts [27], we performed a 2-level BFS (Breadth First Search) to crawl all the followers of above 991 fake accounts as well as the followers of the crawled followers.”
BackgroundMethodDisinformation in Social Networks and Bots: Simulated Scenarios of Its Spread from System Dynamics
Rincón, A. G., Barbosa, R. L. C., Segovia-García, N., & Franco, D. R. A. (2022) · Systems · doi:10.3390/systems10020034 · Cited by 15
Twitter Bot Detection Using Neural Networks and Linguistic Embeddings
Feng, W. Q., & Nguyen, U. T. (2023) · IEEE Open Journal of the Computer Society · doi:10.1109/ojcs.2023.3302286 · Cited by 18
“We prepared two test sets following [25].”
MethodZhang, M., Chen, Z., Liu, X., & Liu, J. (2023) · Asian Journal of Communication · doi:10.1080/01292986.2023.2261112 · Cited by 15
“The study employed data from the Cresci et al. 2017 data set, creating a comprehensive pro fi le dataset for over 6900 Twitter accounts; through this initiative social media can be used for broader research and cyber threat analysis.”
MethodAre We Secure from Bots? Investigating Vulnerabilities of Botometer
Torusdağ, M. B., Kutlu, M., & Selçuk, A. A. (2020) · doi:10.1109/ubmk50275.2020.9219433 · Cited by 15
It’s a Matter of Style: Detecting Social Bots through Writing Style Consistency
Cardaioli, M., Conti, M., Sorbo, A. D., Fabrizio, E., Laudanna, S., & Visaggio, C. A. (2021) · International Conference on Computer Communications and Networks (ICCCN) · doi:10.1109/icccn52240.2021.9522339 · Cited by 15
Bots fired: examining social bot evidence in online mass shooting conversations
Schuchard, R., Crooks, A. T., Stefanidis, A., & Croitoru, A. (2019) · Humanities and Social Sciences Communications · doi:10.1057/s41599-019-0359-x · Cited by 14
“…sophistication of social bots mimicking human behaviour has proven to be quite difficult phenomenon for researchers to keep pace with (Cresci et al., 2017), multiple bot detection research platforms have been launched to aid researchers in the overall detection and analysis of social…”
BackgroundDetecting Malicious Social Bots: Story of a Never-Ending Clash
Cresci, S. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-39627-5_7 · Cited by 16
Self-citationAdversarial Machine Learning for Protecting Against Online Manipulation
Cresci, S., Petrocchi, M., Spognardi, A., & Tognazzi, S. (2021) · IEEE Internet Computing · doi:10.1109/mic.2021.3130380 · Cited by 15
“In fact, those features that initially seemed capable of identifying the sophisticated bots, started yielding unsatisfactory performance soon after their proposal [21].”
BackgroundSelf-citationFake news spreader detection using trust-based strategies in social networks with bot filtration
Rath, B., Salecha, A., & Srivastava, J. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00890-z · Cited by 15
Detecting fake news for COVID-19 using deep learning: a review
Zaheer, H., & Bashir, M. (2024) · Multimedia Tools and Applications · doi:10.1007/s11042-024-18564-7 · Cited by 14
Spam Detection in Social Networks based on Peer Acceptance
Koggalahewa, D. N., Xu, Y., & Foo, E. (2020) · Australasian Computer Science Week Multiconference (ACSW) · doi:10.1145/3373017.3373025 · Cited by 14
“Our new algorithm is tested with three publicly available datasets Social Honeypot [16], HSpam14 [17] and The Fake Project [18].”
MethodExploring the effect of streamed social media data variations on social network analysis
Weber, D., Nasim, M., Mitchell, L., & Falzon, L. (2021) · Social Network Analysis and Mining · doi:10.1007/s13278-021-00770-y · Cited by 13
“The question, however, of whether to remove content from social bots (bots that actively pretend to be human) depends on the research question at hand; because humans are easily fooled by social bots (Cresci et al. 2017; Nasim et al. 2018; Cresci 2020), their contribution to discussions may still be valid (unlike, e.”
BackgroundDetection of automated behavior on Twitter through approximate entropy and sample entropy
Gilmary, R., Akila, V., & Vaiyapuri, G. (2021) · Personal and Ubiquitous Computing · doi:10.1007/s00779-021-01647-9 · Cited by 15
Measuring user engagement with low credibility media sources in a controversial online debate
Vilella, S., Semeraro, A., Paolotti, D., & Ruffo, G. (2022) · EPJ Data Science · doi:10.1140/epjds/s13688-022-00342-w · Cited by 15
“[35] in 2017, neither Twitter, nor humans, nor up-to-date tools were capable of accurately detecting a novel family of social spam-bots.”
BackgroundBot or Human? Detection of DeepFake Text with Semantic, Emoji, Sentiment and Linguistic Features
Chong, A. T. Y., Chua, H. N., Jasser, M. B., & Wong, R. T. K. (2023) · IEEE International Conference on System Engineering and Technology (ICSET) · doi:10.1109/icset59111.2023.10295100 · Cited by 16
Ng, L. H. X., Kloo, I., Clark, S., & Carley, K. M. (2024) · Journal of Computational Social Science · doi:10.1007/s42001-024-00253-y · Cited by 14
Early Detection and Prevention of Malicious User Behavior on Twitter Using Deep Learning Techniques
Sánchez, R., Zubiaga, A., & Almeida, A. (2024) · IEEE Transactions on Computational Social Systems · doi:10.1109/tcss.2024.3419171 · Cited by 18
“…interactions, network structures, and community formations, facilitating the Caverlee-2011 [38] 15483 14833 Cresci-2015 [17] 1950 1950 Cresci-2017 [39] 7049 2764 Varol-2017 [40] 733 1495 Gilani-2017 [41] 1090 1413 Cresci-stock-2018 [42] 7102 6174 Midterm-2018 [43] 42446 8092 Pronbots-2019 [43]…”
Le, T., Tran-Thanh, L., & Lee, D. (2022) · International World Wide Web Conference (WWW) · doi:10.1145/3485447.3512215 · Cited by 14
Named Entity System for Tweets in Hindi Language
Jain, A., & Arora, A. (2018) · International Journal of Intelligent Information Technologies · doi:10.4018/ijiit.2018100104 · Cited by 13
A network topology approach to bot classification
Cornelissen, L. A., Barnett, R. J., Schoonwinkel, P., et al. (2018) · Annual Conference of the South African Institute of Computer Scientists and Information Technologists (SAICSIT) · doi:10.1145/3278681.3278692 · Cited by 13
CrediBot: Applying Bot Detection for Credibility Analysis on Twitter
Aguilera, A., Quinteros, P., Dongo, I., & Cardinale, Y. (2023) · IEEE Access · doi:10.1109/access.2023.3320687 · Cited by 12
Dissecting a social bot powered by generative AI: anatomy, new trends and challenges
López-Joya, S., Diaz-Garcia, J. A., Ruiz, M. D., & Martín-Bautista, M. J. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01410-5 · Cited by 13
“In Cresci et al. (2017) we can see that sophisticated bots can mimic human behaviour so well that even humans have trouble distinguishing a legitimate account from an account that is not.”
Word Embedding for Social Bot Detection Systems
Ellaky, Z., Benabbou, F., Ouahabi, S., & Sael, N. (2021) · International Conference on Intelligent Computing in Data Sciences (ICDS) · doi:10.1109/icds53782.2021.9626752 · Cited by 12
Political polarization on Twitter during the COVID-19 pandemic: a case study in Brazil
Brum, P. P. V., Teixeira, M. C., Vimieiro, R., Araújo, E. F. D. M., Meira, W., & Pappa, G. L. (2022) · Social Network Analysis and Mining · doi:10.1007/s13278-022-00949-x · Cited by 15
Detecting Fake Social Media Profiles Using the Majority Voting Approach
Patil, D. R., Pattewar, T. M., Punjabi, V. D., & Pardeshi, S. M. (2024) · ICST Transactions on Scalable Information Systems · doi:10.4108/eetsis.4264 · Cited by 13
“The y-axis, on the other hand, represents the number of entries linked with each characteristic in the dataset [34].”
BackgroundPerna, D., & Tagarelli, A. (2018) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3209542.3209563 · Cited by 12
“…by controlling a social media account, and when acting as a group (i.e., botnet) they can also pursue malevolent intents, such as fostering fame [11, 42], biasing public opinion [2, 39], spamming or limiting free speech by submerging important messages with a deluge of automated bot messages [16].”
BackgroundSupervised Classification of Social Spammers using a Similarity-based Markov Random Field Approach
El-Mawass, N., Honeiné, P., & Vercouter, L. (2018) · Multidisciplinary International Social Networks Conference (MISNC) · doi:10.1145/3227696.3227712 · Cited by 12
“[34] and several recent studies have independently reached the same result [10, 12].”
MethodResultSocialbots: Impacts, Threat-Dimensions, and Defense Challenges
Abulaish, M., & Fazil, M. (2020) · IEEE Technology and Society Magazine · doi:10.1109/mts.2020.3012327 · Cited by 11
Li, Q., Liu, Q., Liu, S., Di, X., Chen, S., & Zhang, H. (2023) · Communication and the Public · doi:10.1177/20570473231166157 · Cited by 12
“Unsupervised learning-based methods address clusters of accounts without labeled data sets (Cresci et al., 2017).”
MethodJaved, D., Jhanjhi, N. Z., Khan, N. A., Ray, S. K., Al-Dhaqm, A., & Kebande, V. R. (2025) · IEEE Access · doi:10.1109/access.2025.3551993 · Cited by 13
“The same conclusion is obtained from the [27] and [35], which goes to a finer-grained categorization of bots, giving distinct datasets for each sort of bot.”
Enhancing misinformation countermeasures: a multimodal approach to twitter bot detection
Arranz-Escudero, O., Quijano-Sánchez, L., & Liberatore, F. (2025) · Social Network Analysis and Mining · doi:10.1007/s13278-025-01435-w · Cited by 11
“Unlike previous studies that usually use around 20 features (Cresci et al. 2017; Abreu et al. 2020; Jabardi and Hadi 2020), our approach significantly increases this number.”
TL‐PBot: Twitter bot profile detection using transfer learning based on DNN model
Bibi, M., Qaisar, Z. H., Aslam, N., et al. (2024) · Engineering Reports · doi:10.1002/eng2.12838 · Cited by 11
A model for the Twitter sentiment curve
Aletti, G., Crimaldi, I., & Saracco, F. (2021) · PLoS ONE · doi:10.1371/journal.pone.0249634 · Cited by 11
“Due to this abrupt change of paradigm in the fruition of news, we observe a great increase of the diffusion of misinformation [9–11], that appears on the web via the use of automated [12–18] or genuine accounts [6, 18–22].”
BackgroundDispelling the Fake: Social Bot Detection Based on Edge Confidence Evaluation
Qiao, B., Zhou, W. J., Li, K., Li, S., & Hu, S. (2024) · IEEE Transactions on Neural Networks and Learning Systems · doi:10.1109/tnnls.2024.3396192 · Cited by 17
“Specifically, Kudugunta and Ferrara [17] and Wei and Nguyen [18] employ the text content information, while Botometer [1] and 1 Although previous works have proposed other social bot detection datasets, such as Cresci-17 [66], political-bots-2019 [67], and cresci-rtbust-2019 [68].”
Persily, N., & Tucker, J. A. (2020) · Cambridge University Press eBooks · doi:10.1017/9781108890960.002 · Cited by 11
Opinion manipulation on Farsi Twitter
Farzam, A., Moradi, P., Mohammadi, S., Padar, Z., & Siegel, A. A. (2023) · Scientific Reports · doi:10.1038/s41598-022-26921-5 · Cited by 15
“Comparing the findings of [23] with that of [24] shows the significant improvement of Twitter bots in imitating human behavior, a fact confirmed in [25, 26, 27, 24], as well as our manual observations.”
ResultMulti-input BiLSTM deep learning model for social bot detection
Messai, A., Hamida, Z. F., Drif, A., & Giordano, S. (2023) · International Conference on Advances in Electronics, Control and Communication Systems (ICAECCS) · doi:10.1109/icaeccs56710.2023.10104646 · Cited by 10
Are Deep Learning-Generated Social Media Profiles Indistinguishable from Real Profiles?
Rossi, S., Kwon, Y., Auglend, O. H., Mukkamala, R. R., Rossi, M., & Thatcher, J. B. (2023) · Hawaii International Conference on System Sciences (HICSS) · doi:10.24251/hicss.2023.017 · Cited by 14
Agents for Fighting Misinformation Spread on Twitter: Design Challenges
Piccolo, L. S. G., Blackwood, A. C., Farrell, T., & Mensio, M. (2021) · ACM Conference on Conversational User Interfaces (CUI) · doi:10.1145/3469595.3469628 · Cited by 12
IronSense: Towards the Identification of Fake User-Profiles on Twitter Using Machine Learning
Narayanan, A., Garg, A., Arora, I., Sureka, T., Sridhar, M., & Prasad, H. B. (2018) · doi:10.1109/icinpro43533.2018.9096687 · Cited by 11
“We obtained such a dataset of fake and genuine profiles on request from the authors of [1], the details of which are specified in detail in Section III.”
BackgroundMethodInfluentialSocial media bot detection using Dropout-GAN
Shukla, A., Jureček, M., & Stamp, M. (2024) · Journal of Computer Virology and Hacking Techniques · doi:10.1007/s11416-024-00521-5 · Cited by 12
“However, these methods are susceptible to sophisticated bots that may modify the features that are scrutinized for detection, and thereby evade feature-based detection methods [2].”
Evolving Cyber Threats, Combating Techniques, and Open Issues in Online Social Networks
Rao, S., Verma, A. K., & Bhatia, T. (2020) · Advances in information security, privacy, and ethics book series · doi:10.4018/978-1-7998-5728-0.ch012 · Cited by 9
Bot Detection in Twitter: An Overview
López-Joya, S., Diaz-Garcia, J. A., Ruiz, M. D., & Martín-Bautista, M. J. (2023) · Lecture notes in computer science · doi:10.1007/978-3-031-42935-4_11 · Cited by 10
Do You Really Follow Them? Automatic Detection of Credulous Twitter Users
Balestrucci, A., Nicola, R. D., Petrocchi, M., & Trubiani, C. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-33607-3_44 · Cited by 10
“rea, arguing on the differences, the contributions and the novelty w.r.t. our work. Section 6 concludes the paper. 2 The approach 2.1 Datasets We considerthreepublicly availabledatasets4: CR15[10],CR17[12], andVR17[26]. From the merging of these three datasets, we obtain a unique labeled dataset (human-operated/bot) of…”
BackgroundMethodResultInfluentialSelf-citationTask-agnostic representation learning of multimodal twitter data for downstream applications
Rivas, R., Paul, S., Hristidis, V., Papalexakis, E. E., & Roy–Chowdhury, A. K. (2022) · Journal Of Big Data · doi:10.1186/s40537-022-00570-x · Cited by 9
A hybrid framework for bot detection on twitter: Fusing digital DNA with BERT
Chawla, V., & Kapoor, Y. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-14730-5 · Cited by 10
Luo, J., & Jin, C. (2025) · EPJ Data Science · doi:10.1140/epjds/s13688-025-00583-5 · Cited by 8
“Bots on Twitter imitate human tweeting behavior of different types, fill out profile items in different ways, and follow each other to increase the number of their followers [18].”
Bayesian identification of bots using temporal analysis of tweet storms
Kirn, S. L., & Hinders, M. K. (2021) · Social Network Analysis and Mining · doi:10.1007/s13278-021-00783-7 · Cited by 7
“Sophisticated bot creators can continually evolve their methods to bypass protections put in place, leading to an arms race between bot creators and detection researchers (Cresci et al. 2017, 2019).”
BackgroundMalicious accounts detection from online social networks: a systematic review of literature
Sassi, I. B., & Yahia, S. B. (2021) · International Journal of General Systems · doi:10.1080/03081079.2021.1976773 · Cited by 8
VoxPop: An Experimental Social Media Platform for Calibrated (Mis)information Discourse
Sharevski, F., Jachim, P., Pieroni, E., & Jachim, N. (2021) · New Security Paradigms Workshop · doi:10.1145/3498891.3498893 · Cited by 11
“Since the beginning of social media, these groups have attempted to weaponize information, and they constantly evolve to evade the detection of platforms who are always behind playing catch-up [23].”
BackgroundEvolving Bots: The New Generation of Comment Bots and their Underlying Scam Campaigns in YouTube
Na, S. H., Cho, S., & Shin, S. (2023) · ACM Internet Measurement Conference (IMC) · doi:10.1145/3618257.3624822 · Cited by 11
“Research has been conducted on various platforms, including Twitter [4, 5, 13, 21–23, 28–30, 33], Facebook [5, 39], WeChat [40], and Reddit [7].”
Method(Un)Trendy Japan: Twitter bots and the 2017 Japanese general election
Mintal, J. M., & Vancel, R. (2019) · Politics in Central Europe · doi:10.2478/pce-2019-0027 · Cited by 8
“…except the training data for the bot detection model, are available in the Zenodo repository: doi: 10.5281/ zenodo.1413589 The training data for the bot detection model were used under license from Cresci et al. (2017), the data are available on request at mib.projects. iit.cnr.it/dataset.html.”
MethodInfluentialFu, C.-Q., Shi, S., Zhang, Y., et al. (2023) · Electronics · doi:10.3390/electronics13010056 · Cited by 7
Wikipedia in Wartime: Experiences of Wikipedians Maintaining Articles About the Russia-Ukraine War
Kurek, L., Budak, C., & Gilbert, É. (2025) · Proceedings of the ACM on Human-Computer Interaction · doi:10.1145/3711107 · Cited by 10
BotSSCL: Social Bot Detection with Self-Supervised Contrastive Learning
Akhtar, M. M., Bhuiyan, N. S., Masood, R., Ikram, M., & Kanhere, S. S. (2025) · Online Social Networks and Media · doi:10.1016/j.osnem.2025.100318 · Cited by 14
Challenges in machine learning-based social bot detection: a systematic review
Alkathiri, N., & Slhoub, K. (2025) · Discover Artificial Intelligence · doi:10.1007/s44163-025-00448-w · Cited by 8
“Traditional detection models frequently rely on fixed feature sets such as posting frequency, retweet behavior, linguistic patterns, or other behavioral patterns [99].”
Mewada, A., & Dewang, R. K. (2023) · Multimedia Tools and Applications · doi:10.1007/s11042-023-16685-z · Cited by 7
Hamida, Z. F., Refouf, A., Drif, A., & Giordano, S. (2022) · Informatica · doi:10.31449/inf.v46i6.4081 · Cited by 9
Metadata
- Authors
- Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M.
- Affiliations
- University of Pisa (IT)
- Nokia Bell Labs & University of Padua, Paris, France
- National Research Council (IT)
- Technical University of Denmark (DK)
- Venue
- Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion
- Publisher
- ACM Press
- Published
- 2017
- Pages
- 963-972
- OpenAlex
- W4289253852
- Cited by
- 381
- References
- 50
- Citations by year
- 2026: 31 · 2025: 42 · 2024: 48 · 2023: 48 · 2022: 48 · 2021: 46 · 2020: 62 · 2019: 37 · 2018: 15 · 2017: 4
- FWCI
- 24.78
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
- Funders
- EU (654024, 675320)
- Regione Toscana (SmartNews: Social sensing for Breaking News)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- Twitter, spam social media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
