The DOI matches its Crossref record; 48 of the 50 references it cites were found in a registry or an index; none of the 17 sources its claims cite is checked for support yet.
The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Open accessAbstract
Recent studies in social media spam and automation provide anecdotal argumentation of the rise of a new generation of spambots, so-called social spambots. Here, for the first time, we extensively study this novel phenomenon on Twitter and we provide quantitative evidence that a paradigm-shift exists in spambot design. First, we measure current Twitter's capabilities of detecting the new social spambots. Later, we assess the human performance in discriminating between genuine accounts, social spambots, and traditional spambots. Then, we benchmark several state-of-the-art techniques proposed by the academic literature. Results show that neither Twitter, nor humans, nor cutting-edge applications are currently capable of accurately detecting the new social spambots. Our results call for new approaches capable of turning the tide in the fight against this raising phenomenon. We conclude by reviewing the latest literature on spambots detection and we highlight an emerging common research trend based on the analysis of collective behaviors. Insights derived from both our extensive experimental campaign and survey shed light on the most promising directions of research and lay the foundations for the arms race against the novel social spambots. Finally, to foster research on this novel phenomenon, we make publicly available to the scientific community all the datasets used in this study.
Overview
AI snapshot
Written from the paper’s full text.
TL;DR
The paper provides quantitative evidence that a new generation of Twitter “social spambots” mimics genuine users closely enough to evade Twitter, human annotators, and several state-of-the-art detection tools. It argues for a paradigm shift toward group-level/collective-behavior detection and releases annotated datasets.
Key findings
- Twitter’s own suspensions barely affect social spambots: survival rates were 95.2%, 96.1%, and 99.6% for social spambots #1–#3, close to genuine accounts at 96.5%; by contrast, traditional spambots #2 were 1.0% alive / 99.0% suspended, and fake followers were 25.4% alive / 73.5% suspended.
- Human annotators failed on social spambots: accuracy 0.2355 with 1,065 false negatives, versus 0.9136 on traditional spambots and 0.9201 on genuine accounts; Fleiss’ kappa was 0.186 for social spambots, 0.007 for traditional spambots, and 0.410 for genuine accounts.
- Established detection tools also failed: BotOrNot? achieved F-Measure 0.288 and MCC 0.174 on test set #1; Yang et al. achieved F-Measure 0.261 and MCC 0.043; Miller et al. performed worst.
- Group-level approaches performed much better: Ahmed et al. modified with fastgreedy community detection reached MCC 0.886 on test set #1 and 0.847 on test set #2; Cresci et al.’s digital DNA reached MCC 0.952 and 0.867, respectively.
Methods
- The study used Twitter datasets: 3,474 genuine accounts / 8,377,522 tweets; social spambots #1 (991 accounts / 1,610,176 tweets), #2 (3,457 / 428,542), #3 (464 / 1,418,626); traditional spambots #1–#4; fake followers (3,351 / 196,027); and two mixed test sets of 1,982 and 928 accounts.
- Twitter monitoring used API error codes 63 (suspended) and 50 (deleted) to measure account survival. Crowdsourcing used CrowdFlower with 4,428 accounts, at least 3 contributors per account, 247 trusted contributors from 42 countries, 13,284 answers, majority voting, and a 70% test-question accuracy threshold.
- The authors reproduced and benchmarked BotOrNot?, Yang et al.’s supervised classifier, Miller et al.’s stream clustering, Ahmed et al.’s graph clustering (modified with fastgreedy), and Cresci et al.’s digital DNA, evaluating precision, recall, specificity, accuracy, F-Measure, and MCC on the test sets.
Limitations
- Stated: BotOrNot?’s authors note its performance against evolved spambots may be worse than originally reported; Viswanath et al.’s technique cannot directly spot individual tampering accounts; and Table 7 is explicitly not complete but illustrative of an emerging trend.
- Stated/evident: 142 accounts (3.2%) were deleted, suspended, or protected during the crowdsourcing campaign, leaving 4,286 instead of 4,428; contributors were tech-savvy Twitter users and filtered by >70% test accuracy, so human-performance results may not generalize to all users.
- Evident: the datasets come from specific Twitter campaigns and years and rely on manual annotation; the social-spambot ground truth used group comparison, and the paper does not establish performance on other platforms or future spambot generations.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
39 Own result · 21 Own interpretation · 16 Not checked yet · 10 No basis given · 5 Own method
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, we measured a slightly higher agreement for the classification of social spambots than for traditional ones, with κ = 0 . 186.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown, social spambots feature very small differences with respect to genuine accounts ( ∼ ± 3%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 2, genuine accounts feature a very high survival rate (96.5%).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Surprisingly, we found such automated accounts to be similar to genuine ones in every way.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Some of these differences are not even statistically significant, according to a chi-square test.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The deceitful activity was carried out by spamming URLs pointing to the advertised products.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Instead, they showed very little agreement while classifying traditional spambots, as represented by κ = 0 . 007.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“So far, traditional spambots #1 have largely managed to evade suspension, despite dating back to 2009.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Further, we uncovered a third group of social bots, social spambots #3 , which advertise products on sale on Amazon.com .”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, among the no longer available accounts, the vast majority have been deleted by the original owner, rather than suspended by Twitter.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our experiment, human contributors showed a decent agreement for the classification of genuine accounts, with κ = 0 . 410.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Finally, 89.29% (464 accounts) of the accounts that tweeted suspicious Amazon.com URLs were also certified as spambots.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The internal manual verification correctly certifies the automated nature of the accounts.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Among all the distinct retweeters of the Italian political candidate, 50.05% (991 accounts) were certified as spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The internal manual verification correctly certifies the automated nature of the accounts.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Every profile was accurately filled in with detailed - yet fake personal information such as a (stolen) photo, (fake) shortbio, (fake) location, etc.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Traditional spambots #1 have differences ∼ ± 8% that are highly significant ( p < 0 . 01) for alive and suspended accounts.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, the results of our crowdsourcing experiment confirmed that the traditional 'account-byaccount'annotation process used by human workers to evaluate social media accounts is no longer viable when applied to the detection of the novel wave of social spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also discovered a second group of social bots, which we labeled social spambots #2 , who spent several months promoting the #TALNTS hashtag.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Similarly to the retweeters of the Italian political candidate, also this family of spambots interleaved spam tweets with harmless and genuine ones.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“However, every time the political candidate posted a new tweet from his official account, all the automated accounts retweeted it in a time span of just a few minutes.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“We provide empirical evidence of the existence of a novel wave of Twitter spambots, which, up to now, has been just theorized [16].”
What the sources say
- Not checked yetcites [16] E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. The rise of social bots. Commun. ACM, 59(7):96--104, 2016. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The most common form of such human-to-spambot interaction was represented by a human reply to one of the spambot tweets quotes.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“By comparing the behaviors of different accounts, rather than by analyzing them one by one, we were able to spot the social spambots among all the collected accounts, as thoroughly described in Section 2.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, the accounts showed a tweeting behavior that was apparently similar to those of genuine accounts, with a few tweets posted every day, mainly quotes from popular people, songs, and YouTube videos.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Conversely, the simplest kind of malicious accounts, fake followers , have mostly been detected and suspended by Twitter.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Human detection performances for the two other groups of accounts, namely traditional spambots and genuine accounts, are instead quite satisfactory, with an accuracy of 0.91 and 0.92 respectively.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly, 94.50% (3,457 accounts) of the accounts who tweeted the #TALNTS hashtag resulted as spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: The internal manual verification correctly certifies the automated nature of the accounts.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, our experiments highlighted that the majority of existing automated systems, as well as crowdsourcing, erroneously label social spambots as genuine (human-operated) accounts.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“However, the approach failed to identify 2 distinct clusters, since accounts of both our test-sets were assigned to a single cluster.”
What the sources say
- Not checked yetcites [2] F. Ahmed and M. Abulaish. A generic statistical approach for spam detection in online social networks. Computer Communications, 36(10):1120--1129, 2013. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“However, novel social spambots tweet contents similar to that of genuine accounts (e.g., retweets of genuine tweets and famous quotes).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Its detection performances are particularly bad for the accounts of test set #1 - where the spambots are from the social spambots #1 group.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As shown in Table 7, BotOrNot? achieves rather unsatisfactory results for the accounts of both test set #1 and test set #2 (such datasets are described in Table 1).”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“The low values of F-Measure and Mathews Correlation Coefficient (MCC), respectively 0.288 and 0.174, are mainly due to the low Recall.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Results in Table 7 show that the system fails to correctly classify the novel social spambots.”
What the sources say
- Not checked yetcites [44] C. Yang, R. Harkreader, and G. Gu. Empirical evaluation and new design for fighting evolving Twitter spammers. IEEE Trans. Inform. Forens. Sec., 8(8):1280--1293, 2013. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Similarly to the results of the BotOrNot? service, the worst results of this system in both test set #1 and test set #2 are related to the Recall metric.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More specifically, as measured by the LCS metric, social spambots #1 and #3 feature a level of similarity much higher than that of genuine accounts.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We also performed a grid search simulation in order to test the best parameter configuration for MCL 8 , but to no avail.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In our experiments, neither humans nor state-of-the-art spambot detection applications managed to accurately detect the accounts belonging to this new wave of spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Results reported in Table 7 demonstrate that the digital DNA-based technique [14] achieves excellent detection performances.”
What the sources say
- Not checked yetcites [14] S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, and M. Tesconi. DNA-inspired online behavioral modeling and its application to spambot detection. IEEE Intelligent Systems, 5(31):58--64, 2016. source
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“As shown in Table 7, this achieved the worst performances among all those that we have benchmarked in this study.”
What the sources say
- Not checked yetcites [31] Z. Miller, B. Dickinson, W. Deitrick, W. Hu, and A. H. Wang. Twitter spammer detection using data stream clustering. Information Sciences, 260:64--73, 2014. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Our long-lasting experiment on malicious accounts survival rate in Twitter demonstrated that spambot detection is still an open issue.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Applying the technique to our datasets, the similarity curve of genuine accounts is significantly different from that of social spambots, as shown in Figure 4(c).”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, social spambots appear to be more similar to genuine accounts than to traditional spambots, with regards to Twitter suspensions.”
hedgedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
- Rests on an assumption: The genuine accounts that answered the crowdsensing question are human-operated.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Here, for the first time, we extensively study this novel phenomenon on Twitter and we provide quantitative evidence that a paradigm-shift exists in spambot design.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Notably, by designing test questions with traditional spambots and genuine accounts, and by enforcing the policy of at least 70% correct answers, we can guarantee that all our trusted contributors are typically able to detect traditional spambots and to distinguish them from genuine accounts.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Only the trusted contributors who answered correctly to more than the 70% of the test questions have been considered in our study.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Unfortunately, the characteristics of the new wave of social spambots are such that standard classification approaches, where a single account is evaluated according to a set of established features tested over known datasets, are no longer successful.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These important results further highlight the existence of a striking difference between traditional and social spambots.”
boostedcorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Even if the difference between the survival rate of social spambots and that of traditional spambots #1 is marginal, these results nonetheless suggest an increased difficulty for the detection of social spambots.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
In the paper
“However, since 2013, a number of research teams independently started to formalize new approaches, also from the algorithmic point of view, for detecting the coordinated and synchronized behavior that characterizes groups of automated malicious accounts [6].”
What the sources say
- Not checked yetcites [6] A. Beutel, W. Xu, V. Guruswami, C. Palow, and C. Faloutsos. CopyCatch: stopping group attacks by spotting lockstep behavior in social networks. In WWW. ACM, 2013. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Focusing on groups has the advantage that, no matter how sophisticated a single spambot can be, a large enough group of spambots will still leave traces of automation, since they do have a common goal (e.g., increasing someone's reputation score).”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Rests on an assumption: Social spambots have a common goal, such as increasing someone's reputation score.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This means that also this classifier labeled social spambots as genuine accounts.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“In turn, this represents a tendency of labeling social spambots #1 as genuine accounts.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“For this reason, an approach almost solely based on tweet content will not be able to achieve satisfactory results.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By accurately mimicking the characteristics of genuine users, these spambots are intrinsically harder to detect than those studied by Academia in the past years.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Thus, the technique proposed in [40] is capable of spotting the differences between groups of genuine accounts and the new wave of social spambots.”
What the sources say
- Not checked yetcites [40] B. Viswanath, M. A. Bashir, M. B. Zafar, S. Bouget, S. Guha, K. P. Gummadi, A. Kate, and A. Mislove. Strength in numbers: Robust tamper detection in crowd computations. In COSN. ACM, 2015. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Moreover, the already difficult problem to detect spambots in social media is bound to worsen, since the emergence of a new wave of so-called social spambots.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“By performing analyses at group level, this emerging trend might be able to significantly raise the bar for social spambots to evade detection.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: unclear — It is unclear whether it could be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Notably, although working at group level, [14] is capable of spotting single spambot accounts.”
What the sources say
- Not checked yetcites [14] S. Cresci, R. Di Pietro, M. Petrocchi, A. Spognardi, and M. Tesconi. DNA-inspired online behavioral modeling and its application to spambot detection. IEEE Intelligent Systems, 5(31):58--64, 2016. source
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
In the paper
“Among the 126 features, 95 are based on the textual content of tweets.”
What the sources say
- Not checked yetcites [31] Z. Miller, B. Dickinson, W. Deitrick, W. Hu, and A. H. Wang. Twitter spammer detection using data stream clustering. Information Sciences, 260:64--73, 2014. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
None of the sources cited for this claim have been checked yet.
In the paper
“Our initial claim, supported by preliminary work [16, 48], is that social spambots might be so sophisticatedly designed to make it very difficult to distinguish them from genuine accounts, if observed one by one.”
What the sources say
- Not checked yetcites [16] E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. The rise of social bots. Commun. ACM, 59(7):96--104, 2016. source
- Not checked yetcites [48] J. Zhang, R. Zhang, Y. Zhang, and G. Yan. The rise of social botnets: Attacks and countermeasures. arXiv preprint arXiv:1603.02714, 2016.
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Probably not a substantial claim. 13
- It may state two separate claims as one. 21
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Overall, we collected 13,284 answers given by 247 trusted contributors from 42 different countries.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Out of the 247 participating contributors, 60 of them ( ∼ 24%) evaluated our campaign, leading to a convincing aggregated score of 3.7/5, as shown in detail in Table 5.”
boosteddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Instead, traditional spambots #2 and fake followers show massive differences: ∼ ± 96% and ∼ ± 72%, respectively.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“So far, we have demonstrated that neither Twitter nor human operators are currently capable of identifying novel social spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: unclear — It is unclear why the paper says it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Amazingly, we also found tens of human accounts who tried to engage in conversation with some of the spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
None of the sources cited for this claim have been checked yet.
In the paper
“Those accounts also represented credible sources of information since they all had thousands of followers and friends, the majority of which were genuine users 2 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“All the 3 groups of social spambots feature very high survival rates, respectively 95.2%, 96.1%, and 99.6%.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in Table 6, overall, the human annotators obtained an accuracy of less than 0.24 on the social spambots, with more than 1,000 False Negatives (FN), meaning that contributors classified more than 1,000 accounts as genuine, when they actually belonged to the dataset of the last generation of spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The vast majority of tweets were harmless messages, occasionally interspersed by tweets mentioning a specific genuine (human) account and suggesting him to buy the VIP version of the app from a Web store.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Indeed, only 8.6% of the bots have been suspended, while 88.9% of them are still alive.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Whereas the genuine accounts feature distributions that almost uniformly span across the possible range of values, social spambots have anomalous distributions.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
In the paper
“As shown in Table 7, the established works benchmarked in Section 4 largely failed to detect the new wave of social spambots, one notable exception being the unsupervised approach proposed in [2].”
What the sources say
- Not checked yetcites [2] F. Ahmed and M. Abulaish. A generic statistical approach for spam detection in online social networks. Computer Communications, 36(10):1120--1129, 2013. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A result the paper reports from its own data — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“As reported in Table 7, our modified implementation proved effective in detecting social spambots, with an MCC = 0.886 for test set #1 and MCC = 0.847 for test set #2 .”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“This interesting result shows that, overall, the human contributors were able to correctly detect traditional spambots, as shown by the 0.91 accuracy, but also that contributors rarely agreed on the class.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“We only recruited contributors who were tech-savvy and Twitter users themselves, in order to be reasonably sure about their knowledge of Twitter and its dynamics.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“This seems to suggest that Twitter's spambot detection mechanisms are still unable to accurately identify such accounts, while recent solutions proposed by Academia have succeeded in this task [44].”
What the sources say
- Not checked yetcites [44] C. Yang, R. Harkreader, and G. Gu. Empirical evaluation and new design for fighting evolving Twitter spammers. IEEE Trans. Inform. Forens. Sec., 8(8):1280--1293, 2013. source
hedgeddirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Results show that neither Twitter, nor humans, nor cutting-edge applications are currently capable of accurately detecting the new social spambots.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“These results imply that humans generally failed in classifying social spambots (accuracy = 0 . 2355) and, furthermore, that they also were more in agreement on this mistake than they were when (correctly) classifying traditional spambots.”
plaincorrelational- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
- Rests on an assumption: The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
In the paper
“BotOrNot? is a publicly-available service 7 to evaluate the similarity of a Twitter account with the known characteristics of social spambots [15].”
What the sources say
- Not checked yetcites [15] C. A. Davis, O. Varol, E. Ferrara, A. Flammini, and F. Menczer. BotOrNot: A system to evaluate social bots. In WWW companion. ACM, 2016. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Our internal manual annotation has been carried out by comparing every account to all the others, in order to highlight possible similarities and common behaviors.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Thus, the political candidate was able to reach many more accounts in addition to his direct followers and managed to alter Twitter engagement metrics during the electoral campaign.”
plaindirectSingle case- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“More worryingly, they also suggest that humans might not be able to detect social spambots in the wild, and to distinguish them from genuine accounts.”
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
None of the sources cited for this claim have been checked yet.
In the paper
“Recent studies in social media spam and automation provide anecdotal argumentation of the rise of a new generation of spambots, so-called social spambots .”
- Also stated in: § 1. INTRODUCTION · p. 1 · hedged
“Recent investigations anecdotally highlight how new waves of social spambots are rising [16, 48].”
What the sources say
- Not checked yetcites [16] E. Ferrara, O. Varol, C. Davis, F. Menczer, and A. Flammini. The rise of social bots. Commun. ACM, 59(7):96--104, 2016. source
- Not checked yetcites [48] J. Zhang, R. Zhang, Y. Zhang, and G. Yan. The rise of social botnets: Attacks and countermeasures. arXiv preprint arXiv:1603.02714, 2016.
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
- Also stated in: § 1. INTRODUCTION · p. 1 · hedged
What the paper did — it cites no source for this, so there is none to check. The quote shows it is in the paper.
In the paper
“Furthermore, we required each account to be classified by at least 3 different contributors, with the final class decided by majority voting.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“Low values of both Precision and Recall mean incomplete and unreliable spambot detection.”
plaindirect- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
In the paper
“However, the technique cannot directly spot the tampering accounts, and, thus, detection and removal of the single accounts must be performed using a separate methodology.”
What the sources say
- Not checked yetcites [40] B. Viswanath, M. A. Bashir, M. B. Zafar, S. Bouget, S. Guha, K. P. Gummadi, A. Kate, and A. Mislove. Strength in numbers: Robust tamper detection in crowd computations. In COSN. ACM, 2015. source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“In addition, a further property - and advantage - of this recent research wave is that it proposes ad-hoc detection algorithms, rather than adopting generic and off-the-shelf machine learning algorithms.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: fails — The paper’s conclusions may not depend on it.
- Based: fails — The paper may give no reason why it holds.
Probably not a substantial claim. The paper’s conclusions may not depend on it. The paper may give no reason why it holds.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We can observe a paradigm-shift for research and development of spambot detection systems, which may exploit the new concepts to achieve better resilience and robustness and to withstand the next evolution of social media spambots.”
hedgedconditional- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence. It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“The promising outcome of these novel approaches clearly indicates that this is a favorable research avenue.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: fails — It may be too vague to check against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim. It may be too vague to check against evidence.
The paper states this without a source or data behind it here, so there is nothing to check it against. The quote shows it is in the paper.
In the paper
“Despite being based on different key concepts, all these systems propose novel algorithmic solutions and investigate groups of accounts as a whole, marking a significant difference with the previous literature.”
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
It may state two separate claims as one.
A conclusion the paper draws from its own results — it cites no other source for it, so there is none to check. The quote shows it is in the paper.
In the paper
“We demonstrated the need for novel analytic tools capable of turning the tide in the arms race against such sophisticated spambots.”
boosted- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
Probably not a substantial claim.
The paper relies on this without support.
Twitter's API error codes 63 and 50 accurately indicate that an account has been suspended by Twitter or deleted by its original owner, respectively.
“API queries to a suspended account result in Twitter responding with the error code 63. API queries to accounts that have been deleted by their original owner result in Twitter responding with the error code 50.”
The paper relies on this without support.
The internal manual verification correctly certifies the automated nature of the accounts.
“All the accounts collected in this process have then undergone an internal manual verification phase to certify their automated nature.”
The paper relies on this without support.
The genuine accounts that answered the crowdsensing question are human-operated.
“All the replies to our questions were manually verified and all the 3,474 accounts that answered were certified as humans.”
The paper relies on this without support.
The level of agreement measured by Fleiss' kappa can be interpreted as a proxy for the difficulty of a task.
“The level of agreement can be also interpreted as a proxy for the difficulty of a task.”
The paper relies on this without support.
Social spambots have a common goal, such as increasing someone's reputation score.
“since they do have a common goal (e.g., increasing someone's reputation score).”
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“these models can be evaded by malicious actors if they change their behaviors over time [40, 92]”
From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [40]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“we examined the behavioral similarities between accounts during annotation as described in the literature [12] and noticed significant similarities in the content-sharing behavior of anomalous accounts”
From Unsupervised detection of coordinated fake-follower campaigns on social media (2024) · cites this paper as [12]
The cited source says what the claim attributes to it.
In the citing paper
“malicious actors continuously evolve their behaviors to evade platform detection [40, 58]”
From Six Million (Suspected) Fake Stars on GitHub: A Growing Spiral of Popularity Contests, Spams, and Malware (2026) · cites this paper as [40]
In the citing paper
“Fake followers can be bought for as low as 12$ per 1,000 followers in the surface Web.”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Time went by and new studies acknowledged the rise of a third wave of bots that spread through online social networks from 2016 onwards [5, 13], as shown in panel C of Figure 4.”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“As such, botnets leave behind more traces of their automation and coordination than those left behind by sophisticated single bots [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Moreover, hundreds of tech-savvy social media users that participated in a crowdsourcing experiment were able to tell apart newer bots from legitimate users only 24% of the times. The same users were instead able of spotting older bots 91% of the times [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“The generality and ease of deployment of this detector are however counterbalanced by a reduced bot detection accuracy [5, 17].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“Results showed that only 5% of newer bots are removed from social platforms, whereas older ones are removed 60% of the times [5].”
From A decade of social bot detection (2020) · cites this paper as [5]
In the citing paper
“They also report that some bot classes, like social spambots, are hard to distinguish from human accounts.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“Cresci et al. [11] highlight that it is hard for human annotators to assign one label to one account to describe its bot class.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“The performance decay of current detection systems in the wild was reported by Cresci et al. [11], who showed that Twitter, human annotators, and state-of-the-art bot detection systems failed at discriminating between some new social spambots and genuine accounts.”
From Detection of Novel Social Bots by Ensembles of Specialized Classifiers (2020) · cites this paper as [11]
In the citing paper
“By promoting online activity, bots can interact with humans and influence their standing against specific topics such as political issues [10, 7, 12, 13].”
From Influence of augmented humans in online interactions during voting events (2019) · cites this paper as [7]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Each contradiction between two passages listed here was confirmed by an independent check.
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| social spambots #2 | 3,457 | 3,322 (96.1%) | 1 (0.1%) | 134 (3.8%) |”
Reported 0.1 · recomputed 0.028927
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| social spambots #2 | 3,457 | 3,322 (96.1%) | 1 (0.1%) | 134 (3.8%) |”
Reported 3.8 · recomputed 3.876193
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
§ Keywords · p. 2
“| social spambots #1 | retweeters of an Italian political candidate | 991 | 1,610,176 | 2012 | 3.1, 3.2 |”
§ 2. DATASETS · p. 3
“| social spambots #1 | 994 | 946 (95.2%) | 2 (0.2%) | 46 (4.6%) |”
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| traditional spambots #1 | - 7.6% *** | - 0.8% | +8.7% *** |”
Reported 8.7 · recomputed 8.5
Rated minor: it would not touch a result — a value no result rests on, a p-value that stays on the same side of the significance level, or a conflict that is clerical, resolved elsewhere in the paper, or outside its own findings.
In the paper
“| genuine accounts | 3,474 | 3,353 (96.5%) | 115 (3.3%) | 6 (0.1%) |”
Reported 0.1 · recomputed 0.172712
Citations
Cites
Finding community structure in very large networks
Clauset, A., Newman, M. E. J., & Moore, C. (2004) · Physical Review E · doi:10.1103/physreve.70.066111 · Cited by 7,390
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973
Gwet, K. L. (2014) · Medical Entomology and Zoology · Cited by 1,095
Density-Based Clustering over an Evolving Data Stream with Noise
Cao, F., Estert, M., Qian, W., & Zhou, A. (2006) · SIAM International Conference on Data Mining · doi:10.1137/1.9781611972764.29 · Cited by 1,033
BotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909
Detecting spammers on social networks
Stringhini, G., Kruegel, C., & Vigna, G. (2010) · Proceedings of the 26th Annual Computer Security Applications Conference · doi:10.1145/1920261.1920263 · Cited by 791
Graph Clustering Via a Discrete Uncoupling Process
van Dongen, S. (2008) · SIAM Journal on Matrix Analysis and Applications · doi:10.1137/040608635 · Cited by 745
Magno, G., Rodrigues, T., & Almeida, V. A. F. (2010) · Cited by 649
Uncovering social spammers: social honeypots + machine learning
Lee, K., Caverlee, J., & Webb, S. (2010) · Proceedings of the 33rd international ACM SIGIR conference on Research and development in information retrieval · doi:10.1145/1835449.1835522 · Cited by 619
Fame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 475 · cited 1×
“Fake followers are another kind of malicious accounts that recently gained interest both from platform administrators and from the scientific world [13].”
BackgroundSelf-citation
Research works printed but not linked: 1
- M. Jiang, P. Cui, A. Beutel, C. Faloutsos, and S. Yang. Catching synchronized behaviors in large networks: A graph mining approach. ACM Trans. Knowl. Discov. Data, 10(4):35:1--35:27, 2016.
Cited by
Sorted by citations
Tucker, J. A., Guess, A., Barbera, P., Vaccari, C., Siegel, A., Sanovich, S., Stukal, D., & Nyhan, B. (2018) · doi:10.2139/ssrn.3144139 · Cited by 1,222
Weaponized health communication: Twitter bots and Russian trolls amplify the vaccine debate
Broniatowski, D. A., Jamison, A. M., Qi, S., AlKulaib, L., Chen, T., Benton, A., Quinn, S. C., & Dredze, M. (2018) · American Journal of Public Health · doi:10.2105/ajph.2018.304567 · Cited by 1,149
Arming the public with artificial intelligence to counter social bots
Yang, K., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.115 · Cited by 419
“While supervised methods have proven to be effective in many cases, they do not perform well at detecting coordinated social bots that post human-generated content (Cresci et al., 2017; Grimme et al., 2018; Chen and Subramanian, 2018).”
BackgroundSocial Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373
Measuring objective and subjective well-being: dimensions and data sources
Voukelatou, V., Gabrielli, L., Miliou, I., Cresci, S., Sharma, R. R., Tesconi, M., & Pappalardo, L. (2020) · International Journal of Data Science and Analytics · doi:10.1007/s41060-020-00224-2 · Cited by 459
Self-citation
Metadata
- Authors
- Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M.
- Affiliations
- University of Pisa (IT)
- Nokia Bell Labs & University of Padua, Paris, France
- National Research Council (IT)
- Technical University of Denmark (DK)
- Venue
- Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion
- Publisher
- ACM Press
- Published
- 2017
- Pages
- 963-972
- OpenAlex
- W4289253852
- Cited by
- 381
- References
- 50
- Citations by year
- 2026: 31 · 2025: 42 · 2024: 48 · 2023: 48 · 2022: 48 · 2021: 46 · 2020: 62 · 2019: 37 · 2018: 15 · 2017: 4
- FWCI
- 24.78
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- International World Wide Web Conference (WWW) (publishedVersion, cc-by)
- arXiv (Cornell University) (submittedVersion)
- CINECA IRIS Institutial research information system (University of Pisa) (submittedVersion)
- IRIS Research product catalog (Sapienza University of Rome) (submittedVersion)
- Funders
- EU (654024, 675320)
- Regione Toscana (SmartNews: Social sensing for Breaking News)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- Twitter, spam social media
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
A language model drew the claims from the paper on 2026-10-09, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-09 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
