Fame for sale: Efficient detection of fake Twitter followers

Probably not a substantial claim: may not pass S3 (load-bearing).

The metric used to evaluate the performance of different classifiers changes the optimal distribution of the classes for the training set.

“The conclusions of their investigation have shown that the metric used to evaluate the performance of the different classifiers changes the optimal distribution of the classes for the training set.”

• cites [34] [34] G. M. Weiss, F. Provost, Learning when training data are costly: the effect of class distribution on tree induction, Journal of Artificial Intelligence Research 19 (2003) 315-354. - sourceSupported

Probably not a substantial claim: may not pass S3 (load-bearing).

The optimal distribution for training decision-tree classifiers often involves oversampling the minority class.

“Moreover, the optimal distribution has an oversampling of the minority class (there are also cases when the best classifier is obtained with an oversampling up to 90% of the minority class samples).”

• cites [34] [34] G. M. Weiss, F. Provost, Learning when training data are costly: the effect of class distribution on tree induction, Journal of Artificial Intelligence Research 19 (2003) 315-354. - sourcePartly supported

  Section 6.3 (page unavailable) reports that, for AUC, 11 of the 12 data sets whose optimal range excludes the natural distribution have more minority-class examples in that range. This supports the claim that increasing the minority-class share can be optimal. The cited 90% example is not established as the best classifier overall: the paper says it performs better at high true-positive rates in one example, while a balanced distribution has the highest AUC for that data set. (LLM, conf=0.85)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The authors of [8] exploit several characteristics gathered by crawling an account's details, both from its profile and timeline, and use them in a Random Forest algorithm that outputs whether the account is a spambot or not.

“The authors exploit several characteristics that can be gathered crawling an account's details, both from its profile and timeline. For each investigated account, such characteristics are exploited in a Random Forest algorithm [23, 43], that outputs if the account is a spambot or not.”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source
• cites [23] [23] M. Hall, I. Witten, E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 3rd Edition, Morgan Kaufmann, 2011. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [43] [43] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, I. H. Witten, The WEKA data mining software: an update, ACM SIGKDD explorations newsletter 11 (1) (2009) 10-18. - sourceCould not verify

  The WEKA Workbench section describes software for applying classification algorithms, but it does not describe crawling account profiles or timelines to detect spambots. Searches for Random Forest and spambot account features found no passage in our copy establishing the specific method attributed to this source. (LLM, conf=0.76)

  Experimental — accuracy not yet measured

Probably not a substantial claim: may not pass S3 (load-bearing).

Sensitivity analysis aims at assessing the relative importance of the different features used to build a classification model.

“Sensitivity analysis, instead, aims at assessing the relative importance of the different features used to build a classification model [48].”

• cites [48] [48] A. Saltelli, Making best use of model evaluations to compute sensitivity indices, Computer Physics Communications 145 (2) (2002) 280-297. sourcePartly supported

  The Introduction (page unavailable) says global sensitivity analysis quantifies the relative importance of model input variables in determining an output. That supports the general purpose described, but the paper does not specifically address features used to build a classification model. (LLM, conf=0.86)

  Experimental — accuracy not yet measured

The quote only partly states this claim.

Algorithms based on classification rules proposed by Media do not succeed in detecting fake followers in the baseline dataset.

“Our results show that most of the rules proposed by Media provide unsatisfactory performance in revealing fake followers, while features proposed in the past by Academia for spam detection provide good results.”

• Also stated in: § Contributions · p. 3 · plain

  “algorithms based on classification rules do not succeed in detecting the fake followers in our baseline dataset.”

May combine two claims.

The final Class A classifier is general enough to thwart overfitting, lightweight thanks to the usage of the less costly features, and still able to correctly classify more than 95% of the accounts of the original training set.

“The final result is a novel Class A classifier, general enough to thwart overfitting, lightweight thanks to the usage of the less costly features, and still able to correctly classify more than 95% of the accounts of the original training set.”

• Also stated in: § Contributions · p. 3 · plain

  “lightweight classifiers that make use of the less costly features, while still being able to correctly classify more than 95% of the accounts of our training dataset.”

May combine two claims.

Independently from the typology of the account, the rules are almost always satisfied, leading to a severe flaw when dealing with fake followers detection.

“Summarizing, independently from the typology of the account, the rules are almost always satisfied, leading to a severe flaw when dealing with fake followers detection.”

May combine two claims.

The large majority of the accounts are recognized as humans, resulting in high precision, accuracy around 0.5 and very low recall.

“Indeed, the large majority of the accounts are recognized as humans, resulting in a high precision, accuracy around 0.5 and very low recall.”

The quote only partly states this claim.

Trained J48 classifiers obtained their best results on a balanced distribution of humans and fake followers when varying the class distribution from 5%-95% to 95%-5%.

“The trained classifiers obtained their best results on a balanced distribution of humans and fake followers.”

May combine two claims.

On the BAS dataset, only three rules achieved an MCC value higher than 0.6: the threshold of at least 30 followers, the threshold of at least 50 tweets, and the use of a userID in at least one tweet.

“As shown, only three rules obtained a value higher than 0.6, namely: (1) the threshold of at least 30 followers, (2) the threshold of at least 50 tweets, and (3) the use of a userID in at least one tweet.”

May combine two claims.

In the BAS dataset, fake follower accounts almost never tweet from API (instead, they use Twitter.com to tweet), whereas human accounts have posted at least once from outside the website.

“in our dataset, fake followers accounts almost never tweet from API (instead, they use Twitter.com to tweet), whereas human accounts have posted at least once from outside the website.”

May combine two claims.

The Camisani-Calzolari algorithm obtains very good results in detecting real human accounts but achieves poor fake follower account detection.

“Although obtaining very good results in detecting the real human accounts, the algorithm achieves a poor fake follower account detection.”

• cites [12] [12] Camisani-Calzolari M. (digitalevaluations), Analysis of Twitter followers of the US Presidential Election candidates: Barack Obama and Mitt Romney (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

All rules by Van Den Beld have accuracy and precision close to 0.5 or a very low recall.

“As an example, all rules by Van Den Beld have accuracy and precision close to 0.5 or a very low recall.”

• cites [11] [11] Stateofsearch.com, How to recognize Twitterbots: 7 signals to look out for., http://goo.gl/YZbVf, Last checked 23/07/15 (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

If useful rules and features are those with an information gain value higher than 0.5, then even if many rules exhibit a very low I_gain, their feature version becomes much more interesting; for example, rules 18, 20, 21 and 22 have an evident increase in their information gain when used as features.

“If we consider as useful the rules and features that have an information gain value higher than 0.5, we observe that, even if many rules exhibit a very low I gain , their 'feature' version becomes much more interesting: for example, rules 18, 20, 21 and 22 have an evident increase in their information gain when used as features.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The best performing rules also show the highest Pcc values, meaning their satisfaction is more strongly related to the belonging class.

“Coincidentally, we have that the best performing rules also show the highest Pcc values, namely their satisfaction is more strongly related to the belonging class.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Socialbakers customer service answered that how the respective criteria are measured is rather internal information.

“We contacted their customer service, but we were answered that 'how the respective criteria are measured is rather an internal information'.”

The quote only partly states this claim.

The three rules with MCC higher than 0.6 also exhibit a combination of high accuracy, precision, and recall.

“As expected by the definition of MCC , such rules also exhibit a combination of high accuracy, precision, and recall.”

May combine two claims.

Rule 6, '0 tweets', as a rule has an information gain value of 0.02, but when considered as a feature (number of tweets) it obtains 0.621.

“The exception is rule 6, '0 tweets': as a rule, it has an information gain value of 0.02, but when considered as a feature (i.e., number of tweets) it obtains 0.621.”

May combine two claims.

'tweet from API' has an MCC of -0.779, meaning it is strictly related to the class of the account, but by an inverse factor.

“Also, we observe that 'tweet from API' has an MCC of -0.779, meaning that it is strictly related to the class of the account, but by an inverse factor:”

May combine two claims.

The Camisani-Calzolari algorithm applied to BAS has a F-Measure close to 0.1 and a low MCC value.

“This poor performance is also expressed by a F-Measure close to 0.1 and by the low MCC value.”

• cites [12] [12] Camisani-Calzolari M. (digitalevaluations), Analysis of Twitter followers of the US Presidential Election candidates: Barack Obama and Mitt Romney (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

In the BAS dataset, the features number of tweets, ratio between friends and followers^2, bidirectional links ratio, and API ratio are promising (have high information gain and/or Pearson correlation with the class).

“Observing the results in Table 11, we can identify several promising features: 'number of tweets' (already noticed in Section 4), 'ratio between friends and followersˆ2', 'bidirectional links ratio' and 'API ratio'.”

• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

Probably not a substantial claim: may not pass S3 (load-bearing).

The test set comprises a broad range of accounts created during the nine years since Twitter's advent.

“This test set therefore comprises a broad range of accounts created during the nine years since Twitter's advent.”

May combine two claims.

In the BAS dataset, fake followers are characterized by low URL ratio values, while human accounts have higher URL ratio values.

“Observing the tree structure of our J48 classifier, instead, low values for this feature characterize fake followers, compared with higher values that indicate human accounts in our baseline dataset.”

• AssumptionRests on an assumption: The labels in the BAS dataset (human vs fake follower) are accurate.
• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

The quote only partly states this claim.

The reduced error pruning with 50 folds produces a classifier whose MCC drops from 0.966 to 0.866 when the bi-link ratio is omitted.

“The reduced error pruning with 50 folds produces a classifier that has MCC dropping from 0.966 to 0.866.”

May combine two claims.

The tree that does not use the bi-link ratio, pruned with subtree raising confidence of 0.25, requires 101 nodes, whereas the tree that uses it requires only 33 nodes.

“For example, the tree that does not use that feature, pruned with subtree raising confidence of 0.25, requires 101 nodes, whereas the tree that uses it requires only 33 nodes.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

All the classifiers of the previous sections are classifiers of Class B, with the exception of the classifier with the feature set of Yang et al. [2], which belongs to Class C.

“Then, all the classifiers of the previous sections are classifiers of Class B , with the exception of the classifier with the feature set of Yang et al. [2], that belongs to Class C .”

• cites Yang et al. [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

May combine two claims.

The decision tree obtained with the feature set of Stringhini et al. [8] has 22 leaves, 43 nodes, and a height of 7, with the friends/(followers^2) ratio at the root.

“The decision tree obtained with the feature set of Stringhini et al. [8] has 22 leaves, 43 nodes, and a height of 7, whereas the best feature is the friends/(followersˆ2) ratio that places at the root.”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source

• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

May combine two claims.

More than 72% of the fake followers in the training dataset have a URL ratio lower than 0.05, compared to 14% of human accounts.

“More than 72% of the fake followers in our training dataset have a URL ratio lower than 0.05, oppositely to 14% of human accounts.”

• AssumptionRests on an assumption: The labels in the BAS dataset (human vs fake follower) are accurate.
• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

May combine two claims.

In this experiment, SVM scores only slightly worse than RF, D, and J48, and better than AB.

“Indeed, in this experiment SVM scores only slightly worse than RF, D and J48, and better than AB.”

• AssumptionRests on an assumption: The labels in the BAS dataset (human vs fake follower) are accurate.
• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

Probably not a substantial claim: may not pass S3 (load-bearing).

All the rules and features considered in this study fall into one of the three categories (profile, timeline, relationship), as reported in Table 14.

“All the rules and features considered in this study fall into one of the three aforementioned categories, as reported in Table 14.”

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The decision tree with the feature set of Yang et al. [2] has 17 leaves, 33 nodes, and a height of 8, with the bi-directional link ratio as the root.

“The decision tree with the feature set of Yang et al. [2] has 17 leaves, 33 nodes and a height of 8, with the bi-directional link ratio as the root.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

May combine two claims.

The pruned version of Stringhini et al. [8] reduces its recall by 0.017, while the pruned version of Yang et al. [2] drops recall by only 0.004.

“Furthermore, we clearly observe that the pruned version of Stringhini et al. [8] reduces its recall of 0.017, while the one of Yang et al. [2] only drops of 0.004”

• cites Stringhini et al. [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source
• cites Yang et al. [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

May combine two claims.

The proposed Class A classifiers achieve overall good performances while only exploiting cost-efficient features.

“For this reason, we trained and evaluated the proposed Class A classifiers, which achieve overall good performances, while only exploiting cost-efficient features.”

May combine two claims.

Even reducing the features, it is possible to have a detection rate higher than 0.95, as shown for Stringhini et al. [8] and Yang et al. [2] in the last lines of Table 13.

“Overall, the results of this experiment show that, even reducing the features, it is possible to have a detection rate higher than 0.95 (as in the last lines of Table 13, for [8] and [2], respectively).”

• cites Stringhini et al. [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source
• cites Yang et al. [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Both classifiers employ a set of features that includes features proposed by the grey literature for fake detection and by Academia for spam and bot detection.

“A further interesting observation is that both the classifiers employ a set of features that includes both features proposed by the grey literature for fake detection, which we introduced in Section 4, and by Academia for spam and bot detection, as detailed in Section 5.”

May combine two claims.

Leaving out the bi-link ratio from the Yang et al. [2] feature set causes an evident decrease in accuracy, precision, recall, F-Measure, AUC, and an even noticeable decrease in MCC for the less pruned trees (subtree raising 0.25 and reduced error 3 folds).

“The results in Table 13 show an evident decrease in accuracy , precision and recall for the less pruned trees (subtree raising 0.25 and reduced error 3 folds), as well as for both F-Measure and AUC , and an even noticeable decrease of the MCC measure.”

• cites Yang et al. [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

May combine two claims.

The Class A classifier with BN outperforms the Class C competitor, obtaining an increase in all metrics except AUC.

“It is worth noting that the Class A classifier with BN outperforms the Class C competitor, noticeably obtaining an increase in all the metrics, but the AUC .”

May combine two claims.

The simpler pruned trees generally use subsets of the feature set while still maintaining very good performance on the BAS dataset.

“Such simpler trees generally use subsets of the feature set, still maintaining very good performance on our BAS dataset.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The two decision trees obtained with the J48 algorithm are comparable, since both are composed of 31 nodes and 16 leaves and have a height of 7.

“Concerning the complexity of the two decision trees obtained with the J48 algorithm, they are comparable, since both of them are composed by 31 nodes and 16 leaves and they have a height of 7.”

May combine two claims.

Timeline-based (Class B) features are less time-demanding while still effective.

“Timeline-based ( Class B ) features have been shown to be less time-demanding, while still effective”

May combine two claims.

In the two validation experiments, Adaptive Boosting (AB) raises its accuracy by 0.1 and its precision by around 0.08, outperforming the Bayesian Network-based classifier.

“In the two validation experiments, AB and BN noticeably switch their performances: the Adaptive Boosting algorithm raises its accuracy of 0.1 and its precision of around 0.08, outperforming the Bayesian Network-based one.”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

None of the 19 Class A features is clearly dominant; all give a significant contribution to the detection of fake Twitter followers.

“Notably, none of the features is clearly dominant: all the 19 Class A features give a significant contribution to the detection of fake Twitter followers.”

May combine two claims.

The SVM classifier is the second best classifier when validated against random sampled accounts, and still performs very well against sampled Obama followers.

“The SVM classifier, instead, proves to be extremely effective, being the second best classifier when validated against random sampled accounts, and still performing very well against sampled Obama followers.”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

The Class A classifiers achieve detection rates comparable with the best of breed classifiers, whereas the latter necessitate overhead-demanding features.

“With this final outcome, we were able to achieve detection rates comparable with the best of breed classifiers, whereas these latter necessitate overhead-demanding features.”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

In Tables 12 and 16 almost all the considered classifiers achieved comparable performances, while in Table 17 there are major differences.

“Indeed, while in Tables 12 and 16 almost all the considered classifiers achieved comparable performances, here we can see some major differences.”

May combine two claims.

AB and BN obtain lower accuracy and precision than the top classifiers but still a noticeably high recall on the random sampled accounts test set.

“AB and BN obtain lower results for both accuracy and precision, but still a noticeably high recall”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

Relationships-based (Class C) features are more demanding in terms of data to be downloaded and require significant analysis time, making them unsuitable for analyses on massive amounts of followers.

“However, relationships-based ( Class C ) features are more demanding in terms of data to be downloaded and, consequently, they require a significant analysis time, making them unsuitable for analyses on massive amounts of followers.”

May combine two claims.

In the Class A validation on a test set of random sampled accounts, the five Class A classifiers obtain an accuracy above 0.9, a precision close to or higher than 0.9, and a recall above 0.94.

“Observing the validation against the random sampled accounts, we can see that the five Class A classifiers obtain an accuracy above 0.9, a precision close to (when not much higher than) 0.9 and a recall above 0.94”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

The RF, D, and J48 classifiers obtain comparable results for the two test sets, with an accuracy greater than 0.9 and a precision close to 0.87, with a reduction of only 0.025 on average.

“The RF, D, and J48 classifiers obtain results that are comparable for the two test sets, with an accuracy greater than 0.9 and a precision close to 0.87, with a reduction of only 0.025 on average.”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

The AUC metric is always very high and does not consistently reflect the real performances obtained by the five classifiers, as the F-measure and the MCC actually do.

“We can finally observe that the AUC metric is always very high and that does not consistently reflect the real performances obtained by the five classifiers, as the F-measure and the MCC actually do.”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

May combine two claims.

The RF classifier loses 0.4 of accuracy and a considerable 0.1 in precision in the validation against sampled Obama followers.

“Also the RF classifier looses 0.4 of accuracy and a considerable 0.1 in precision”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The accounts labeled as human in the baseline dataset are genuine human accounts.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.

Probably not a substantial claim: may not pass S3 (load-bearing).

The results in [12] lack a validation phase.

“The results in [12], however, lack a validation phase.”

• cites [12] [12] Camisani-Calzolari M. (digitalevaluations), Analysis of Twitter followers of the US Presidential Election candidates: Barack Obama and Mitt Romney (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

If the practice of buying fake followers is adopted by malicious accounts such as spammers, it can act as a way to post more authoritative messages and launch more effective advertising campaigns.

“Similarly, if the practice of buying fake followers is adopted by malicious accounts, as spammers, it can act as a way to post more authoritative messages and launch more effective advertising campaigns [6].”

• cites [6] [6] C. Castillo, M. Mendoza, B. Poblete, Information credibility on Twitter, in: Proceedings of the 20th international conference on World Wide Web, ACM, 2011, pp. 675-684. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based). May combine two claims.

Despite fake followers constituting a widespread phenomenon with both economical and social impacts, the topic has not been deeply investigated yet in the literature.

“To the best of our knowledge, however, despite fake followers constitute a widespread phenomenon with both economical and social impacts, in the literature the topic has not been deeply investigated yet.”

Probably not a substantial claim: may not pass S4 (based). May combine two claims.

The rules proposed by bloggers are usually paired neither with analytic algorithms to aggregate them nor with validation mechanisms.

“However, such rules are usually paired neither with analytic algorithms to aggregate them, nor with validation mechanisms.”

May combine two claims.

The criteria used by specialized social media analysis companies are not publicly disclosed and just partially deductible from their web sites.

“However, the criteria used for the analysis are not publicly disclosed and just partially deductible from information available on their web sites.”

• cites [13] [13] Statuspeople.com, Status People Fakers, http://goo.gl/0Jpky, Last checked 23/07/15. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [14] [14] SocialBakers, Fake follower check, http://goo.gl/chWn0, Last checked 23/07/15. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

• cites [15] [15] Twitteraudit, How many of your followers are real?, www.twitteraudit.com, Last checked 23/07/15. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Socialbakers website provides eight criteria to evaluate the fakeness degree of followers of an account, but it omits details on how to combine such criteria to classify accounts.

“While the company website provides eight criteria to evaluate the fakeness degree of the followers of a certain account, it omits details on how to combine such criteria to classify the accounts.”

• cites [14] [14] SocialBakers, Fake follower check, http://goo.gl/chWn0, Last checked 23/07/15. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

May combine two claims.

Pcc evaluates the linear dependency between two variables that assume very similar values (the output of the rule and the class), while Pcc* considers variables with more heterogeneous values.

“Actually, this happens because Pcc evaluates the linear dependency between two variables that assume very similar values, namely the output of the rule and the class, while the Pcc* considers variables with more heterogeneous values.”

May combine two claims.

The rule is based on a right assumption (for example, the use of hashtags), but the rule definition is too simple to be effective; the algorithm proposed by [12] is too naive for the complex task of fake accounts detection in Twitter.

“Thus, we can derive that the rule is based on a right assumption (for example, the use of hashtags), but the rule definition is too simple to be effective: the algorithm proposed by [12] is simply too naive for the complex task of fake accounts detection in Twitter.”

• cites [12] [12] Camisani-Calzolari M. (digitalevaluations), Analysis of Twitter followers of the US Presidential Election candidates: Barack Obama and Mitt Romney (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Our fake followers dataset is illustrative, not exhaustive, of all possible existing sets of fake followers.

“We acknowledge that our fake followers dataset is just illustrative, and not exhaustive, of all the possible existing sets of fake followers.”

Probably not a substantial claim: may not pass S3 (load-bearing).

There are situations where some predictive models perform better than others, even having a lower accuracy [38].

“Moreover, there are situations where some predictive models perform better than others, even having a lower accuracy [38].”

• cites [38] [38] D. M. W. Powers, Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation, International Journal of Machine Learning Technologies 2 (1) (2011) 37-63. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

The Camisani-Calzolari algorithm [12] has enough details to be reproducible: it assigns human/active and bot/inactive scores and classifies an account considering the gap between the sum of the two scores.

“In [12] it is detailed an algorithm to evaluate an account based on some of its public features. The cited algorithm has enough details to be reproducible: it assigns human/active and bot/inactive scores and classifies an account considering the gap between the sum of the two scores.”

• cites [12] [12] Camisani-Calzolari M. (digitalevaluations), Analysis of Twitter followers of the US Presidential Election candidates: Barack Obama and Mitt Romney (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim. May combine two claims.

A single rule or feature alone may not perform well in classifying fake and human accounts, but it may improve detection if used in combination with other ones.

“This is because a single rule - or feature - alone may not perform well in classifying fake and human accounts, but it may improve the detection if used in combination with other ones.”

Classified as probably not a substantial claim.

The 469 accounts out of 574 followers that successfully completed the CAPTCHA are considered certified humans.

“We consider as 'certified humans' all the 469 accounts out of the 574 followers that successfully completed the CAPTCHA.”

May combine two claims.

MAUs can be assimilated neither to a random sample of Twitter accounts nor to the followers of a given account.

“However, MAUs can be assimilated neither to a random sample of Twitter accounts nor to the followers of a given account.”

Probably not a substantial claim: may not pass S3 (load-bearing).

MCC is considered the unbiased version of the F-Measure, since it uses all four elements of the confusion matrix [38].

“Furthermore, MCC is considered the unbiased version of the F-Measure , since it uses all the four elements of the confusion matrix [38].”

• cites [38] [38] D. M. W. Powers, Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation, International Journal of Machine Learning Technologies 2 (1) (2011) 37-63. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

The E13 set was sampled with confidence level 95% and confidence interval 2.5.

“This last set has been sampled (with confidence level 95% and confidence interval 2.5), leading to a final set of 1488 accounts, that have been subject to a manual verification to determine the nature of their profiles and tweets.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The FakeFollowerCheck tool analyzes the followers of an account and considers them likely fake when the criteria listed in Table 4 are satisfied.

“The FakeFollowerCheck tool analyzes the followers of an account and considers them likely fake when the criteria listed in Table 4 are satisfied.”

• cites [14] [14] SocialBakers, Fake follower check, http://goo.gl/chWn0, Last checked 23/07/15. -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

Works [8, 2] propose a list of features to be extracted from manually classified datasets of accounts.

“Both of them propose a list of features to be extracted from manually classified datasets of accounts.”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source
• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

May combine two claims.

The only way to check a user's follow/unfollow behavior (rule 7) is to continuously monitor full friends and followers complete lists; the same applies to measuring the delay when a user follows and replies to other users (rule 6).

“This means that the only way to check a user's follow/unfollow behavior (rule 7) is to continuously monitor full friends and followers complete lists. The same applies with respect to the measurement of the delay experienced when a user follows (and replies to) other users (rule 6).”

• AssumptionRests on an assumption: Twitter only gives information about the current friends/followers list, with no details about past friends or followers, and does not disclose temporal data about when a user started following or got followed by another user.

Classified as probably not a substantial claim.

Only accounts labeled as humans by both sociologists were included in the E13 dataset.

“Only accounts labeled as humans by both the sociologists have been included in the E13 dataset.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Camisani-Calzolari [12] carried out a series of tests over samples of Twitter followers of Romney and Obama, for the last US presidential elections, as well as for popular Italian politicians.

“Camisani-Calzolari [12] carried out a series of tests over samples of Twitter followers of Romney and Obama, for the last US presidential elections, as well as for popular Italian politicians.”

• cites [12] [12] Camisani-Calzolari M. (digitalevaluations), Analysis of Twitter followers of the US Presidential Election candidates: Barack Obama and Mitt Romney (August 2012). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Probably not a substantial claim: may not pass S3 (load-bearing).

In 2013, Twitter staff conjectured that the number of false or spam accounts should represent less than 5% of monthly active users.

“In 2013, the Twitter staff conjectured that the number of 'false or spam accounts should represent less than 5% of our MAUs' (where MAUs refer to monthly active users) [35].”

• cites [35] [35] Twitter Inc., Twitter's IPO filing, http://www.sec.gov/Archives/edgar/data/1418091/000119312513390321/d564001ds1.htm, Last checked 23/07/15 (Oct 2013). -Not matched

  Couldn’t be matched to a paper in Crossref, so it can’t be checked.

Classified as probably not a substantial claim.

The feature sets from [8, 2] are used to train and test machine learning classifiers in order to distinguish between humans and spammers.

“Such feature sets are then used to train and test machine learning classifiers in order to distinguish between humans and spammers.”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source
• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

Adaptive Boost is considered one of the most effective machine learning algorithms for classification tasks.

“We have decided to include AB in our work, since it is considered one of the most effective machine learning algorithms for classification tasks [39].”

• cites [39] [39] J. Friedman, T. Hastie, R. Tibshirani, The elements of statistical learning, Vol. 1, Springer Series in Statistics, 2001. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Without the original training set, the authors were unable to reproduce the same classifier, but they picked the five features and used them to train a set of classifiers with the BAS dataset.

“Without the original training set, we were unable to reproduce the same classifier, but we picked the five features and used them to train a set of classifiers with our BAS dataset.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The study presented in [8] focuses on spambot detection.

“The study presented in [8] focuses on spambot detection.”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source

May combine two claims.

Following rate reflects the speed at which an account follows other accounts; spammers usually feature high values of this rate.

“9. following rate : this metric reflects the speed at which an accounts follows other accounts. Spammers usually feature high values of this rate.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors of [2] observed that Twitter spammers often modify their behavior in order to evade existing spam detection techniques.

“The authors of [2] observed that Twitter spammers often modify their behavior in order to evade existing spam detection techniques.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

In glass-box models, such as Decision Trees and Regression Models, the inner structure of the models can be understood by humans, also providing insights on how the classifiers identify fake accounts.

“In 'glassbox' models, such as Decision Trees and Regression Models, the inner structure of the models can be understood by humans, also providing insights on how the classifiers identify fake accounts [39].”

• cites [39] [39] J. Friedman, T. Hastie, R. Tibshirani, The elements of statistical learning, Vol. 1, Springer Series in Statistics, 2001. -Not checked yet

  Not checked in the free run.

May combine two claims.

Even though the proposed features were originally designed for spam detection, this paper is the first to consider them to spot fake followers on Twitter.

“Even if the proposed features have been originally designed for spam detection, here, for the first time, we consider them to spot another category of Twitter accounts, i.e., the fake followers.”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source
• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Spambots have sent less than 20 tweets.

“2. spambots have sent less than 20 tweets;”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The content of spambots' tweets exhibits the so-called message similarity.

“3. the content of spambots' tweets exhibits the so-called message similarity ;”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

The features proposed by Yang et al. [2] are robust against evasion techniques adopted by spammers.

“Instead, here we evaluate how those features, which proved to be quite robust against evasion techniques adopted by spammers, perform in detecting fake Twitter followers.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The classifiers' performances were estimated using 10-fold cross validation.

“Then, we have used a 10-fold cross validation [23] to estimate the performances of each obtained classifier.”

• cites [23] [23] M. Hall, I. Witten, E. Frank, Data Mining: Practical Machine Learning Tools and Techniques, 3rd Edition, Morgan Kaufmann, 2011. -Not checked yet

  Not checked in the free run.

Probably not a substantial claim: may not pass S3 (load-bearing).

Beyond the features directly available from the account profile lookup, the authors of [2] propose some graph-, automation-, and timing-based features.

“Beyond the features directly available from the account profile lookup, the authors propose some graph-, automation-, and timing-based features.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Spambots have a high tweets containing URLs ratio (URL ratio).

“4. spambots have a high tweets containing URLs ratio ( URL ratio );”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

Age of the account: the more an account is aged, the more it could be considered a good one.

“1. age of the account (this feature also appears in [28]): the more an account is aged, the more it could be considered a good one;”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source
• cites [28] [28] K. Lee, J. Caverlee, S. Webb, Uncovering social spammers: social honeypots + machine learning, in: SIGIR Conference on Research and Development in Information Retrieval, ACM, 2010, pp. 435-442. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

Spambots have a high friends/(followers^2) ratio value (i.e., lower ratio values mean legitimate users).

“total tweets 5. spambots have a high friends ( followersˆ2 ) ratio value (i.e., lower ratio values mean legitimate users).”

• cites [8] [8] G. Stringhini, C. Kruegel, G. Vigna, Detecting spammers on social networks, in: 26th Annual Computer Security Applications Conference, ACSAC '10, ACM, 2010, pp. 1-9. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

The authors of [2] suggested considering some new features, making evasion more difficult for spammers.

“Thus, they suggested to consider some new features, making evasion more difficult for spammers.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Classified as probably not a substantial claim.

The other features in [2] are less robust with regards to evasion techniques.

“However, as claimed by the same authors, they are less robust with regards to evasion techniques and thus we decided not to include them in our evaluation.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

Probably not a substantial claim: may not pass S3 (load-bearing).

To get the total time, one should not consider the sum of the time needed for each of the three cost classes, but just the most costly one.

“In other words, to get the total time, one should not consider the sum of the time needed for each of the three cost classes, but just the most costly one.”

• AssumptionRests on an assumption: Twitter limits the number of calls of the same API, but different APIs can be called in parallel.

Probably not a substantial claim: may not pass S3 (load-bearing).

The total time required to collect all data depends on the category that requires more time, i.e., the relationship one.

“The total time required to collect all data depends on the category that requires more time, i.e., the relationship one.”

• AssumptionRests on an assumption: Twitter limits the number of calls of the same API, but different APIs can be called in parallel.

Classified as probably not a substantial claim.

In [2], the average neighbor's tweets feature was found to be lower for spammers.

“A similar behavior has been observed for the average neighbor's tweets feature, that has been found to be lower for spammers in [2], but higher for our fake followers.”

• cites [2] [2] C. Yang, R. Harkreader, G. Gu, Empirical evaluation and new design for fighting evolving Twitter spammers, Information Forensics and Security, IEEE Transactions on 8 (8) (2013) 1280-1293. - sourceNot checked — no full text of the source

May combine two claims.

Fake followers appear to be more passive compared to spammers and do not make use of automated mechanisms for posting their tweets, as spammers usually do.

“These initial observations highlight a behavioral difference between a spam account and a fake follower. In particular, fake followers appear to be more passive compared to spammers and they do not make use of automated mechanisms for posting their tweets, as spammers usually do.”

• AssumptionRests on an assumption: The labels in the BAS dataset (human vs fake follower) are accurate.
• AssumptionRests on an assumption: The features from [2] and [8] are correctly implemented and computed.

Classified as probably not a substantial claim.

A classifier that uses a certain feature set belongs to the class of the more expensive feature in that set.

“a classifier that uses a certain feature set belongs to the class of the more expensive feature.”

Probably not a substantial claim: may not pass S3 (load-bearing).

By capturing the nature of the social ties between an account and its neighbors, the bidirectional link ratio is intrinsically harder to beat than features based on simpler characteristics such as other information in the account's profile.

“By capturing the nature of the social ties between an account and its neighbors, this feature is intrinsically harder to beat than those based on simpler characteristics, like, e.g., other information in the account's profile.”

Probably not a substantial claim: may not pass S4 (based).

The set of 1000 Twitter accounts randomly selected by numeric user IDs represents an unbiased sample of all currently available (not closed, banned or suspended) Twitter accounts.

“This represents an unbiased sample of all the currently available (i.e., not closed, banned or suspended) Twitter accounts.”

Probably not a substantial claim: may not pass S3 (load-bearing), S4 (based).

The use of the most expensive Class B or even Class C features can provide stronger evidence about the more suspicious followers.

“we can consider the use of the most expensive Class B or even Class C features to have stronger evidences about the more suspicious followers.”

May combine two claims.

The bidirectional link ratio is an important feature for fake follower detection: even if not essential, it is extremely effective for the detection process.

“From the results shown in Table 13, we conclude that the bidirectional link ratio is an important feature for fake follower detection: even if not essential, it is extremely effective for the detection process.”

Probably not a substantial claim: may not pass S3 (load-bearing).

The second experiment resembles the practical application scenario of the proposed classifier, since it investigates a sample of a single account's followers, in this case a major politician.

“This experiment resembles the practical application scenario of the proposed classifier, since it investigates a sample of a single account's followers, in this case a major politician.”

Probably not a substantial claim: may not pass S3 (load-bearing).

Information fusion is a technique aimed at leveraging the predictive power of several different models to achieve a combined prediction accuracy better than the predictions of the single models.

“Information fusion is a technique aimed at leveraging the predictive power of several different models in order to achieve a combined prediction accuracy which is better than the predictions of the single models [47].”

• cites [47] [47] C. W. Chase Jr, Composite forecasting: combining forecasts for improved accuracy, Journal of Business Forecasting Methods & Systems 19 (2) (2000) 2. -Not checked — no full text of the source

May combine two claims.

Automatically labeling the sampled accounts as humans would result in an error of at most 5%, still allowing an overall correct validation of the classifiers.

“Automatically labeling the sampled accounts as humans would result in an error of at most 5%, still allowing an overall correct validation of our classifiers.”

May combine two claims.

For AB and BN, only few fake followers are left behind, but a considerably higher number of human accounts are classified as fake followers on the random sampled accounts test set.

“meaning that only few fake followers are left behind, but a considerably higher number of human accounts are classified as fake followers”

• AssumptionRests on an assumption: Fake followers bought from online markets are representative of fake Twitter followers in general.
• AssumptionRests on an assumption: The test sets (random sampled accounts and sampled Obama followers) are representative of Twitter populations for validating fake follower detection.