The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
TwiBot-20: A Comprehensive Twitter Bot Detection Benchmark
Open accessAbstract
Twitter has become a vital social media platform while an ample amount of malicious Twitter bots exist and induce undesirable social effects. Successful Twitter bot detection proposals are generally supervised, which rely heavily on large-scale datasets. However, existing benchmarks generally suffer from low levels of user diversity, limited user information and data scarcity. Therefore, these datasets are not sufficient to train and stably benchmark bot detection measures. To alleviate these problems, we present TwiBot-20, a massive Twitter bot detection benchmark, which contains 229,573 users, 33,488,192 tweets, 8,723,736 user property items and 455,958 follow relationships. TwiBot-20 covers diversified bots and genuine users to better represent the real-world Twittersphere. TwiBot-20 also includes three modals of user information to support both binary classification of single users and community-aware approaches. To the best of our knowledge, TwiBot-20 is the largest Twitter bot detection benchmark to date. We reproduce competitive bot detection methods and conduct a thorough evaluation on TwiBot-20 and two other public datasets. Experiment results demonstrate that existing bot detection measures fail to match their previously claimed performance on TwiBot-20, which suggests that Twitter bot detection remains a challenging task and requires further research efforts.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Comparison of the predicted and observed secondary structure of T4 phage lysozyme
Matthews, B. W. (1975) · Biochimica et Biophysica Acta (BBA) - Protein Structure · doi:10.1016/0005-2795(75)90109-9 · Cited by 5,399 · cited 1×
“We adopt Accuracy, F1-score and MCC [18] as evaluation metrics.”
MethodOnline Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 981 · cited 3×
“Property Semantic Neighbor varol-icwsm [27] pronbots [31] ✓ celebrity [31] ✓ gilani-17 [13] ✓ cresci-rtbust [19] ✓ cresci-stock [7] ✓ midterm-18 [32] ✓ botwiki [32] ✓ verified [32] ✓ PAN-19 3 ✓ caverlee [15] ✓ ✓ cresci-17 [6] ✓ ✓ TwiBot-20 ✓ ✓ ✓ are also encouraged to download and use the full…”
BackgroundBotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909 · cited 2×
“• Botometer [9]: Botometer is a publicly available demo that leverages more than one thousand features to identify Twitter bots on demand.”
BackgroundDetecting Automation of Twitter Accounts: Are You a Human, Bot, or Cyborg?
Chu, Z., Gianvecchio, S., Wang, H., & Jajodia, S. (2012) · IEEE Transactions on Dependable and Secure Computing · doi:10.1109/tdsc.2012.75 · Cited by 634 · cited 3×
“[3], which strengthens the claim that TwiBot-20 annotation is generally trustworthy.”
BackgroundInfluentialDisinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524 · cited 1×
“Some Twitter bots exploit Twitter features to pursue malicious goals such as election interference [10, 11] and extreme propaganda [2].”
BackgroundFame for sale: Efficient detection of fake Twitter followers
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2015) · Decision Support Systems · doi:10.1016/j.dss.2015.09.003 · Cited by 475 · cited 1×
“Another early dataset is cresci-2015 [4] which provides a dataset of genuine accounts and fake followers.”
BackgroundArming the public with artificial intelligence to counter social bots
Yang, K., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.115 · Cited by 419 · cited 4×
“Property Semantic Neighbor varol-icwsm [27] pronbots [31] ✓ celebrity [31] ✓ gilani-17 [13] ✓ cresci-rtbust [19] ✓ cresci-stock [7] ✓ midterm-18 [32] ✓ botwiki [32] ✓ verified [32] ✓ PAN-19 3 ✓ caverlee [15] ✓ ✓ cresci-17 [6] ✓ ✓ TwiBot-20 ✓ ✓ ✓ are also encouraged to download and use the full…”
BackgroundMethodInfluentialThe Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3055135 · Cited by 381 · cited 6×
“Regarding dataset size, the number of users included in each dataset also range from 693 users in cresci-rtbust [19] to 50,538 in midterm-18 [32], and one of the most widely used dataset cresci-17 [6] contains 2,764 human users and 7,049 bots.”
BackgroundMethodInfluential(2020) · doi:10.1609/aaai.v34i01.5460 · Cited by 361
Empirical evaluation and new design for fighting evolving Twitter spammers
Yang, C., Harkreader, R., & Gu, G. (2013) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2013.2267732 · Cited by 346
Cited by
Few voices, strong echo: Measuring follower homogeneity of politicians’ Twitter accounts
Rusche, F. (2024) · New Media & Society · doi:10.1177/14614448221099860 · Cited by 6
Metadata
- Authors
- Feng, S., Wan, H., Wang, N., Li, J., & Luo, M.
- Venue
- Proceedings of the 30th ACM International Conference on Information & Knowledge Management
- Publisher
- ACM
- Published
- 2021-10-26
- Pages
- 4485-4494
- ISSN
- 2155-0751 (ISSN-L)
- OpenAlex
- W3175498457
- Cited by
- 197
- References
- 15
- FWCI
- 23.40
- Type
- conference paper
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: TwiBot-20: A Comprehensive Twitter Bot Detection Benchmark. https://app.cochat.ai/papers/10.1145/3459637.3482019
- The paper
- Feng, S., Wan, H., Wang, N., Li, J., & Luo, M. (2021). TwiBot-20: A Comprehensive Twitter Bot Detection Benchmark. Proceedings of the 30th ACM International Conference on Information & Knowledge Management. https://doi.org/10.1145/3459637.3482019
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.