BotOrNot: A System to Evaluate Social Bots
Open accessAbstract
While most online social media accounts are controlled by humans, these platforms also host automated agents called social bots or sybil accounts. Recent literature reported on cases of social bots imitating humans to manipulate discussions, alter the popularity of users, pollute content and spread misinformation, and even perform terrorist propaganda and recruitment actions. Here we present BotOrNot, a publicly-available service that leverages more than one thousand features to evaluate the extent to which a Twitter account exhibits similarity to the known characteristics of social bots. Since its release in May 2014, BotOrNot has served over one million requests via our website and APIs.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
BotOrNot? is a publicly-available service to evaluate the similarity of a Twitter account with the known characteristics of social spambots.
“BotOrNot? is a publicly-available service 7 to evaluate the similarity of a Twitter account with the known characteristics of social spambots [15].”
From The Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race (2017) · cites this paper as [15]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Ratkiewicz (2011) · doi:10.1145/1963192.1963301 · Cited by 541 · see citing works
Seven Months with the Devils: A Long-Term Study of Content Polluters on Twitter
Lee, K., Eoff, B. D., & Caverlee, J. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v5i1.14106 · Cited by 448 · see citing works
Design and analysis of a social botnet
Boshmaf, Y., Muslukhov, I., Beznosov, K., & Ripeanu, M. (2012) · Computer Networks · doi:10.1016/j.comnet.2012.06.006 · Cited by 230 · see citing works
The Rise of Social BotsPreprint
Ferrara, E., Varol, O., Davis, C. B., et al. (2014) · SSRN Electronic Journal · doi:10.48550/arxiv.1407.5225 · Cited by 2 · see citing works
Self-citation
Flagged first, then by citations
Research works printed but not linked: 1
- [5][5] V. Subrahmanian, A. Azaria, S. Durst, V. Kagan, A. Galstyan, K. Lerman, L. Zhu, E. Ferrara, A. Flammini, F. Menczer, et al. The darpa twitter bot challenge. arXiv preprint arXiv:1601.05140 , 2016. · arXiv
Cited by
Sorted by citations
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847 · see citing works
Ferrara, E., Varol, O., Davis, C., Menczer, F., & Flammini, A. (2016) · Communications of the ACM · doi:10.1145/2818717 · Cited by 1,973 · see citing works
Self-citationShifting attention to accuracy can reduce misinformation online
Pennycook, G., Epstein, Z., Mosleh, M., Arechar, A. A., Eckles, D., & Rand, D. G. (2021) · Nature · doi:10.1038/s41586-021-03344-2 · Cited by 1,155 · see citing works
Online Human-Bot Interactions: Detection, Estimation, and Characterization
Varol, O., Ferrara, E., Davis, C., Menczer, F., & Flammini, A. (2017) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v11i1.14871 · Cited by 928 · see citing works
“This system leverages more than one thousand features to evaluate the extent to which a Twitter account exhibits similarity to the known characteristics of social bots (Davis et al. 2016).”
BackgroundMethodSelf-citationTop concerns of tweeters during the COVID-19 pandemic: Infoveillance study
Abd-Alrazaq, A., Alhuwail, D., Househ, M., Hamdi, M., & Shah, Z. (2020) · Journal of Medical Internet Research · doi:10.2196/19016 · Cited by 774 · see citing works
Combating Fake News: A Survey on Identification and Mitigation Techniques
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3305260 · Cited by 514 · see citing works
Disinformation and social bot operations in the run up to the 2017 French presidential election
Ferrara, E. (2017) · First Monday · doi:10.5210/fm.v22i8.8005 · Cited by 524 · see citing works
Self-citationArming the public with artificial intelligence to counter social bots
Yang, K., Varol, O., Davis, C. A., Ferrara, E., Flammini, A., & Menczer, F. (2019) · Human Behavior and Emerging Technologies · doi:10.1002/hbe2.115 · Cited by 419 · see citing works
“Botometer is based on a supervised machine learning approach (Davis et al., 2016; Varol et al., 2017a).”
BackgroundMethodSelf-citationThe Paradigm-Shift of Social Spambots: Evidence, Theories, and Tools for the Arms Race
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Proceedings of the 26th International Conference on World Wide Web Companion - WWW '17 Companion · doi:10.1145/3041021.3055135 · Cited by 381 · see citing works
Social Media, Echo Chambers, and Political Polarization
Barberá, P. (2020) · Social Media and Democracy · doi:10.1017/9781108890960.004 · Cited by 373 · see citing works
False Information on Web and Social Media: A Survey
Kumar, S., & Shah, N. (2018) · arXiv · doi:10.48550/arxiv.1804.08559 · Cited by 289 · see citing works
Unpacking the Social Media Bot: A Typology to Guide Research and Policy
Gorwa, R., & Guilbeault, D. (2020) · Policy & Internet · doi:10.1002/poi3.184 · Cited by 270 · see citing works
DeBot: Twitter Bot Detection via Warped Correlation
Chavoshi, N., Hamooni, H., & Mueen, A. (2016) · 2016 IEEE 16th International Conference on Data Mining (ICDM) · doi:10.1109/icdm.2016.0096 · Cited by 260 · see citing works
ChatGPT in education: global reactions to AI innovations
Fütterer, T., Fischer, C., Alekseeva, A., Chen, X., Tate, T., Warschauer, M., & Gerjets, P. (2023) · Scientific Reports · doi:10.1038/s41598-023-42227-6 · Cited by 242 · see citing works
The False positive problem of automatic bot detection in social science research
Rauchfleisch, A., & Kaiser, J. (2020) · PLOS ONE · doi:10.1371/journal.pone.0241045 · Cited by 184 · see citing works
RTbust: Exploiting Temporal Patterns for Botnet Detection on Twitter
Mazza, M., Cresci, S., Avvenuti, M., Quattrociocchi, W., & Tesconi, M. (2019) · Proceedings of the 10th ACM Conference on Web Science · doi:10.1145/3292522.3326015 · Cited by 163 · see citing works
Twitter Bot Detection Using Bidirectional Long Short-Term Memory Neural Networks and Word Embeddings
Wei, F., & Nguyen, U. T. (2019) · 2019 First IEEE International Conference on Trust, Privacy and Security in Intelligent Systems and Applications (TPS-ISA) · doi:10.1109/tps-isa48467.2019.00021 · Cited by 168 · see citing works
Detection of Novel Social Bots by Ensembles of Specialized Classifiers
Sayyadiharikandeh, M., Varol, O., Yang, K.-C., Flammini, A., & Menczer, F. (2020) · Proceedings of the 29th ACM International Conference on Information & Knowledge Management · doi:10.1145/3340531.3412698 · Cited by 159 · see citing works
Self-citationWhat types of COVID-19 conspiracies are populated by Twitter bots?
Ferrara, E. (2020) · First Monday · doi:10.5210/fm.v25i6.10633 · Cited by 223 · see citing works
“(1) Botometer3, also used here, is a bot-detection tool developed at Indiana University [52]; (2) BotSlayer4 is an application to detect and track potential manipulation of information on Twitter; (3) the Bot Repository5 is a centralized database to share annotated datasets of Twitter bots.”
MethodSelf-citationDeepfakes: current and future trends
Gambín, Á. F., Yazidi, A., Vasilakos, A., Haugerud, H., & Djenouri, Y. (2024) · Artificial Intelligence Review · doi:10.1007/s10462-023-10679-x · Cited by 145 · see citing works
Charting the Landscape of Online Cryptocurrency Manipulation
Nizzoli, L., Tardelli, S., Avvenuti, M., Cresci, S., Tesconi, M., & Ferrara, E. (2020) · IEEE Access · doi:10.1109/access.2020.3003370 · Cited by 132 · see citing works
Self-citationWho Falls for Online Political Manipulation?
Badawy, A., Lerman, K., & Ferrara, E. (2019) · Companion Proceedings of The 2019 World Wide Web Conference · doi:10.1145/3308560.3316494 · Cited by 105 · see citing works
Self-citationUsing BERT to Extract Topic-Independent Sentiment Features for Social Media Bot Detection
Heidari, M., & Jones, J. H. (2020) · 2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) · doi:10.1109/uemcon51285.2020.9298158 · Cited by 102 · see citing works
Detecting Social Bots on Twitter: A Literature Review
Alothali, E., Zaki, N., Mohamed, E. A., & Alashwal, H. (2018) · 2018 International Conference on Innovations in Information Technology (IIT) · doi:10.1109/innovations.2018.8605995 · Cited by 102 · see citing works
Investigating Italian disinformation spreading on Twitter in the context of 2019 European elections
Pierri, F., Artoni, A., & Ceri, S. (2020) · PLOS ONE · doi:10.1371/journal.pone.0227821 · Cited by 93 · see citing works
Changing Perspectives: Is It Sufficient to Detect Social Bots?
Grimme, C., Assenmacher, D., & Adam, L. (2018) · Lecture Notes in Computer Science · doi:10.1007/978-3-319-91521-0_32 · Cited by 93 · see citing works
Sexist Slurs: Reinforcing Feminine Stereotypes Online
Felmlee, D., Inara Rodis, P., & Zhang, A. (2020) · Sex Roles · doi:10.1007/s11199-019-01095-z · Cited by 91 · see citing works
Detecting Malicious Social Bots Based on Clickstream Sequences
Shi, P., Zhang, Z., & Choo, K.-K. R. (2019) · IEEE Access · doi:10.1109/access.2019.2901864 · Cited by 72 · see citing works
LOBO: Evaluation of Generalization Deficiencies in Twitter Bot Classifiers
Echeverra, J., De Cristofaro, E., Kourtellis, N., Leontiadis, I., Stringhini, G., & Zhou, S. (2018) · Proceedings of the 34th Annual Computer Security Applications Conference · doi:10.1145/3274694.3274738 · Cited by 61 · see citing works
Journalists on Twitter: self-branding, audiences, and involvement of bots
Varol, O., & Uluturk, I. (2020) · Journal of Computational Social Science · doi:10.1007/s42001-019-00056-6 · Cited by 39 · see citing works
Self-citationA language framework for modeling social media account behavior
Nwala, A. C., Flammini, A., & Menczer, F. (2023) · EPJ Data Science · doi:10.1140/epjds/s13688-023-00410-9 · Cited by 21 · see citing works
Self-citationInvestigating coordinated account creation using burst detection and network analysis
Bellutta, D., & Carley, K. M. (2023) · Journal of Big Data · doi:10.1186/s40537-023-00695-7 · Cited by 15 · see citing works
A socialbots analysis-driven graph-based approach for identifying coordinated campaigns in twitter
Fazil, M., & Abulaish, M. (2020) · Journal of Intelligent & Fuzzy Systems · doi:10.3233/jifs-182895 · Cited by 14 · see citing works
- Authors
- Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F.
- Affiliations
- Indiana University Bloomington (US)
- University of Southern California (US)
- Venue
- Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion
- Published
- 2016-04-11
- Pages
- 273-274
- OpenAlex
- W2263846226
- Cited by
- 909
- References
- 4
- Citations by year
- 2026: 43 · 2025: 46 · 2024: 62 · 2023: 70 · 2022: 111 · 2021: 119 · 2020: 176 · 2019: 130 · 2018: 92 · 2017: 44 · 2016: 16
- FWCI
- 80.13
- Type
- conference paper
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- International World Wide Web Conference (WWW) (publishedVersion)
- arXiv (Cornell University) (submittedVersion)
- Funders
- DARPA (W911NF-12-1-0037)
- NSF (CCF-1101743)
- J.S. McDonnell Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- social bots, bot detection, Twitter, online manipulation, misinformation, terrorist propaganda, user popularity
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
