The DOI matches its Crossref record; 35 of the 37 references it cites were found in a registry or an index.
Using BERT to Extract Topic-Independent Sentiment Features for Social Media Bot Detection
Abstract
Millions of online posts about different topics and products are shared on popular social media platforms. One use of this content is to provide crowd-sourced information about a specific topic, event, or product. However, this use raises an important question: what percentage of the information available through these services is trustworthy? In particular, might some of this information be generated by a machine, i.e., a "bot" instead of a human? Bots can be, and often are, purposely designed to generate enough volume to skew an apparent trend or position on a topic, yet the consumer of such content cannot easily distinguish a bot post from a human post. This paper introduces a new model that uses Bidirectional Encoder Representations from Transformers (Google Bert) for sentiment classification of tweets to identify topic-independent features for the social media bot detection model. Using a Natural Language Processing approach to derive topic-independent features for the new bot detection model distinguishes this work from previous bot detection models. We achieve 94% accuracy classifying the contents of Cresci data set [1] as generated by a bot or a human, where the most accurate prior work achieved an accuracy of 92%.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Content posted by bots also provides essential clues (Kudugunta and Ferrara, 2018;Heidari and Jones, 2020).”
From Anatomy of an AI-powered malicious social botnet (2024) · cites this paper as Heidari and Jones, 2020
Citations
Cites
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2018) · arXiv · doi:10.48550/arxiv.1810.04805 · Cited by 73,173
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019) · Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · doi:10.18653/v1/n19-1423 · Cited by 54,325
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071
Deep Contextualized Word Representations
Peters, M., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & Zettlemoyer, L. (2018) · Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · doi:10.18653/v1/n18-1202 · Cited by 11,983
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., & Potts, C. (2013) · Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d13-1170 · Cited by 6,878
Fake News Detection on Social Media: A Data Mining Perspective
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3137597.3137600 · Cited by 3,304
BotOrNot: A System to Evaluate Social Bots
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · Proceedings of the 25th International Conference Companion on World Wide Web - WWW 16 Companion · doi:10.1145/2872518.2889302 · Cited by 909
Howard, P. N., Woolley, S., & Calo, R. (2018) · Journal of Information Technology & Politics · doi:10.1080/19331681.2018.1448735 · Cited by 398
Empirical evaluation and new design for fighting evolving Twitter spammers
Yang, C., Harkreader, R., & Gu, G. (2013) · IEEE Transactions on Information Forensics and Security · doi:10.1109/tifs.2013.2267732 · Cited by 346
Twitter spammer detection using data stream clustering
Miller, Z., Dickinson, B., Deitrick, W., Hu, W., & Wang, A. H. (2014) · Information Sciences · doi:10.1016/j.ins.2013.11.016 · Cited by 332
Using sentiment to detect bots on Twitter: Are humans more opinionated than bots?
Dickerson, J. P., Kagan, V., & Subrahmanian, V. S. (2014) · IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM) · doi:10.1109/asonam.2014.6921650 · Cited by 288
The paradigm-shift of social spambots: Evidence, theories, and tools for the arms race
Cresci, S., Pietro, R. D., Petrocchi, M., Spognardi, A., & Tesconi, M. (2017) · Technical University of Denmark, DTU Orbit (Technical University of Denmark, DTU) · doi:10.48550/arxiv.1701.03017 · Cited by 237
A generic statistical approach for spam detection in Online Social Networks
Ahmed, F., & Abulaish, M. (2013) · Computer Communications · doi:10.1016/j.comcom.2013.04.004 · Cited by 199
DNA-Inspired Online Behavioral Modeling and Its Application to Spambot Detection
Cresci, S., Di Pietro, R., Petrocchi, M., Spognardi, A., & Tesconi, M. (2016) · IEEE Intelligent Systems · doi:10.1109/mis.2016.29 · Cited by 194 · cited 1×
“The last section of this work evaluates the proposed new bot detection model compared with existing state-of-the-art bot technology techniques in the same training and test data set of Cresci [27].”
MethodPolitical Bots and the Manipulation of Public Opinion in VenezuelaPreprint
Forelle, M., Howard, P. N., Monroy‐Hernández, A., & Savage, S. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2635800 · Cited by 116
BotOrNot: A System to Evaluate Social BotsPreprint
Davis, C. A., Varol, O., Ferrara, E., Flammini, A., & Menczer, F. (2016) · arXiv (Cornell University) · doi:10.48550/arxiv.1602.00975 · Cited by 105
Heidari, M., Jones, J. H., & Uzuner, Ö. (2020) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw51313.2020.00071 · Cited by 99
Self-citationSocial Network Analysis with Content and Graphs
Campbell, W. M., Dagli, C. K., & Weinstein, C. J. (2013) · Cited by 68
Heidari, M., & Rafatirad, S. (2020) · International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP) · doi:10.1109/smap49528.2020.9248443 · Cited by 67
Self-citationAn Unsupervised Approach to Detect Spam Campaigns that Use Botnets on TwitterPreprint
Chen, Z., & Subramanian, D. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1804.05232 · Cited by 53
Garibay, I. I., Mantzaris, A. V., Rajabi, A., & Taylor, C. E. (2019) · Scientific Reports · doi:10.1038/s41598-019-55178-8 · Cited by 51
Deep Reinforcement Learning-based Text Anonymization against Private-Attribute Inference
Mosallanezhad, A., Beigi, G., & Liu, H. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1240 · Cited by 46
Yousefi, N., Alaghband, M., & Garibay, I. I. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1912.02629 · Cited by 34
Saadati, M., Nelson, J. K., & Ayaz, H. (2019) · IEEE International Workshop on Machine Learning for Signal Processing (MLSP) · doi:10.1109/mlsp.2019.8918861 · Cited by 33
Application of the Benford’s law to Social bots and Information Operations activities
Madahali, L., & Hall, M. (2020) · International Conference on Cyber Situational Awareness, Data Analytics and Assessment (CyberSA) · doi:10.1109/cybersa49311.2020.9139709 · Cited by 26
Scraping Social Media Photos Posted in Kenya and Elsewhere to Detect and Analyze Food Types
Jalal, M., Wang, K., Jefferson, S., Zheng, Y., Nsoesie, E. O., & Betke, M. (2019) · International Workshop on Multimedia Assisted Dietary Management (MADiMa) · doi:10.1145/3347448.3357170 · Cited by 26
TOT-Net: An Endeavor Toward Optimizing Ternary Neural Networks
Nazari, N., Loni, M., Salehi, M. E., Daneshtalab, M., & Sjödin, M. (2019) · Euromicro Conference on Digital System Design (DSD) · doi:10.1109/dsd.2019.00052 · Cited by 24
User Behavior Modelling for Fake Information Mitigation on Social Web
Rajabi, Z., Shehu, A., & Purohit, H. (2019) · Lecture notes in computer science · doi:10.1007/978-3-030-21741-9_24 · Cited by 21
Behnia, F., Mirzaeian, A., Sabokrou, M., Manoj, S., Mohsenin, T., Khasawneh, K. N., Zhao, L., Homayoun, H., & Sasan, A. (2020) · International Symposium on Quality Electronic Design (ISQED) · doi:10.1109/isqed48828.2020.9136987 · Cited by 17
Probabilistic Model of Narratives Over Topical Trends in Social Media
Oghaz, T. A., Mutlu, E. Ç., Jasser, J., Yousefi, N., & Garibay, I. I. (2020) · ACM Conference on Hypertext and Social Media (HT) · doi:10.1145/3372923.3404790 · Cited by 14
Personalized recommender system based on social relations
Ebrahimi, F., & Golpayegani, A. H. (2016) · Iranian Conference on Electrical Engineering · doi:10.1109/iraniancee.2016.7585521 · Cited by 12
Curtailing Fake News Propagation with PsychographicsPreprint
Safadi, H., Li, W., Rahmati, P., & Soleymani, S. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3558236 · Cited by 10
Rajabi, A., Gunaratne, C., Mantzaris, A. V., & Garibay, I. I. (2020) · Lecture notes in computer science · doi:10.1007/978-3-030-61255-9_13 · Cited by 8
Maximal Sequence Mining approach for topic detection from microblog streams
Jafariakinabad, F., & Hua, K. A. (2016) · IEEE Symposium Series on Computational Intelligence (SSCI) · doi:10.1109/ssci.2016.7849940 · Cited by 5
Rajabi, A., Gunaratne, C., Mantzaris, A. V., & Garibay, I. I. (2020) · Adaptive Agents and Multi-Agents Systems · doi:10.65109/yjde7402 · Cited by 4
Research works printed but not linked: 1
- 0, IMDB Movie Reviews Dataset
Cited by
Sorted by citations
Emotion Detection of Textual Data: An Interdisciplinary Survey
Zad, S., Heidari, M., Jones, J. H., & Uzuner, Ö. (2021) · IEEE World AI IoT Congress (AIIoT) · doi:10.1109/aiiot52608.2021.9454192 · Cited by 115
Self-citationHeidari, M., Jones, J. H., & Uzuner, Ö. (2020) · IEEE ... International Conference on Data Mining workshops · doi:10.1109/icdmw51313.2020.00071 · Cited by 99
Self-citationA Survey on Concept-Level Sentiment Analysis Techniques of Textual Data
Zad, S., Heidari, M., Jones, J. H., & Uzuner, Ö. (2021) · IEEE World AI IoT Congress (AIIoT) · doi:10.1109/aiiot52608.2021.9454169 · Cited by 92
Self-citationUnsupervised Social Bot Detection via Structural Information Theory
Peng, H. M., Zhang, J., Huang, X., Hao, Z., Li, A., Yu, Z., & Yu, P. S. (2024) · ACM Transactions on Information Systems · doi:10.1145/3660522 · Cited by 82
Anatomy of an AI-powered malicious social botnet
Yang, K., & Menczer, F. (2024) · Journal of Quantitative Description: Digital Media · doi:10.51685/jqd.2024.icwsm.7 · Cited by 71
Metadata
- Authors
- Heidari, M., & Jones, J. H.
- Affiliations
- George Mason University (US)
- ORCID
- Jones, James H
- Venue
- 2020 11th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON)
- Publisher
- IEEE
- Published
- 2020-10-28
- Pages
- 0542-0547
- OpenAlex
- W3116492894
- Cited by
- 102
- References
- 37
- Citations by year
- 2026: 4 · 2025: 13 · 2024: 12 · 2023: 18 · 2022: 21 · 2021: 27 · 2020: 7
- FWCI
- 13.84
- Type
- conference paper
- Language
- en
- Open access
- closed
- Licence
- Locations
- IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (publishedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- BERT, sentiment classification, tweets, machine-generated content
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review