Separating Facts from Fiction: Linguistic Models to Classify Suspicious and Trusted News Posts on Twitter
Open accessAbstract
Pew research polls report 62 percent of U.S. adults get news on social media (Gottfried and Shearer, 2016). In a December poll, 64 percent of U.S. adults said that “made-up news” has caused a “great deal of confusion” about the facts of current events (Barthel et al., 2016). Fabricated stories in social media, ranging from deliberate propaganda to hoaxes and satire, contributes to this confusion in addition to having serious effects on global stability. In this work we build predictive models to classify 130 thousand news posts as suspicious or verified, and predict four sub-types of suspicious news – satire, hoaxes, clickbait and propaganda. We show that neural network models trained on tweet content and social network interactions outperform lexical models. Unlike previous work on deception detection, we find that adding syntax and grammar features to our models does not improve performance. Incorporating linguistic features improves classification results, however, social interaction features are most informative for finer-grained separation between four types of suspicious news posts.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The text we found for this paper looks like another document, so the self-check wasn’t run.
Cites
Adam: A Method for Stochastic Optimization
Kingma, D. P., & Ba, J. (2014) · arXiv · doi:10.48550/arxiv.1412.6980 · Cited by 98,946 · see citing works
Glove: Global Vectors for Word Representation
Pennington, J., Socher, R., & Manning, C. (2014) · Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/v1/d14-1162 · Cited by 34,071 · see citing works
Large-Scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · 2014 IEEE Conference on Computer Vision and Pattern Recognition · doi:10.1109/cvpr.2014.223 · Cited by 6,354 · see citing works
Liberals and conservatives rely on different sets of moral foundations.
Graham, J., Haidt, J., & Nosek, B. A. (2009) · Journal of Personality and Social Psychology · doi:10.1037/a0015141 · Cited by 4,560 · see citing works
When Morality Opposes Justice: Conservatives Have Moral Intuitions that Liberals may not Recognize
Haidt, J., & Graham, J. (2007) · Social Justice Research · doi:10.1007/s11211-007-0034-z · Cited by 2,385 · see citing works
Linguistic Inquiry and Word Count (LIWC)
Chung, C. K., & Pennebaker, J. W. (2012) · IGI Global eBooks · doi:10.4018/978-1-60960-741-8.ch012 · Cited by 2,152 · see citing works
Liu, B., Hu, M., & Cheng, J. (2005) · International World Wide Web Conference (WWW) · doi:10.1145/1060745.1060797 · Cited by 1,612 · see citing works
Large-scale Video Classification with Convolutional Neural Networks
Karpathy, A., Toderici, G. D., Shetty, S., Leung, T., Sukthankar, R., & Fei-Fei, L. (2014) · Cited by 1,266 · see citing works
News use across social media platforms 2016
Gottfried, J. A., & Shearer, E. (2016) · Cited by 1,165 · see citing works
Learning extraction patterns for subjective expressions
Riloff, E., & Wiebe, J. (2003) · Proceedings of the 2003 conference on Empirical methods in natural language processing - · doi:10.3115/1119355.1119369 · Cited by 1,059 · see citing works
Automatic deception detection: Methods for finding fake news
Conroy, N. K., Rubin, V. L., & Chen, Y. (2015) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2015.145052010082 · Cited by 990 · see citing works
Twitter under crisis: can we trust what we RT?
Mendoza, M., Poblete, B., & Castillo, C. (2010) · Proceedings of the First Workshop on Social Media Analytics · doi:10.1145/1964858.1964869 · Cited by 955 · see citing works
Effective Use of Word Order for Text Categorization with Convolutional Neural Networks
Johnson, R., & Zhang, T. (2015) · Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT) · doi:10.3115/v1/n15-1011 · Cited by 787 · see citing works
Deception detection for news: Three types of fakes
Rubin, V. L., Chen, Y., & Conroy, N. K. (2015) · Proceedings of the Association for Information Science and Technology · doi:10.1002/pra2.2015.145052010083 · Cited by 558 · see citing works
On Lying and Being Lied To: A Linguistic Analysis of Deception in Computer-Mediated Communication
Hancock, J. T., Curry, L. E., Goorha, S., & Woodworth, M. (2007) · Discourse Processes · doi:10.1080/01638530701739181 · Cited by 523 · see citing works
Syntactic Stylometry for Deception Detection
Feng, S., Banerjee, R., & Choi, Y. (2012) · Cited by 499 · see citing works
Mihalcea, R. F., & Strapparava, C. (2009) · doi:10.3115/1667583.1667679 · Cited by 341 · see citing works
Disinformation on the Web: Impact, Characteristics, and Detection of Wikipedia Hoaxes
Kumar, S., West, R., & Leskovec, J. (2016) · Proceedings of the 25th International Conference on World Wide Web · doi:10.1145/2872427.2883085 · Cited by 310 · cited 3× · see citing works
“In addition to contrasting linguistic realizations behind different types of suspicious news on Twitter, we are interested in qualitatively evaluating differences in retweet patterns across suspicious news types (Lumezanu and Klein, 2012; Mendoza et al., 2010; Kumar et al., 2016).”
BackgroundLinguistic Models for Analyzing and Detecting Biased Language
Recasens, M. V., Danescu-Niculescu-Mizil, C., & Jurafsky, D. (2013) · Cited by 294 · see citing works
Cues to deception and ability to detect lies as a function of police interview styles.
Vrij, A., Mann, S. A., Kristen, S., & Fisher, R. P. (2007) · Law and Human Behavior · doi:10.1007/s10979-006-9066-4 · Cited by 277 · see citing works
HOOPER, J. B. (1975) · BRILL eBooks · doi:10.1163/9789004368828_005 · Cited by 218 · see citing works
Combining multiple sources of knowledge in deep CNNs for action recognition
Park, E., Han, X., Berg, T. L., & Berg, A. C. (2016) · IEEE Winter Conference on Applications of Computer Vision · doi:10.1109/wacv.2016.7477589 · Cited by 167 · see citing works
Experiments in open domain deception detection
Pérez-Rosas, V., & Mihalcea, R. (2015) · Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing · doi:10.18653/v1/d15-1133 · Cited by 97 · see citing works
#bias: Measuring the Tweeting Behavior of Propagandists
Lumezanu, C., Feamster, N., & Klein, H. K. (2021) · Proceedings of the International AAAI Conference on Web and Social Media · doi:10.1609/icwsm.v6i1.14247 · Cited by 59 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
A survey on fake news and rumour detection techniques
Bondielli, A., & Marcelloni, F. (2019) · Information Sciences · doi:10.1016/j.ins.2019.05.035 · Cited by 658 · see citing works
Combating Fake News: A Survey on Identification and Mitigation Techniques
Sharma, K., Qian, F., Jiang, H., Ruchansky, N., Zhang, M., & Liu, Y. (2019) · ACM Transactions on Intelligent Systems and Technology · doi:10.1145/3305260 · Cited by 514 · see citing works
Multimodal fake news detection via progressive fusion networks
Jing, J., Wu, H., Sun, J., Fang, X., & Zhang, H. (2023) · Information Processing & Management · doi:10.1016/j.ipm.2022.103120 · Cited by 243 · see citing works
Leveraging the Crowd to Detect and Reduce the Spread of Fake News and Misinformation
Kim, J., Tabibian, B., Oh, A., Schölkopf, B., & Gomez-Rodriguez, M. (2018) · Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining · doi:10.1145/3159652.3159734 · Cited by 230 · see citing works
Fake News Detection in Social Networks via Crowd Signals
Tschiatschek, S., Singla, A., Gomez Rodriguez, M., Merchant, A., & Krause, A. (2018) · Companion of the The Web Conference 2018 on The Web Conference 2018 - WWW '18 · doi:10.1145/3184558.3188722 · Cited by 229 · see citing works
Falling for Fake News: Investigating the Consumption of News via Social Media
Flintham, M., Karner, C., Bachour, K., Creswick, H., Gupta, N., & Moran, S. (2018) · Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems · doi:10.1145/3173574.3173950 · Cited by 194 · see citing works
Automating fake news detection system using multi-level voting model
Kaur, S., Kumar, P., & Kumaraguru, P. (2020) · Soft Computing · doi:10.1007/s00500-019-04436-y · Cited by 190 · see citing works
The limitations of stylometry for detecting machine-generated fake news
Schuster, T., Schuster, R., Shah, D. J., & Barzilay, R. (2020) · Computational Linguistics · doi:10.1162/coli_a_00380 · Cited by 109 · see citing works
The Web of False Information: Rumors, Fake News, Hoaxes, Clickbait, and Various Other Shenanigans
Zannettou, S., Sirivianos, M., Blackburn, J., & Kourtellis, N. (2019) · Journal of Data and Information Quality · doi:10.1145/3309699 · Cited by 86 · see citing works
A Reasoning and Value Alignment Test to Assess Advanced GPT Reasoning
Mcintosh, T. R., Liu, T., Susnjak, T., Watters, P., & Halgamuge, M. N. (2024) · ACM Transactions on Interactive Intelligent Systems · doi:10.1145/3670691 · Cited by 42 · see citing works
Morality classification in natural language text
Pavan, M. C., dos Santos, V. G., Lan, A. G. J., Martins, J. T., dos Santos, W. R., Deutsch, C., da Costa, P. B., Hsieh, F. C., & Paraboni, I. (2023) · IEEE Transactions on Affective Computing · doi:10.1109/taffc.2020.3034050 · Cited by 30 · see citing works
A Reliable Weighting Scheme for the Aggregation of Crowd Intelligence to Detect Fake News
Tchakounté, F., Faissal, A., Atemkeng, M., & Ntyam, A. (2020) · Information · doi:10.3390/info11060319 · Cited by 23 · see citing works
- Authors
- Volkova, S., Shaffer, K., Jang, J. Y., & Hodas, N.
- Affiliations
- Pacific Northwest National Laboratory (US)
- Venue
- Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)
- Published
- 2017
- Pages
- 647-653
- ISSN
- 0736-587X (ISSN-L)
- OpenAlex
- W2740560510
- Cited by
- 381
- References
- 24
- Citations by year
- 2026: 31 · 2025: 24 · 2024: 33 · 2023: 35 · 2022: 50 · 2021: 69 · 2020: 53 · 2019: 50 · 2018: 32 · 2017: 4
- FWCI
- 121.13
- Type
- conference paper
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- cc-by
- Locations
- Proceedings of the conference - Association for Computational Linguistics. Meeting (publishedVersion, cc-by)
- Funders
- U.S. Department of Energy
- Battelle
- Laboratory Directed Research and Development
- Pacific Northwest National Laboratory
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Physical Sciences › Computer Science › Artificial Intelligence › Hate Speech and Cyberbullying Detection
- Keywords
- fake news, hoaxes, satire, clickbait, propaganda, Twitter, linguistic features, social network interactions, deception detection, news verification, neural network classifier
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review