The DOI matches its Crossref record; 52 of the 52 references it cites were found in a registry or an index.
The Limitations of Stylometry for Detecting Machine-Generated Fake News
Open accessAbstract
Recent developments in neural language models (LMs) have raised concerns about their potential misuse for automatically spreading misinformation. In light of these concerns, several studies have proposed to detect machine-generated fake news by capturing their stylistic differences from human-written text. These approaches, broadly termed stylometry, have found success in source attribution and misinformation detection in human-written texts. However, in this work, we show that stylometry is limited against machine-generated misinformation. Whereas humans speak differently when trying to deceive, LMs generate stylistically consistent text, regardless of underlying motive. Thus, though stylometry can successfully prevent impersonation by identifying text provenance, it fails to distinguish legitimate LM applications from those that introduce false information. We create two benchmarks demonstrating the stylistic similarity between malicious and legitimate uses of LMs, utilized in auto-completion and editing-assistance settings. 1 Our findings highlight the need for non-stylometry approaches in detecting machine-generated misinformation, and open up the discussion on the desired evaluation benchmarks.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Figures
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“In the initial stages of generative modeling's emergence, when it had yet to pose significant societal challenges, the emphasis was on assessing the truth or falsity of the content in LLM-generated text, with less regard for its source (Schuster et al. 2020b).”
From A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions (2025) · cites this paper as Schuster et al. 2020b
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., & Polosukhin, I. (2017) · arXiv · doi:10.48550/arxiv.1706.03762 · Cited by 120,404
The spread of true and false news online
Vosoughi, S., Roy, D., & Aral, S. (2018) · Science · doi:10.1126/science.aap9559 · Cited by 8,847
Fake News Detection on Social Media: A Data Mining Perspective
Shu, K., Sliva, A., Wang, S., Tang, J., & Liu, H. (2017) · ACM SIGKDD Explorations Newsletter · doi:10.1145/3137597.3137600 · Cited by 3,304
The Dishonesty of Honest People: A Theory of Self-Concept Maintenance
Mazar, N., Amir, O., & Ariely, D. (2008) · Journal of Marketing Research · doi:10.1509/jmkr.45.6.633 · Cited by 2,875
Information credibility on twitter
Castillo, C., Mendoza, M., & Poblete, B. (2011) · Proceedings of the 20th international conference on World wide web · doi:10.1145/1963405.1963500 · Cited by 2,593
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2067 · Cited by 2,010
The Curious Case of Neural Text Degeneration
Holtzman, A., Buys, J., Du, L., Forbes, M., & Choi, Y. (2019) · arXiv · doi:10.48550/arxiv.1904.09751 · Cited by 1,304
Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Ott, M., Choi, Y., Cardie, C., & Hancock, J. T. (2011) · arXiv · doi:10.48550/arxiv.1107.4557 · Cited by 1,008
Truth of Varying Shades: Analyzing Language in Fake News and Political Fact-Checking
Rashkin, H., Choi, E., Jang, J. Y., Volkova, S., & Choi, Y. (2017) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d17-1317 · Cited by 965
Automatic Detection of Fake NewsPreprint
Pérez‐Rosas, V., Kleinberg, B., Lefevre, A., & Mihalcea, R. F. (2017) · arXiv (Cornell University) · doi:10.48550/arxiv.1708.07104 · Cited by 709
NewsQA: A Machine Comprehension Dataset
Trischler, A., Wang, T., Yuan, X., Harris, J., Sordoni, A., Bachman, P., & Suleman, K. (2017) · Proceedings of the 2nd Workshop on Representation Learning for NLP · doi:10.18653/v1/w17-2623 · Cited by 706 · cited 5×
“We use the newsQA dataset (Trischler et al. 2017) that contains Cable News Network (CNN) articles with corresponding questions and answers (segments copied verbatim from the text).”
BackgroundMethodInfluentialA Stylometric Inquiry into Hyperpartisan and Fake News
Potthast, M., Kiesel, J., Reinartz, K., Bevendorff, J., & Stein, B. (2018) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p18-1022 · Cited by 700
Liu, Y., & Wu, Y.-F. (2018) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v32i1.11268 · Cited by 663 · cited 2×
“(c) In the article extension case (Section 5), the bold red text is the generation of GPT-2 Medium to extend the prefix. incomplete data, especially given the need to detect fake news early (Liu and Wu 2018) (before propagation and user engagement patterns are fully formed).”
BackgroundEnquiring Minds: Early Detection of Rumors in Social Media from Enquiry Posts
Zhao, Z., Resnick, P., & Mei, Q. (2015) · Proceedings of the 24th International Conference on World Wide Web · doi:10.1145/2736277.2741637 · Cited by 583
Syntactic Stylometry for Deception Detection
Feng, S., Banerjee, R., & Choi, Y. (2012) · Cited by 499
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics: System Demonstrations · doi:10.18653/v1/p19-3019 · Cited by 470
Mosteller, F., & Wallace, D. L. (1963) · Journal of the American Statistical Association · doi:10.1080/01621459.1963.10500849 · Cited by 466
GLTR: Statistical Detection and Visualization of Generated Text
Gehrmann, S., Strobelt, H., & Rush, A. M. (2019) · arXiv · doi:10.48550/arxiv.1906.04043 · Cited by 460
Volkova, S., Shaffer, K., Jang, J. Y., & Hodas, N. (2017) · Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) · doi:10.18653/v1/p17-2102 · Cited by 381
TweetCred: Real-Time Credibility Assessment of Content on Twitter
Gupta, A., Kumaraguru, P., Castillo, C., & Meier, P. P. (2014) · Lecture notes in computer science · doi:10.1007/978-3-319-13734-6_16 · Cited by 376
Mihalcea, R. F., & Strapparava, C. (2009) · Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP) · doi:10.3115/1667583.1667679 · Cited by 340
DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning
Popat, K., Mukherjee, S., Yates, A., & Weikum, G. (2018) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d18-1003 · Cited by 336
Detecting Hoaxes, Frauds, and Deception in Writing Style Online
Afroz, S., Brennan, M., & Greenstadt, R. (2012) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2012.34 · Cited by 320
Fake News Detection in Social Networks via Crowd Signals
Tschiatschek, S., Singla, A., Gomez Rodriguez, M., Merchant, A., & Krause, A. (2018) · Companion of the The Web Conference 2018 on The Web Conference 2018 - WWW '18 · doi:10.1145/3184558.3188722 · Cited by 229
MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims
Augenstein, I., Lioma, C., Wang, D., Lima, L. C., Hansen, C., Hansen, C., & Simonsen, J. G. (2019) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.18653/v1/d19-1475 · Cited by 226
Surveying Stylometry Techniques and Applications
Neal, T., Sundararajan, K., Fatima, A., Yan, Y., Xiang, Y., & Woodard, D. (2018) · ACM Computing Surveys · doi:10.1145/3132039 · Cited by 222
Fake News: A Survey of Research, Detection Methods, and Opportunities.Preprint
Zhou, X., & Zafarani, R. (2018) · arXiv (Cornell University) · Cited by 214
Popat, K., Mukherjee, S., Strötgen, J., & Weikum, G. (2017) · International World Wide Web Conference (WWW) · doi:10.1145/3041021.3055133 · Cited by 213
The Fact Extraction and VERification (FEVER) Shared Task
Thorne, J., Vlachos, A., Cocarascu, O., Christodoulopoulos, C., & Mittal, A. (2018) · Proceedings of the First Workshop on Fact Extraction and VERification (FEVER) · doi:10.18653/v1/w18-5501 · Cited by 212
A Retrospective Analysis of the Fake News Challenge Stance Detection Task
Hanselowski, A., Pvs, A., Schiller, B., Caspelherr, F., Chaudhuri, D., Meyer, C. M., & Gurevych, I. (2018) · TUbilio (Technical University of Darmstadt) · doi:10.48550/arxiv.1806.05180 · Cited by 210
Neural network applications in stylometry: The Federalist Papers
Tweedie, F. J., Singh, S. K., & Holmes, D. I. (1996) · Computers and the Humanities · doi:10.1007/bf00054024 · Cited by 209
Bond, G. D., & Lee, A. Y. (2005) · Applied Cognitive Psychology · doi:10.1002/acp.1087 · Cited by 195
Brennan, M., Afroz, S., & Greenstadt, R. (2012) · ACM Transactions on Information and System Security · doi:10.1145/2382448.2382450 · Cited by 194
Towards Debiasing Fact Verification Models
Schuster, T., Shah, D., Yeo, Y. J. S., Roberto Filizzola Ortiz, D., Santus, E., & Barzilay, R. (2019) · Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · doi:10.18653/v1/d19-1341 · Cited by 171
Self-citationBarack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling
Logan, R., Liu, N. F., Peters, M. E., Gardner, M., & Singh, S. (2019) · Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/p19-1598 · Cited by 168
De-anonymizing programmers via code stylometry
Islam, A. C., Harang, R., Liu, A., Narayanan, A., Voss, C. R., Yamaguchi, F., & Greenstadt, R. (2015) · USENIX Security Symposium · Cited by 166
All-in-one: Multi-task Learning for Rumour VerificationPreprint
Kochkina, E., Liakata, M., & Zubiaga, A. (2018) · arXiv (Cornell University) · doi:10.48550/arxiv.1806.03713 · Cited by 142
Doppelgänger Finder: Taking Stylometry to the Underground
Afroz, S., Islam, A. C., Stolerman, A., Greenstadt, R., & McCoy, D. (2014) · IEEE Symposium on Security and Privacy (S&P) · doi:10.1109/sp.2014.21 · Cited by 101
Language-Aware Truth Assessment of Fact Candidates
Nakashole, N., & Mitchell, T. M. (2014) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/v1/p14-1095 · Cited by 90
Real or Fake? Learning to Discriminate Machine from Human Generated Text
Bakhtin, A., Gross, S., Ott, M., Deng, Y., Ranzato, M., & Szlam, A. (2019) · arXiv · doi:10.48550/arxiv.1906.03351 · Cited by 77
Continuous N-gram Representations for Authorship Attribution
Sari, Y. A. P., Vlachos, A., & Stevenson, M. (2017) · Conference of the European Chapter of the Association for Computational Linguistics · doi:10.18653/v1/e17-2043 · Cited by 62
MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of ClaimsPreprint
Augenstein, I., Lioma, C., Wang, D., Lima, L. C., Hansen, C., Hansen, C., & Simonsen, J. G. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1909.03242 · Cited by 50
Human Behavior and Deception Detection
Frank, M. G., Menasco, M. A., & O’Sullivan, M. (2008) · Wiley eBooks · doi:10.1002/9780470087923.hhs299 · Cited by 39
(2019) · doi:10.18653/v1/d19-1292 · Cited by 36
Unifying Human and Statistical Evaluation for Natural Language GenerationPreprint
Hashimoto, T., Zhang, H., & Liang, P. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1904.02792 · Cited by 33
Detecting deception using critical segments
Enos, F., Shriberg, E. E., Graciarena, M., Hirschberg, J., & Stolcke, A. (2007) · Interspeech (USB) · doi:10.21437/interspeech.2007-619 · Cited by 32
Deception Detection in News Reports in the Russian Language: Lexics and Discourse
Pisarevskaya, D. (2017) · doi:10.18653/v1/w17-4213 · Cited by 31
Towards a description of negatively conditioned subject operator inversion in English
Rudanko, J. (1982) · English Studies · doi:10.1080/00138388208598192 · Cited by 29
The Language of Legal and Illegal Activity on the Darknet
Choshen, L., Eldad, D., Hershcovich, D., Sulem, E., & Abend, O. (2019) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p19-1419 · Cited by 25
von Davier, M. (2019) · arXiv (Cornell University) · doi:10.48550/arxiv.1908.08594 · Cited by 21
Automatic IQ Estimation Using Stylometric Methods
Abramov, P. S., & Yampolskiy, R. V. (2019) · Advances in educational technologies and instructional design book series · doi:10.4018/978-1-5225-8431-5.ch004 · Cited by 8
Automatic IQ estimation using stylometry methods.
Abramov, P. S. (2018) · doi:10.18297/etd/2922 · Cited by 2
Research works printed but not linked: 18
- Caliskan-Islam Aylin, 2015, 24th USENIX Security Symposium (USENIX Security 15), 255
- Dostoyevsky Fyodor, 1866, Crime and Punishment
- Enos Frank, 2007, International Speech Communication Association, 2281
- Feng Song, 2012, Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics: Short Papers - Volume 2, 171
- Goswami Sumit, 2009, International AAAI Conference on Web and Social Media
- Hanselowski Andreas, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 1859
- Kochkina Elena, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 3402
- Liu Yang, 2018, AAAI Conference on Artificial Intelligence, 354
- Matsumoto David, 2011, FBI Law Enforcement Bulletin, 1
- Mosteller Frederick, 1963, Journal of the American Statistical Association, 58, 275
- Neal Tempestt J., 2017, ACM Computing Surveys, 50, 86:1
- Ott Myle, 2011, Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, 309
- Pérez-Rosas Verónica, 2018, Proceedings of the 27th International Conference on Computational Linguistics, 3391
- Radford Alec, 2019, OpenAI Blog, 1
- Vaswani Ashish, 2017, Advances in Neural Information Processing Systems, 5998
- Wolf Thomas, 2019, ArXiv
- Zellers Rowan, 2019, Advances in Neural Information Processing Systems 32, 9054
- Zhou Xinyi, 2018, arXiv preprint arXiv:1812.00315 · arXiv
Cited by
Sorted by citations
A Survey on Automated Fact-Checking
Guo, Z., Schlichtkrull, M., & Vlachos, A. (2022) · Transactions of the Association for Computational Linguistics · doi:10.1162/tacl_a_00454 · Cited by 500
Automatic Detection of Generated Text is Easiest when Humans are Fooled
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2019) · arXiv · doi:10.48550/arxiv.1911.00650 · Cited by 264
“Schuster et al. (2019) argue that the language distributional features implicitly or explicitly employed by these detectors are insufficient; instead, one should look to explicit fact-verification models.”
BackgroundAutomatic Detection of Generated Text is Easiest when Humans are Fooled
Ippolito, D., Duckworth, D., Callison-Burch, C., & Eck, D. (2020) · Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics · doi:10.18653/v1/2020.acl-main.164 · Cited by 264
A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
Wu, J., Yang, S., Zhan, R., Yuan, Y., Chao, L. S., & Wong, D. F. (2025) · Computational Linguistics · doi:10.1162/coli_a_00549 · Cited by 212
Automatic Detection of Machine Generated Text: A Critical Survey
Jawahar, G., Abdul-Mageed, M., & Lakshmanan, V. S. (2020) · Proceedings of the 28th International Conference on Computational Linguistics · doi:10.18653/v1/2020.coling-main.208 · Cited by 156
Metadata
- Authors
- Schuster, T., Schuster, R., Shah, D. J., & Barzilay, R.
- Affiliations
- MIT Computer Science and Artificial Intelligence Laboratory (US)
- Tel Aviv University (IL)
- Venue
- Computational Linguistics
- Publisher
- MIT Press - Journals
- Published
- 2020-06
- Volume
- 46
- Issue
- 2
- Pages
- 499-510
- ISSN
- 0891-2017 (ISSN-L)
- 1530-9312
- OpenAlex
- W3008170323
- Cited by
- 109
- References
- 52
- Citations by year
- 2026: 8 · 2025: 14 · 2024: 15 · 2023: 22 · 2022: 29 · 2021: 13 · 2020: 8
- FWCI
- 24.02
- Type
- journal article
- Language
- en
- Open access
- diamond
- doi.org
- Licence
- cc-by-nc-nd
- Locations
- Computational Linguistics (publishedVersion, cc-by-nc-nd)
- arXiv (Cornell University) (submittedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-nd)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- arXiv (Cornell University)
- Funders
- National Science Foundation (1650589)
- Topics
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Authorship Attribution and Profiling
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Keywords
- stylometry, misinformation detection, neural language models, stylistic similarity, source attribution, autocomplete, evaluation benchmark, deception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
