The DOI matches its DataCite record; 42 of the 42 references it cites were found in a registry or an index.
Finding Deceptive Opinion Spam by Any Stretch of the Imagination
Open accessAbstract
Consumers increasingly rate, review and research products online. Consequently, websites containing consumer reviews are becoming targets of opinion spam. While recent work has focused primarily on manually identifiable instances of opinion spam, in this work we study deceptive opinion spam---fictitious opinions that have been deliberately written to sound authentic. Integrating work from psychology and computational linguistics, we develop and compare three approaches to detecting deceptive opinion spam, and ultimately develop a classifier that is nearly 90% accurate on our gold-standard opinion spam dataset. Based on feature analysis of our learned models, we additionally make several theoretical contributions, including revealing a relationship between deceptive opinions and imaginative writing.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“The dataset of deceptive reviews from Ott et al. [46, 47] has 800 truthful reviews and 800 deceptive reviews for hotels in Chicago.”
From Improving Human-AI Collaboration With Descriptions of AI Behavior (2023) · cites this paper as [47]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Text categorization with Support Vector Machines: Learning with many relevant features
Joachims, T. (1998) · Lecture notes in computer science · doi:10.1007/bfb0026683 · Cited by 8,271
Machine learning in automated text categorization
Sebastiani, F. (2002) · ACM Computing Surveys · doi:10.1145/505282.505283 · Cited by 7,927
SRILM - an extensible language modeling toolkit
Stolcke, A. (2002) · International Conference on Spoken Language Processing (ICSLP) · doi:10.21437/icslp.2002-303 · Cited by 4,385
Electronic word-of-mouth in hospitality and tourism management
Litvin, S. W., Goldsmith, R. E., & Pan, B. (2008) · Tourism Management · doi:10.1016/j.tourman.2007.05.011 · Cited by 3,267
Generalized additive models for location, scale and shape.
Rigby, R. A., & Stasinopoulos, D. M. (2005) · Journal of the Royal Statistical Society Series C: Applied Statistics · doi:10.1111/j.1467-9876.2005.00510.x · Cited by 3,228
Accurate unlexicalized parsing
Klein, D., & Manning, C. D. (2003) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.3115/1075096.1075150 · Cited by 3,055
Longman Grammar of Spoken and Written English
Biber, D., & Quirk, R. (1999) · TESOL Quarterly · doi:10.2307/3587792 · Cited by 2,766
Accuracy of Deception Judgments
Bond, C. F., & DePaulo, B. M. (2006) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr1003_2 · Cited by 1,953
The Development and Psychometric Properties of LIWC2015
Pennebaker, J. W., Boyd, R. L., Jordan, K., & Blackburn, K. (2015) · University of Texas at Austin · doi:10.15781/t25p41 · Cited by 1,674
Making Large-Scale Support Vector Machine Learning Practical
Joachims, T. (1998) · The MIT Press eBooks · doi:10.7551/mitpress/1130.003.0015 · Cited by 1,639
Jindal, N., & Liu, B. (2008) · Proceedings of the international conference on Web search and web data mining - WSDM '08 · doi:10.1145/1341531.1341560 · Cited by 1,542
Lying Words: Predicting Deception from Linguistic Styles
Newman, M. L., Pennebaker, J. W., Berry, D. S., & Richards, J. M. (2003) · Personality and Social Psychology Bulletin · doi:10.1177/0146167203029005010 · Cited by 1,489
Support vector machines for spam categorization
Drucker, H., Wu, D., & Vapnik, V. N. (1999) · IEEE Transactions on Neural Networks · doi:10.1109/72.788645 · Cited by 1,476
Detecting Lies and Deceit: Pitfalls and Opportunities
Vrij, A. (2008) · Cited by 1,309
Interpersonal Deception Theory
Buller, D. B., & Burgoon, J. K. (1996) · Communication Theory · doi:10.1111/j.1468-2885.1996.tb00127.x · Cited by 1,212
Combating Web Spam with TrustRank
Gyöngyi, Z., García-Molina, H., & Pedersen, J. (2004) · Elsevier eBooks · doi:10.1016/b978-012088469-8.50052-8 · Cited by 1,087
Using linguistic cues for the automatic recognition of personality in conversation and text
Mairesse, F., Walker, M. A., Mehl, M. R., & Moore, R. K. (2007) · Journal of Artificial Intelligence Research · doi:10.1613/jair.2349 · Cited by 1,046
Detecting product review spammers using rating behaviors
Lim, E., Nguyen, V.-A., Jindal, N., Liu, B., & Lauw, H. W. (2010) · ACM International Conference on Information and Knowledge Management (CIKM) · doi:10.1145/1871437.1871557 · Cited by 720
An Empirical Study of Smoothing Techniques for Language ModelingPreprint
Chen, S. F., & Goodman, J. (1996) · arXiv (Cornell University) · doi:10.48550/arxiv.cmp-lg/9606011 · Cited by 626
Detecting spam web pages through content analysis
Ntoulas, A., Najork, M., Manasse, M. S., & Fetterly, D. (2006) · International World Wide Web Conference (WWW) · doi:10.1145/1135777.1135794 · Cited by 614
On Lying and Being Lied To: A Linguistic Analysis of Deception in Computer-Mediated Communication
Hancock, J. T., Curry, L. E., Goorha, S., & Woodworth, M. (2007) · Discourse Processes · doi:10.1080/01638530701739181 · Cited by 549
Self-citationAutomatically assessing review helpfulness
Kim, S.-M., Pantel, P., Chklovski, T. A., & Marco Pennacchiotti, B. (2006) · Conference on Empirical Methods in Natural Language Processing (EMNLP) · doi:10.3115/1610075.1610135 · Cited by 527
Apples-to-apples in cross-validation studies: pitfalls in classifier performance measurement
Forman, G., & Scholz, M. (2010) · ACM SIGKDD Explorations Newsletter · doi:10.1145/1882471.1882479 · Cited by 368
Mihalcea, R. F., & Strapparava, C. (2009) · Joint Conference of the Annual Meeting of the Association for Computational Linguistics and the International Joint Conference on Natural Language Processing (ACL-IJCNLP) · doi:10.3115/1667583.1667679 · Cited by 340
What Lies Beneath: The Linguistic Traces of Deception in Online Dating Profiles
Toma, C. L., & Hancock, J. T. (2012) · Journal of Communication · doi:10.1111/j.1460-2466.2011.01619.x · Cited by 335
Self-citationOutsmarting the liars: The benefit of asking unanticipated questions.
Vrij, A., Leal, S., Granhag, P. A., Mann, S. A., Fisher, R. P., Hillman, J., & Sperry, K. (2008) · Law and Human Behavior · doi:10.1007/s10979-008-9143-y · Cited by 298
Cues to deception and ability to detect lies as a function of police interview styles.
Vrij, A., Mann, S. A., Kristen, S., & Fisher, R. P. (2007) · Law and Human Behavior · doi:10.1007/s10979-006-9066-4 · Cited by 293
A Comparison of Classification Methods for Predicting Deception in Computer-Mediated Communication
Zhou, L., Burgoon, J. K., Twitchell, D. P., Qin, T., & Nunamaker, J. F. (2004) · Journal of Management Information Systems · doi:10.1080/07421222.2004.11045779 · Cited by 255
How opinions are received by online communities
Danescu-Niculescu-Mizil, C., Kossinets, G., Kleinberg, J. M., & Lee, L. (2009) · International World Wide Web Conference (WWW) · doi:10.1145/1526709.1526729 · Cited by 239
Comparison of Deceptive and Truthful Travel Reviews
Yoo, K., & Ulrike Gretzel, B. (2009) · Springer eBooks · doi:10.1007/978-3-211-93971-0_4 · Cited by 230
Estimating Labels from Label Proportions
Quadrianto, N., Smola, A. J., Caetano, T. S., & Le, Q. V. (2009) · doi:10.5555/1577069.1755865 · Cited by 172
Combining Naive Bayes and n-Gram Language Models for Text Classification
Peng, F., & Schuurmans, D. (2003) · Lecture notes in computer science · doi:10.1007/3-540-36618-0_24 · Cited by 170
Distortion as a validation criterion in the identification of suspicious reviews
Wu, G., Greene, D., Smyth, B., & Cunningham, P. (2010) · doi:10.1145/1964858.1964860 · Cited by 159
Learning to recommend helpful hotel reviews
O’Mahony, M. P., & Smyth, B. (2009) · ACM Conference on Recommender Systems (RecSys) · doi:10.1145/1639714.1639774 · Cited by 150
Modeling Statistical Properties of Written Text
Serrano, M. Á., Flammini, A., & Menczer, F. (2009) · PLoS ONE · doi:10.1371/journal.pone.0005372 · Cited by 129
Grammatical word class variation within the British National Corpus Sampler
Rayson, P., Wilson, A., & Leech, G. (2002) · BRILL eBooks · doi:10.1163/9789004334113_020 · Cited by 104
A Statistical Language Modeling Approach to Online Deception Detection
Zhou, L., Shi, Y., & Zhang, D. (2008) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2007.190624 · Cited by 98
Credibility improves topical blog post retrieval
Weerkamp, W., & de Rijke, M. (2008) · UvA-DARE (University of Amsterdam) · Cited by 97
Automatically assessing the post quality in online discussions on software
Weimer, M., Gurevych, I., & Mühlhäuser, M. (2007) · doi:10.3115/1557769.1557806 · Cited by 96
Amazon Mechanical Turk for Subjectivity Word Sense Disambiguation
Akkaya, C., Conrad, A. P., Wiebe, J., & Mihalcea, R. F. (2010) · University of North Texas Digital Library (University of North Texas) · Cited by 68
An Exploration of Off Topic Conversation
Cade, W., Lehman, B. A., & Olney, A. M. (2010) · Cited by 12
Research works printed but not linked: 2
- [6]6. the North American Chapter of the Association for Computational Linguistics , pages 669-672. Association for Computational Linguistics.
- [17]17. M.K. Johnson and C.L. Raye. 1981. Reality monitoring. Psychological Review , 88(1):67-85.
Cited by
Sorted by citations
Sentiment Analysis and Opinion Mining
Liu, B. (2012) · Synthesis lectures on human language technologies · doi:10.2200/s00416ed1v01y201204hlt016 · Cited by 3,995
Sentiment Analysis and Opinion Mining
Zhang, L., & Liu, B. (2017) · Springer eBooks · doi:10.1007/978-1-4899-7687-1_907 · Cited by 3,307
"Liar, Liar Pants on Fire": A New Benchmark Dataset for Fake News Detection
Wang, W. Y. (2017) · Proceedings of the conference - Association for Computational Linguistics. Meeting · doi:10.18653/v1/p17-2067 · Cited by 2,010
Graph based anomaly detection and description: a survey
Akoglu, L., Tong, H., & Koutra, D. (2015) · Data Mining and Knowledge Discovery · doi:10.1007/s10618-014-0365-y · Cited by 1,504
A survey on opinion mining and sentiment analysis: Tasks, approaches and applications
Ravi, K., & Ravi, V. (2015) · Knowledge-Based Systems · doi:10.1016/j.knosys.2015.06.015 · Cited by 1,376
Metadata
- Authors
- Ott, M., Choi, Y., Cardie, C., & Hancock, J. T.
- Affiliations
- Cornell University (US)
- Venue
- arXiv
- Publisher
- arXiv
- Published
- 2011
- Pages
- 309-319
- ISSN
- 2331-8422 (ISSN-L)
- arXiv
- 1107.4557
- OpenAlex
- W2161283199
- Cited by
- 1,008
- References
- 42
- Citations by year
- 2026: 1 · 2025: 30 · 2024: 57 · 2023: 72 · 2022: 74 · 2021: 106 · 2020: 114 · 2019: 103 · 2018: 89 · 2017: 88 · 2016: 49 · 2015: 68 · 2014: 65 · 2013: 47 · 2012: 43
- Type
- posted content
- Language
- en
- Open access
- green
- arxiv.org
- Locations
- arXiv (Cornell University) (submittedVersion)
- arXiv (Cornell University)
- Funders
- National Science Foundation (0624267, 0904822, 0968450)
- Jack Kent Cooke Foundation
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Physical Sciences › Computer Science › Artificial Intelligence › Sentiment Analysis and Opinion Mining
- Keywords
- opinion spam, opinion spam detection, consumer reviews, fake reviews, creative writing, feature analysis, classifier comparison, psychology of deception
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
