The Turing test of online reviews: Can we tell the difference between human-written and GPT-4-written online reviews?
The DOI matches its Crossref record; 38 of the 38 references it cites were found in a registry or an index.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Although a large body of studies have demonstrated that people struggle to discriminate between human and machine generated texts when passively reading them [25,30,40]”
From People cannot distinguish GPT-4 from a human in a Turing test (2025) · cites this paper as [25]
Citations
Cites
I.—COMPUTING MACHINERY AND INTELLIGENCE
TURING, A. M. (1950) · Mind · doi:10.1093/mind/lix.236.433 · Cited by 10,290 · cited 1×
“In this paper, we investigate whether the text created by the currently (March 2024) cutting-edge LLM, GPT-4, can generate restaurant reviews that pass the Turing test (Turing, 1950), which is defined as readers being unable to distinguish whether the reviews were written by a human or an AI.”
The Effect of Word of Mouth on Sales: Online Book Reviews
Chevalier, J. A., & Mayzlin, D. (2006) · Journal of Marketing Research · doi:10.1509/jmkr.43.3.345 · Cited by 6,003 · cited 2×
“Both components can convey a message, and scholars have identified how ratings (Chevalier & Mayzlin, 2006; Dellarocas et al., 2007) and the review text (Archak et al., 2011; Netzer et al., 2012; Pavlou & Dimoka, 2006) can help purchasing decision of future users.”
BackgroundThe Field behind the Screen: Using Netnography for Marketing Research in Online Communities
Kozinets, R. V. (2002) · Journal of Marketing Research · doi:10.1509/jmkr.39.1.61.18935 · Cited by 3,962 · cited 2×
“These involve human moderators and community reporting for suspicious reviews (Cheung & Lee, 2012; Kozinets, 2002).”
BackgroundThe Digitization of Word of Mouth: Promise and Challenges of Online Feedback Mechanisms
Dellarocas, C. (2003) · Management Science · doi:10.1287/mnsc.49.10.1407.17308 · Cited by 3,433
Building Effective Online Marketplaces with Institution-Based Trust
Pavlou, P. A., & Gefen, D. (2004) · Information Systems Research · doi:10.1287/isre.1040.0015 · Cited by 2,516 · cited 2×
“Of course, platforms have been putting significant efforts into fighting fake 1 reviews (Han et al., 2022; He et al., 2022; Luca & Zervas, 2016; Pavlou & Gefen, 2004), but many of these meth-ods have relied on the ability of humans and algorithms to recognize fake reviews from the review text.”
BackgroundWhat Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.Com1
Mudambi, S. M., & Schuff, D. (2010) · MIS Quarterly · doi:10.2307/20721420 · Cited by 2,490 · cited 2×
“These include text analysis tools targeting red flags like unnatural language or extreme sentiment (Han et al., 2022) and filters that consider feedback specificity, word use, or tone (Mudambi & Schuff, 2010).”
BackgroundExploring the value of online product reviews in forecasting sales: The case of motion pictures
Dellarocas, C., Zhang, X. (., & Awad, N. F. (2007) · Journal of Interactive Marketing · doi:10.1002/dir.20087 · Cited by 1,518 · cited 1×
“Both components can convey a message, and scholars have identified how ratings (Chevalier & Mayzlin, 2006; Dellarocas et al., 2007) and the review text (Archak et al., 2011; Netzer et al., 2012; Pavlou & Dimoka, 2006) can help purchasing decision of future users.”
BackgroundManagement Information Systems, Managing Digital Firm,
Laudon, K. C., & Laudon, J. P. (1988) · University of Maribor digital library (University of Maribor) · Cited by 1,411
Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud
Luca, M., & Zervas, G. (2016) · Management Science · doi:10.1287/mnsc.2015.2304 · Cited by 1,263 · cited 5×
“Of course, platforms have been putting significant efforts into fighting fake 1 reviews (Han et al., 2022; He et al., 2022; Luca & Zervas, 2016; Pavlou & Gefen, 2004), but many of these meth-ods have relied on the ability of humans and algorithms to recognize fake reviews from the review text.”
BackgroundInfluentialSelf-Selection and Information Role of Online Product Reviews
Li, X., & Hitt, L. M. (2008) · Information Systems Research · doi:10.1287/isre.1070.0154 · Cited by 1,125 · cited 1×
“It is now well-established that online reviews influence buying decisions and thus drive sales and profits (e.g., Archak et al., 2011; Chevalier & Mayzlin, 2006; Li & Hitt, 2008; Netzer et al., 2012; Pavlou & Dimoka, 2006) (for literature reviews, see Sharkey et al. (2023); Tadelis (2016)).”
Background
Research works printed but not linked: 1
- Han, J., Pei, J., & Tong, H. (2022). Data mining: Concepts and techniques. Morgan Kaufmann.
Cited by
Do LLMs write like humans? Variation in grammatical and rhetorical styles
Reinhart, A., Markey, B., Laudenbach, M., Pantusen, K., Yurko, R., Weinberg, G., & Brown, D. W. (2025) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2422455122 · Cited by 107
“Each human text was split into two chunks of roughly 500 words; the first chunk was used to prompt an LLM to create text that was compared to the second human chunk.”
AI language model rivals expert ethicist in perceived moral expertise
Dillion, D., Mondal, D., Tandon, N., & Gray, K. (2025) · Scientific Reports · doi:10.1038/s41598-025-86510-0 · Cited by 47
Li, H., Xi, J., Hsu, C. H. C., Yu, B. X. B., & Zheng, X. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105179 · Cited by 101
Yu-peng, L., & Yu, Z. (2024) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2024.2359222 · Cited by 102
Unpacking the impact of AI vs. human-generated review summary on hotel booking intentions
Jia, S., Chi, O. H., & Chi, C. G. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2024.104030 · Cited by 82
Metadata
- Authors
- Kovács, B.
- Affiliations
- Yale University (US)
- Corresponding author
- Kovács, Balázs
- ORCID
- Kovács, Balázs
- Venue
- Marketing Letters
- Published
- 2024-04-12
- Volume
- 35
- Issue
- 4
- Pages
- 651-666
- ISSN
- 0923-0645 (ISSN-L)
- 1573-059X
- OpenAlex
- W4394764084
- Cited by
- 44
- References
- 38
- Citations by year
- 2026: 14 · 2025: 20 · 2024: 10
- FWCI
- 16.59
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.springernature.com/gp/researchers/text-and-data-mining (tdm)
- www.springernature.com/gp/researchers/text-and-data-mining (vor)
- Locations
- Marketing Letters (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- Topics
- Physical Sciences › Computer Science › Information Systems › Spam and Phishing Detection
- Social Sciences › Business, Management and Accounting › Marketing › Digital Marketing and Social Media
- Social Sciences › Sociology and Political Science › Misinformation and Its Impacts
- Keywords
- online reviews, AI-generated text, GPT-4, large language models, generative AI, AI detection, trust erosion, consumer choice, online review platforms, AI-generated reviews
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The Turing test of online reviews: Can we tell the difference between human-written and GPT-4-written online reviews? https://app.cochat.ai/papers/10.1007/s11002-024-09729-3
- The paper
- Kovács, B. (2024). The Turing test of online reviews: Can we tell the difference between human-written and GPT-4-written online reviews? Marketing Letters. https://doi.org/10.1007/s11002-024-09729-3
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.