The DOI matches its Crossref record; 93 of the 93 references it cites were found in a registry or an index.
Extrinsic versus Intrinsic Rewards for Contributing Reviews in an Online Platform
Open accessAbstract
Firms have considered various forms of incentives for writing reviews, including the use of extrinsic rewards to attract reviewers. Building on this literature, we study the implications of monetary incentives on online reviews in the context of a natural experiment, where one review platform suddenly began offering monetary incentives for writing reviews. We refer to this as the treated platform. Along with data from Amazon.com and using the difference-in-differences approach, we compare the quantity and quality of reviews before and after rewards were introduced in the treated platform. We find that reviews are significantly more positive but that the quality decreases. Taking advantage of the panel data, we also evaluate the effect of rewards on existing reviewers. We find that their level of participation after monetary incentives decreases but not their quality of participation. Last, even though the platform enjoys an increase in the number of new reviewers, disproportionately more reviews appear to be written for highly rated products. The online appendix is available at https://doi.org/10.1287/isre.2017.0750 .
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Prospect Theory: An Analysis of Decision under Risk
Kahneman, D., & Tversky, A. (1979) · Econometrica · doi:10.2307/1914185 · Cited by 48,524
Deci, E. L., Koestner, R., & Ryan, R. M. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.6.627 · Cited by 7,176 · cited 1×
“Prior research has shown that when behavior is primarily driven by intrinsic motivation, introducing extrinsic rewards could lead to weaker engagement (e.g., Kreps, 1997, Deci et al., 1999).”
BackgroundGriffiths, T. L., & Steyvers, M. (2004) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0307752101 · Cited by 6,049
When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks
LOUGHRAN, T., & MCDONALD, B. (2011) · The Journal of Finance · doi:10.1111/j.1540-6261.2010.01625.x · Cited by 6,015
The Effect of Word of Mouth on Sales: Online Book Reviews
Chevalier, J. A., & Mayzlin, D. (2006) · Journal of Marketing Research · doi:10.1509/jmkr.43.3.345 · Cited by 6,003
Hennig-Thurau, T., Gwinner, K. P., Walsh, G., & Gremler, D. D. (2004) · Journal of Interactive Marketing · doi:10.1002/dir.10073 · Cited by 5,916 · cited 1×
“Hennig-Thurau et al. (2004) surveyed participants on a German web-based customer opinion platform and found that the main factors motivating reviewers are the desire for social interaction and economic incentives, the concern for other consumers, and the opportunity to reach self-actualization.”
BackgroundInformation and Consumer Behavior
Nelson, P. (1970) · Journal of Political Economy · doi:10.1086/259630 · Cited by 5,758
Blei, D. M. (2012) · Communications of the ACM · doi:10.1145/2133806.2133826 · Cited by 5,516
Theory of Industrial Organization.
Rubinstein, A., & Tirole, J. (1989) · Economica · doi:10.2307/2554286 · Cited by 5,142
Incentives and Prosocial Behavior
Bénabou, R., & Tirole, J. (2006) · American Economic Review · doi:10.1257/aer.96.5.1652 · Cited by 3,357
What Makes a Helpful Online Review? A Study of Customer Reviews on Amazon.Com1
Mudambi, S. M., & Schuff, D. (2010) · MIS Quarterly · doi:10.2307/20721420 · Cited by 2,490
Zhu, F., & Zhang, X. (. (2010) · Journal of Marketing · doi:10.1509/jm.74.2.133 · Cited by 2,363
More Than Words: Quantifying Language to Measure Firms' Fundamentals
TETLOCK, P. C., SAAR‐TSECHANSKY, M., & MACSKASSY, S. (2008) · The Journal of Finance · doi:10.1111/j.1540-6261.2008.01362.x · Cited by 2,304
When and Why Incentives (Don't) Work to Modify Behavior
Gneezy, U., Meier, S., & Rey-Biel, P. (2011) · Journal of Economic Perspectives · doi:10.1257/jep.25.4.191 · Cited by 1,735
Ghose, A., & Ipeirotis, P. G. (2011) · IEEE Transactions on Knowledge and Data Engineering · doi:10.1109/tkde.2010.188 · Cited by 1,400
Punished by Rewards: The Trouble with Gold Stars, Incentive Plans, A'S, Praise, and Other Bribes
Kohn, A. (1999) · Cited by 1,356
Word-of-Mouth Processes within a Services Purchase Decision Context
Bansal, H. S., & Voyer, P. A. (2000) · Journal of Service Research · doi:10.1177/109467050032005 · Cited by 1,334
Word of Mouth for Movies: Its Dynamics and Impact on Box Office Revenue
Liu, Y. (2006) · Journal of Marketing · doi:10.1509/jmkg.70.3.074 · Cited by 1,329
Fake It Till You Make It: Reputation, Competition, and Yelp Review Fraud
Luca, M., & Zervas, G. (2016) · Management Science · doi:10.1287/mnsc.2015.2304 · Cited by 1,263
Self-Selection and Information Role of Online Product Reviews
Li, X., & Hitt, L. M. (2008) · Information Systems Research · doi:10.1287/isre.1070.0154 · Cited by 1,125
Babić Rosario, A., Sotgiu, F., De Valck, K., & Bijmolt, T. H. A. (2016) · Journal of Marketing Research · doi:10.1509/jmr.14.0380 · Cited by 1,073
topicmodels : An R Package for Fitting Topic Models
Grün, B., & Hornik, K. (2011) · Journal of Statistical Software · doi:10.18637/jss.v040.i13 · Cited by 1,072
Deriving the Pricing Power of Product Features by Mining Consumer Reviews
Archak, N., Ghose, A., & Ipeirotis, P. G. (2011) · Management Science · doi:10.1287/mnsc.1110.1370 · Cited by 1,007
Promotional Reviews: An Empirical Investigation of Online Review Manipulation
Mayzlin, D., Dover, Y., & Chevalier, J. (2014) · American Economic Review · doi:10.1257/aer.104.8.2421 · Cited by 937
Pavlou, P. A., & Dimoka, A. (2006) · Information Systems Research · doi:10.1287/isre.1060.0106 · Cited by 930
Chintagunta, P. K., Gopinath, S., & Venkataraman, S. (2010) · Marketing Science · doi:10.1287/mksc.1100.0572 · Cited by 920
A density-based method for adaptive LDA model selection
Cao, J., Xia, T., Li, J., Zhang, Y., & Tang, S. (2009) · Neurocomputing · doi:10.1016/j.neucom.2008.06.011 · Cited by 880
Brynjolfsson, E., Hu, Y. (., & Simester, D. (2011) · Management Science · doi:10.1287/mnsc.1110.1371 · Cited by 847
Park, D.-H., & Kim, S. (2008) · Electronic Commerce Research and Applications · doi:10.1016/j.elerap.2007.12.001 · Cited by 837
Availability Cascades and Risk Regulation
Sunstein, C. R., & Kuran, T. (1999) · eYLS (Yale Law School) · Cited by 755
On Finding the Natural Number of Topics with Latent Dirichlet Allocation: Some Observations
Arun, R., Suresh, V., Veni Madhavan, C. E., & Narasimha Murthy, M. N. (2010) · Lecture Notes in Computer Science · doi:10.1007/978-3-642-13657-3_43 · Cited by 719
Permutation, Parametric and Bootstrap Tests of Hypotheses
(2005) · Springer Series in Statistics · doi:10.1007/b138696 · Cited by 707
Natural Experiments in the Social Sciences: A Design-Based Approach
Dunning, T. (2012) · Cambridge University Press eBooks · doi:10.1017/cbo9781139084444 · Cited by 705
Erat, S., & Gneezy, U. (2012) · Management Science · doi:10.1287/mnsc.1110.1449 · Cited by 698
Overcoming the J-shaped distribution of product reviews
Hu, N., Zhang, J., & Pavlou, P. A. (2009) · Communications of the ACM · doi:10.1145/1562764.1562800 · Cited by 686
Crowding Out in Blood Donation: Was Titmuss Right?
Mellström, C., & Johannesson, M. (2008) · Journal of the European Economic Association · doi:10.1162/jeea.2008.6.4.845 · Cited by 684
The Value of Social Dynamics in Online Product Ratings Forums
Moe, W. W., & Trusov, M. (2011) · Journal of Marketing Research · doi:10.1509/jmkr.48.3.444 · Cited by 672
How Effective Are Electronic Reputation Mechanisms? An Experimental Investigation
Bolton, G. E., Katok, E., & Ockenfels, A. (2004) · Management Science · doi:10.1287/mnsc.1030.0199 · Cited by 667
Reviews, Reputation, and Revenue: The Case of Yelp.Com
Luca, M. (2011) · SSRN Electronic Journal · doi:10.2139/ssrn.1928601 · Cited by 627
A General Definition of the Lorenz Curve
Gastwirth, J. L. (1971) · Econometrica · doi:10.2307/1909675 · Cited by 618
Online Product Opinions: Incidence, Evaluation, and Evolution
Moe, W. W., & Schweidel, D. A. (2012) · Marketing Science · doi:10.1287/mksc.1110.0662 · Cited by 616
Born Unequal: A Study of the Helpfulness of User-Generated Product Reviews
Pan, Y., & Zhang, J. Q. (2011) · Journal of Retailing · doi:10.1016/j.jretai.2011.05.002 · Cited by 610
Accurate and effective latent concept modeling for ad hoc information retrieval
Deveaud, R., SanJuan, E., & Bellot, P. (2014) · Document numérique · doi:10.3166/dn.17.1.61-84 · Cited by 609
Rishika, R., Kumar, A., Janakiraman, R., & Bezawada, R. (2013) · Information Systems Research · doi:10.1287/isre.1120.0460 · Cited by 547
Social Comparisons and Contributions to Online Communities: A Field Experiment on MovieLens
Chen, Y., Harper, F. M., Konstan, J., & Li, S. X. (2010) · American Economic Review · doi:10.1257/aer.100.4.1358 · Cited by 495
Sequential and Temporal Dynamics of Online Opinion
Godes, D., & Silva, J. C. (2012) · Marketing Science · doi:10.1287/mksc.1110.0653 · Cited by 441
Anderson, M., & Magruder, J. (2012) · The Economic Journal · doi:10.1111/j.1468-0297.2012.02512.x · Cited by 435
The Effect of Need for Uniqueness on Word of Mouth
Cheema, A., & Kaikati, A. M. (2010) · Journal of Marketing Research · doi:10.1509/jmkr.47.3.553 · Cited by 428
Permutation, Parametric, and Bootstrap Tests of Hypotheses
(2005) · Technometrics · doi:10.1198/tech.2005.s333 · Cited by 412
The Gift Relationship: From Human Blood to Social Policy
Titmuss, R. (2018) · Policy Press eBooks · doi:10.1332/policypress/9781447349570.001.0001 · Cited by 406
“Popularity Effect” in User-Generated Content: Evidence from Online Product Reviews
Goes, P. B., Lin, M., & Au Yeung, C.-M. (2014) · Information Systems Research · doi:10.1287/isre.2013.0512 · Cited by 405
The Fog Index After Twenty Years
Gunning, R. (1969) · Journal of Business Communication · doi:10.1177/002194366900600202 · Cited by 389
Do I Follow My Friends or the Crowd? Information Cascades in Online Movie Ratings
Lee, Y.-J., Hosanagar, K., & Tan, Y. (2015) · Management Science · doi:10.1287/mnsc.2014.2082 · Cited by 377
Third-Party Product Review and Firm Marketing Strategy
Chen, Y., & Xie, J. (2005) · Marketing Science · doi:10.1287/mksc.1040.0089 · Cited by 375
Yin, D., Mitra, S., & Zhang, H. (2016) · Information Systems Research · doi:10.1287/isre.2015.0617 · Cited by 365
topicmodels: An R Package for Fitting Topic Models
Grün, B., Kepler, J., Linz, U., Hornik, K., & Wien, W. W. (2011) · Cited by 345
Upstream reciprocity and the evolution of gratitude
Nowak, M. A., & Roch, S. (2007) · Proceedings of the Royal Society B: Biological Sciences · doi:10.1098/rspb.2006.0125 · Cited by 339
Nair, H. (2007) · Quantitative Marketing and Economics · doi:10.1007/s11129-007-9026-4 · Cited by 331
Collaborative prediction and ranking with non-random missing data
Marlin, B. M., & Zemel, R. S. (2009) · Proceedings of the third ACM conference on Recommender systems · doi:10.1145/1639714.1639717 · Cited by 315
Are Consumers More Likely to Contribute Online Reviews for Hit or Niche Products?
Dellarocas, C., Gao, G., & Narayan, R. (2010) · Journal of Management Information Systems · doi:10.2753/mis0742-1222270204 · Cited by 311
Two-sided advertising: A meta-analysis
Eisend, M. (2006) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2005.11.001 · Cited by 305 · cited 1×
“In contrast, Eisend (2006) argued that moderate reviews enhance product credibility and brand favorability by providing both pros and cons of the product.”
BackgroundVocal Minority and Silent Majority: How Do Online Ratings Reflect Population Perceptions of Quality?
Gao, G. (., Greenwood, B. N., Agarwal, R., & McCullough, P. (2015) · SSRN Electronic Journal · doi:10.2139/ssrn.2629837 · Cited by 278
Size matters: word count as a measure of quality on wikipedia
Blumenstock, J. E. (2008) · Proceedings of the 17th international conference on World Wide Web · doi:10.1145/1367497.1367673 · Cited by 270
Hosanagar, K., Fleder, D., Lee, D., & Buja, A. (2014) · Management Science · doi:10.1287/mnsc.2013.1808 · Cited by 264
Schlosser, A. E. (2011) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2011.04.002 · Cited by 264
All Reviews are Not Created Equal: The Disaggregate Impact of Reviews and Reviewers at Amazon.Com
Chen, P.-Y., Dhanasobhon, S., & Smith, M. D. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.918083 · Cited by 240
Crowdsourcing with All-Pay Auctions: A Field Experiment on Taskcn
Liu, T. X., Yang, J., Adamic, L. A., & Chen, Y. (2014) · Management Science · doi:10.1287/mnsc.2013.1845 · Cited by 238
Comparison of Deceptive and Truthful Travel Reviews
Yoo, K.-H., & Gretzel, U. (2009) · Information and Communication Technologies in Tourism 2009 · doi:10.1007/978-3-211-93971-0_4 · Cited by 219
Competing with Free: The Impact of Movie Broadcasts on DVD Sales and Internet Piracy1
Smith, M. D., & Telang, R. (2009) · MIS Quarterly · doi:10.2307/20650294 · Cited by 219
The Impact of Fake Reviews on Online Visibility: A Vulnerability Assessment of the Hotel Industry
Lappas, T., Sabnis, G., & Valkanas, G. (2016) · Information Systems Research · doi:10.1287/isre.2016.0674 · Cited by 218
Grewal, R., Cline, T. W., & Davies, A. (2003) · Journal of Consumer Psychology · doi:10.1207/no_doi · Cited by 208
Promotional Reviews: An Empirical Investigation of Online Review Manipulation
Mayzlin, D., Dover, Y., & Chevalier, J. A. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2128860 · Cited by 201
A Structural Model of Employee Behavioral Dynamics in Enterprise Social Media
Huang, Y. H., Singh, P. V., & Ghose, A. (2015) · Management Science · doi:10.1287/mnsc.2014.2125 · Cited by 180
Internet’s Dirty Secret: Assessing the Impact of Online Intermediaries on HIV Transmission1
Chan, J., & Ghose, A. (2014) · MIS Quarterly · doi:10.25300/misq/2014/38.4.01 · Cited by 178
Collaborative Filtering and the Missing at Random Assumption
Marlin, B. M., Zemel, R. S., Roweis, S. T., & Slaney, M. (2012) · arXiv (Cornell University) · doi:10.48550/arxiv.1206.5267 · Cited by 178
Paying It Forward vs. Rewarding Reputation: Mechanisms of Generalized Reciprocity
Baker, W. E., & Bulkley, N. (2014) · Organization Science · doi:10.1287/orsc.2014.0920 · Cited by 178
A Statistical Measure of a Population’s Propensity to Engage in Post-Purchase Online Word-of-Mouth
Dellarocas, C., & Narayan, R. (2006) · Statistical Science · doi:10.1214/088342306000000169 · Cited by 173
Competing for Attention: An Empirical Study of Online Reviewers’ Strategic Behavior1
Shen, W., Hu, Y. J., & Ulmer, J. R. (2015) · MIS Quarterly · doi:10.25300/misq/2015/39.3.08 · Cited by 167
Toward a Better Measure of Business Proximity: Topic Modeling for Industry Intelligence1
Shi, Z., Lee, G. M., & Whinston, A. B. (2016) · MIS Quarterly · doi:10.25300/misq/2016/40.4.11 · Cited by 165
Jiang, L., Bazarova, N. N., & Hancock, J. T. (2011) · Communication Research · doi:10.1177/0093650211405313 · Cited by 160
May, R. M. (1987) · Nature · doi:10.1038/327015a0 · Cited by 151
How to Attract and Retain Readers in Enterprise Blogging?
Singh, P. V., Sahoo, N., & Mukhopadhyay, T. (2014) · Information Systems Research · doi:10.1287/isre.2013.0509 · Cited by 134
Indirect reciprocity in cyclical networks
Greiner, B., & Levati, M. V. (2005) · Journal of Economic Psychology · doi:10.1016/j.joep.2004.04.003 · Cited by 133
Lu, Y., Jerath, K., & Singh, P. V. (2013) · Management Science · doi:10.1287/mnsc.1120.1685 · Cited by 126
Knowledge Market Design: A Field Experiment at Google Answers
CHEN, Y., Ho, T., & Kim, Y. (2010) · Journal of Public Economic Theory · doi:10.1111/j.1467-9779.2010.01468.x · Cited by 112
Competing with Free: The Impact of Movie Broadcasts on DVD Sales and Internet PiracyPreprint
Smith, M. D., & Telang, R. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1028306 · Cited by 76
Hsieh, G., Kraut, R. E., & Hudson, S. E. (2010) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/1753326.1753373 · Cited by 73
Thanks, but I'm used to better: A relative rank model of gratitude.
Wood, A. M., Brown, G. D. A., & Maltby, J. (2011) · Emotion · doi:10.1037/a0021553 · Cited by 73
Impact of Recommender Systems on Sales Volume and Diversity
Lee, D., & Hosanagar, K. (2014) · Cited by 50
Bonus, Disclosure, and Choice: What Motivates the Creation of High-Quality Paid Reviews?
Wang, J., Ghose, A., & Ipeirotis, P. G. (2012) · Cited by 49
Quantifying the impact of badges on user engagement in online Q&A communities
Li, Z., Huang, K.-W., & Cavusoglu, H. (2012) · National University of Singapore · Cited by 42
Stephen, A. T., Bart, Y., du Plessis, C., & Gonçalves, D. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2125526 · Cited by 23
Li, H., Zhang, Z., Janakiraman, R., & Meng, F. (2016) · Scholarworks (University of Massachusetts Amherst) · Cited by 8
Research works printed but not linked: 8
- Anderson M, 2012, Econom. J., 122, 957
- 10.25300/MISQ/2015/39.3.03 · doi:10.25300/misq/2015/39.3.03 · Paper page
- 10.1207/S15327663JCP1303_01 · doi:10.1207/s15327663jcp1303_01 · Paper page
- 10.1007/978-3-540-87781-3_46 · DOI
- Kreps DM, 1997, Amer. Econom. Rev., 87, 359
- 10.2307/1229439 · DOI
- Titmuss RM, 1970, The Gift Relationship
- Wooldridge JM, 2010, Econometric Analysis of Cross Section and Panel Data
Cited by
Leung, F. F., Gu, F. F., & Palmatier, R. W. (2022) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-021-00829-4 · Cited by 580
Stimulating Online Reviews by Combining Financial Incentives and Social Norms
Burtch, G., Hong, Y., Bapna, R., & Griskevicius, V. (2018) · Management Science · doi:10.1287/mnsc.2016.2715 · Cited by 405
Does Money Talk? The Impact of Monetary Incentives on User-Generated Content Contributions
Liu, Y., & Feng, J. (2021) · Information Systems Research · doi:10.1287/isre.2020.0971 · Cited by 103
Monetary Incentive and Stock Opinions on Social Media
Chen, H., Hu, Y. J., & Huang, S. (2019) · Journal of Management Information Systems · doi:10.1080/07421222.2019.1598686 · Cited by 76
Your Hometown Matters: Popularity-Difference Bias in Online Reputation Platforms
Kokkodis, M., & Lappas, T. (2020) · Information Systems Research · doi:10.1287/isre.2019.0895 · Cited by 48
Metadata
- Authors
- Khern-am-nuai, W., Kannan, K., & Ghasemkhani, H.
- Venue
- Information Systems Research
- Published
- 2018-12
- Volume
- 29
- Issue
- 4
- Pages
- 871-892
- ISSN
- 1047-7047 (ISSN-L)
- 1526-5536
- OpenAlex
- W2900211783
- Cited by
- 260
- References
- 93
- FWCI
- 33.60
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.