Economic Games on the Internet: The Effect of $1 Stakes
Open accessThe DOI matches its Crossref record; 39 of the 39 references it cites were found in a registry or an index.
Abstract
Online labor markets such as Amazon Mechanical Turk (MTurk) offer an unprecedented opportunity to run economic game experiments quickly and inexpensively. Using Mturk, we recruited 756 subjects and examined their behavior in four canonical economic games, with two payoff conditions each: a stakes condition, in which subjects' earnings were based on the outcome of the game (maximum earnings of $1); and a no-stakes condition, in which subjects' earnings are unaffected by the outcome of the game. Our results demonstrate that economic game experiments run on MTurk are comparable to those run in laboratory settings, even when using very low stakes.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“While decision-making experiments on MTurk allow high control over confounding variables and are comparable to those in laboratory settings, even in low-stakes scenarios (Amir et al., 2012)”
From Exploring the effects of human-centered AI explanations on trust and reliance (2023) · cites this paper as Amir et al., 2012
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
An experimental analysis of ultimatum bargaining
Güth, W., Schmittberger, R., & Schwarze, B. (1982) · Journal of Economic Behavior & Organization · doi:10.1016/0167-2681(82)90011-7 · Cited by 4,816 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
MethodRunning experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767 · cited 1×
“Recent work has replicated classical findings such as framing and priming on MTurk [8–10], found a high level of test-retest reliability on Mturk [10–12], and shown quantitative agreement in behavior between MTurk and the physical laboratory [6, 8].”
ResultConducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Fairness and the assumptions of economics
Kahneman, D., Knetsch, J. L., & Thaler, R. H. (1986) · The Journal of Business · doi:10.1086/296367 · Cited by 2,463 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
MethodFairness in Simple Bargaining Experiments
Forsythe, R., Horowitz, J. L., Savin, N. E., & Sefton, M. (1994) · Games and Economic Behavior · doi:10.1006/game.1994.1021 · Cited by 2,366 · cited 2×
“Prior work on the dictator game found that subjects became significantly less generous when going from no stakes to low stakes [13], but that going from low stakes to high stakes did not affect donations [13,14].”
BackgroundResultEngel, C. (2011) · Experimental Economics · doi:10.1007/s10683-011-9283-7 · Cited by 1,783 · cited 1×
“4% aggregated over more than 616 DG treatments as reviewed in a recent metaanalysis [21].”
BackgroundAndreoni, J. (1988) · Journal of Public Economics · doi:10.1016/0047-2727(88)90043-6 · Cited by 1,118 · cited 1×
“In the absence of a meta-analysis that breaks contributions down by MPCR, we compare the average contribution level in our experiment to the range of average contributions observed in various previous studies using the same MPCR [31–36].”
ResultCooperation in public-goods experiments: kindness or confusion?
Andreoni, J. (2002) · Market Failure or Success · doi:10.4337/1843760258.00027 · Cited by 912
Andreoni, J. (1995) · The Quarterly Journal of Economics · doi:10.2307/2118508 · Cited by 912 · cited 1×
“experiment to the range of average contributions observed in various previous studies using the same MPCR [31–36].”
MethodOxytocin increases generosity in humans
Zak, P. J., Stanton, A. A., & Ahmadi, S. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001128 · Cited by 831 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
Cultural Differences in Ultimatum Game Experiments: Evidence from a Meta-Analysis
Oosterbeek, H., Sloof, R., & Van De Kuilen, G. (2004) · Experimental Economics · doi:10.1023/b:exec.0000026978.14316.74 · Cited by 828 · cited 1×
“Considering the UG, it has been shown that using the strategy method significantly affects behavior [22], and to our knowledge no meta-analyses exist which focus on strategy-method UGs.”
MethodRand, D. G. (2012) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2011.03.004 · Cited by 683 · cited 1×
“Recent work has replicated classical findings such as framing and priming on MTurk [8– 10], found a high level of test-retest reliability on Mturk [10–12], and shown quantitative agreement in behavior between MTurk and the physical laboratory [6,8].”
ResultSelf-citationDynamic social networks promote cooperation in experiments with humans
Rand, D. G., Arbesman, S., & Christakis, N. A. (2011) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1108243108 · Cited by 653 · cited 1×
“MTurk is becoming increasingly popular as a platform for conducting experiments across the social sciences [1–7].”
BackgroundSelf-citationRAISING THE STAKES IN THE ULTIMATUM GAME: EXPERIMENTAL EVIDENCE FROM INDONESIA
Cameron, L. A. (1999) · Economic Inquiry · doi:10.1111/j.1465-7295.1999.tb01415.x · Cited by 552 · cited 1×
“Prior work on the ultimatum game has found that adding stakes does not affect the average proposal but may increase the variance in proposals [13,15], while the results for responder behavior are more mixed, with one study finding no effect [13] and another finding a significant decrease in rejection rates [15].”
ResultGender differences in the ultimatum game
Solnick, S. (2001) · Economic Inquiry · doi:10.1093/ei/39.2.189 · Cited by 420 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
MethodBreaking monotony with meaning: Motivation in crowdsourcing markets
Chandler, D., & Kapelner, A. (2013) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2013.03.003 · Cited by 398 · cited 1×
“MTurk is becoming increasingly popular as a platform for conducting experiments across the social sciences [1–7].”
BackgroundThe Online Laboratory: Conducting Experiments in a Real Labor Market
Horton, J., Rand, D., & Zeckhauser, R. (2010) · National Bureau of Economic Research · doi:10.3386/w15961 · Cited by 388
Self-citationOn expectations and the monetary stakes in ultimatum games
Hoffman, E., McCabe, K. A., & Smith, V. L. (1996) · International Journal of Game Theory · doi:10.1007/bf02425259 · Cited by 356
Cooperation and Contagion in Web-Based, Networked Public Goods Experiments
Suri, S., & Watts, D. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016836 · Cited by 354
The evolution of antisocial punishment in optional public goods games
Rand, D. G., & Nowak, M. A. (2011) · Nature Communications · doi:10.1038/ncomms1442 · Cited by 350 · cited 1×
“MTurk is becoming increasingly popular as a platform for conducting experiments across the social sciences [1–7].”
BackgroundSelf-citationCooperation and contagion in web-based, networked public goods experiments
Suri, S., & Watts, D. J. (2011) · ACM SIGecom Exchanges · doi:10.1145/1998549.1998550 · Cited by 318 · cited 1×
“Recent work has replicated classical findings such as framing and priming on MTurk [8– 10], found a high level of test-retest reliability on Mturk [10–12], and shown quantitative agreement in behavior between MTurk and the physical laboratory [6,8].”
ResultStraub, P. G., & Murnighan, J. K. (1995) · Journal of Economic Behavior & Organization · doi:10.1016/0167-2681(94)00072-m · Cited by 271 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
MethodIndividual behaviour in a free riding experiment
Weimann, J. (1994) · Journal of Public Economics · doi:10.1016/0047-2727(94)90059-0 · Cited by 268 · cited 1×
“experiment to the range of average contributions observed in various previous studies using the same MPCR [31–36].”
MethodStakes Matter in Ultimatum Games
Andersen, S., Ertaç, S., Gneezy, U., Hoffman, M., & List, J. A. (2011) · American Economic Review · doi:10.1257/aer.101.7.3427 · Cited by 228 · cited 1×
“It has also been found that increasing from low to high stakes has little effect on either proposals or rejection rates, unless the stakes are extremely large [13–17].”
BackgroundLooking Like a Winner: Candidate Appearance and Electoral Success in New Democracies
Lawson, C., Lenz, G. S., Baker, A., & Myers, M. (2010) · World Politics · doi:10.1017/s0043887110000195 · Cited by 210 · cited 1×
“MTurk is becoming increasingly popular as a platform for conducting experiments across the social sciences [1–7].”
BackgroundHeritability of ultimatum game responder behavior
Wallace, B., Cesarini, D., Lichtenstein, P., & Johannesson, M. (2007) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.0706642104 · Cited by 209 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
MethodThe effect of stakes in distribution experiments
Carpenter, J., Verhoogen, E., & Burks, S. (2005) · Economics Letters · doi:10.1016/j.econlet.2004.08.007 · Cited by 206 · cited 1×
“Prior work on the dictator game found that subjects became significantly less generous when going from no stakes to low stakes [13], but that going from low stakes to high stakes did not affect donations [13,14].”
BackgroundDictator Games: A Meta StudyPreprint
Engel, C. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1568732 · Cited by 171
The claiming effect: Why players are more generous in social dilemmas than in ultimatum games.
Larrick, R. P., & Blount, S. (1997) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.72.4.810 · Cited by 161 · cited 1×
“Therefore, we examine behavior in various previous UG experiments that used the strategy method [23–29], and compare the range of outcomes to what we observe in our data.”
MethodDoes stake size matter in trust games
Johansson-Stenman, O., Mahmud, M., & Martinsson, P. (2005) · Economics Letters · doi:10.1016/j.econlet.2005.03.007 · Cited by 143 · cited 1×
“Considering the increase of stake size, Kocher, Martinsson and Visser [18] found no significant difference in subjects’ contributions in the public goods game when going from low to high stakes, and Johansson-Stenman, Mahmud and Martinsson [19] found that in the trust game, the amount sent by investors decreased when…”
BackgroundCultural Differences in Ultimatum Game Experiments: Evidence from a Meta-AnalysisPreprint
Oosterbeek, H., Sloof, R., & van de Kuilen, G. (2001) · SSRN Electronic Journal · doi:10.2139/ssrn.286428 · Cited by 124
Confusion or fairness in the field? Rejections in the ultimatum game under the strategy method
Bahry, D., & Wilson, R. K. (2005) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2004.07.005 · Cited by 107 · cited 1×
“As in the physical lab, some subjects were ‘inconsistent’ in that they were willing to accept some of the lower offers, but rejected higher offers (that is, they did not have a threshold for acceptance) [20].”
BackgroundDoes Stake Size Matter for Cooperation and Punishment
Kocher, M. G., Martinsson, P., & Visser, M. (2008) · Economics Letters · doi:10.1016/j.econlet.2007.09.048 · Cited by 36 · cited 1×
“Considering the increase of stake size, Kocher, Martinsson and Visser [18] found no significant difference in subjects’ contributions in the public goods game when going from low to high stakes, and Johansson-Stenman, Mahmud and Martinsson [19] found that in the trust game, the amount sent by investors decreased when…”
BackgroundCultures of Kindness: A Meta-Analysis of Trust Game ExperimentsPreprint
Johnson, N. D., & Mislin, A. A. (2008) · SSRN Electronic Journal · doi:10.2139/ssrn.1315325 · Cited by 30
Beauty Queens and Battling Knights: Risk Taking and Attractiveness in ChessPreprint
Dreber, A., Gerdes, C., & Gränsmark, P. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1697533 · Cited by 16
The Causes of Peer Effects in Production: Evidence from a Series of Field ExperimentsPreprint
Horton, J. J., & Zeckhauser, R. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1652993 · Cited by 14
Partners contribute more to Public goods than Strangers.
Keser, C., & van Winden, F. A. A. M. (1997) · UvA-DARE (University of Amsterdam) · Cited by 4
Beauty Queens and Battling Knights: Risk Taking and Attractiveness in ChessPreprint
Dreber, A., Gerdes, C., & Gränsmark, P. (2010) · SSRN Electronic Journal · doi:10.2139/ssrn.1708760 · Cited by 2
Research works printed but not linked: 3
- J Horton, 2010, Employer expectations, peer effects and productivity: Evidence from a series of field experiments., SSRN eLibrary
- W Mason, 2010, Conducting behavioral research on amazons mechanical turk., Behavior Research Methods, 1
- J Brandts, 1996, Cooperative gains or noise in public goods experiments., Tinbergen Institute
Cited by
Prolific.ac—A subject pool for online experiments
Palan, S., & Schitter, C. (2018) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2017.12.004 · Cited by 3,450
Inside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
“In addition, the service has been validated as a tool for conducting survey research [3,4], one-shot decision-making research [5,6], collective behavior experiments [7,8], for norming stimuli, and conducting behavioral linguistics experiments [9,10].”
MethodStatistical physics of human cooperation
Perc, M., Jordan, J. J., Rand, D. G., Wang, Z., Boccaletti, S., & Szolnoki, A. (2017) · Physics Reports · doi:10.1016/j.physrep.2017.05.004 · Cited by 1,590
“Moreover, evidence demonstrates that, across a wide range of economic games, game play is very similar on Amazon Mechanical Turk and in the physical laboratory — even though stakes are generally an order of magnitude higher in the lab [136,137].”
BackgroundSelf-citationSpontaneous giving and calculated greed
Rand, D. G., Greene, J. D., & Nowak, M. A. (2012) · Nature · doi:10.1038/nature11467 · Cited by 1,334
Self-citation
Metadata
- Authors
- Amir, O., Rand, D. G., & Gal, Y. K.
- Affiliations
- Ben-Gurion University of the Negev (IL)
- Harvard University (US)
- Corresponding author
- Amir, Ofra
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2012-02-21
- Volume
- 7
- Issue
- 2
- Pages
- e31461
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3283743
- OpenAlex
- W3124416153
- Cited by
- 364
- References
- 39
- Citations by year
- 2026: 12 · 2025: 12 · 2024: 18 · 2023: 24 · 2022: 23 · 2021: 22 · 2020: 24 · 2019: 31 · 2018: 33 · 2017: 40 · 2016: 28 · 2015: 31 · 2014: 34 · 2013: 20 · 2012: 9
- FWCI
- 53.45
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- SSRN Electronic Journal (publishedVersion)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Digital Access to Scholarship at Harvard (DASH) (Harvard University) (submittedVersion, cc-by)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Edinburgh Research Explorer (University of Edinburgh) (submittedVersion, cc-by)
- Funders
- John Templeton Foundation
- Harvard University
- European Commission (268362)
- Topics
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Experimental Behavioral Economics Studies
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Decision Sciences › General Decision Sciences › Decision-Making and Behavioral Economics
- Keywords
- Amazon Mechanical Turk, digital labour markets, financial incentives, experimental design, participant behavior
- MeSH terms
- Behavior, Economics, Games, Experimental, Humans, Internet
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
