The DOI matches its Crossref record; 80 of the 80 references it cites were found in a registry or an index.
Scoundrels or Stars? Theory and Evidence on the Quality of Workers in Online Labor Markets
Open accessAbstract
Online labor markets allow rapid recruitment of large numbers of workers for very low pay. Although online workers are often used as research participants, there is little evidence that they are motivated to make costly choices to forgo wealth or leisure that are often central to addressing accounting research questions. Thus, we investigate the validity of using online workers as a proxy for non-experts when accounting research designs use more demanding tasks than these workers typically complete. Three experiments examine the costly choices of online workers relative to student research participants. We find that online workers are at least as willing as students to make costly choices, even at significantly lower wages. We also find that online workers are sensitive to performance-based wages, which are just as effective in inducing high effort as high fixed wages. We discuss implications of our results for conducting accounting research with online workers. Data Availability: Contact the authors.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Ryan, R. M., & Deci, E. L. (2000) · American Psychologist · doi:10.1037//0003-066x.55.1.68 · Cited by 54,599
Preacher, K. J., & Hayes, A. F. (2008) · Behavior Research Methods · doi:10.3758/brm.40.3.879 · Cited by 34,117
The Market for "Lemons": Quality Uncertainty and the Market Mechanism
Akerlof, G. A. (1970) · The Quarterly Journal of Economics · doi:10.2307/1879431 · Cited by 23,016
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
A Theory of Fairness, Competition, and Cooperation
Fehr, E., & Schmidt, K. M. (1999) · The Quarterly Journal of Economics · doi:10.1162/003355399556151 · Cited by 11,264
An integrative model of organizational trust
Mayer, R. C., Davis, J. H., & Schoorman, F. D. (1995) · The Academy of Management Review · doi:10.2307/258792 · Cited by 10,736
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Deci, E. L., Koestner, R., & Ryan, R. M. (1999) · Psychological Bulletin · doi:10.1037/0033-2909.125.6.627 · Cited by 7,176
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
TRUST AND DISTRUST IN ORGANIZATIONS: Emerging Perspectives, Enduring Questions
Kramer, R. M. (1999) · Annual Review of Psychology · doi:10.1146/annurev.psych.50.1.569 · Cited by 3,243
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Reality is Broken Why Games Make Us Better and How They Can Change the World
(2012) · Choice Reviews Online · doi:10.5860/choice.49-6095 · Cited by 2,933
McAuley, E., Duncan, T., & Tammen, V. V. (1989) · Research Quarterly for Exercise and Sport · doi:10.1080/02701367.1989.10607413 · Cited by 2,694
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · cited 12×
“preferences and susceptibilities to heuristics and biases found in prior literature (see, e.g., Paolacci et al. 2010; Horton et al. 2011; Goodman et al. 2012), Third, as noted earlier, online”
BackgroundInfluentialPay Enough or Don't Pay at All*
Gneezy, U., & Rustichini, A. (2000) · Quarterly Journal of Economics · doi:10.1162/003355300554917 · Cited by 2,175 · cited 1×
“Conversely, Goodman et al. (2012) find that online workers pay less attention than laboratory participants to experimental materials, and Crump et al.”
BackgroundInside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Intrinsic motivation and extrinsic incentives jointly predict performance: A 40-year meta-analysis.
Cerasoli, C. P., Nicklin, J. M., & Ford, M. T. (2014) · Psychological Bulletin · doi:10.1037/a0035661 · Cited by 1,981
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
The Online Laboratory: Conducting Experiments in a Real Labor Market∗Preprint
Zeckhauser, R. J. (2016) · Cited by 1,529
Psychological foundations of incentives
Fehr, E., & Falk, A. (2002) · European Economic Review · doi:10.1016/s0014-2921(01)00208-2 · Cited by 1,240
Experimental research in financial accounting
Libby, R., Bloomfield, R., & Nelson, M. W. (2002) · Accounting, Organizations and Society · doi:10.1016/s0361-3682(01)00011-3 · Cited by 1,201
Social Distance and Other-Regarding Behavior in Dictator Games: Reply
Hoffman, E., McCabe, K., & Smith, V. L. (1999) · American Economic Review · doi:10.1257/aer.89.1.340 · Cited by 1,158
Fehr, E., & Fischbacher, U. (2002) · The Economic Journal · doi:10.1111/1468-0297.00027 · Cited by 1,061
Roberts, J. A., Hann, I.-H., & Slaughter, S. A. (2006) · Management Science · doi:10.1287/mnsc.1060.0554 · Cited by 1,024
Who are the crowdworkers?: shifting demographics in mechanical turk
Ross, J., Irani, L., Silberman, M. S., Zaldivar, A., & Tomlinson, B. (2010) · CHI '10 Extended Abstracts on Human Factors in Computing Systems · doi:10.1145/1753846.1753873 · Cited by 1,004
Social Distance and Other-Regarding Behavior in Dictator Games
Hoffman, E., McCabe, K. A., & Smith, V. L. (2000) · Bargaining and Market Behavior · doi:10.1017/cbo9780511528347.009 · Cited by 979
Bonner, S. E., & Sprinkle, G. B. (2002) · Accounting, Organizations and Society · doi:10.1016/s0361-3682(01)00052-6 · Cited by 966
Flow online: lessons learned and future prospects.
Hoffman, D. L., & Novak, T. P. (2009) · Journal of Interactive Marketing · doi:10.1016/j.intmar.2008.10.003 · Cited by 954
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950
The Role of Working Memory in Visual Selective Attention
de Fockert, J. W., Rees, G., Frith, C. D., & Lavie, N. (2001) · Science · doi:10.1126/science.1056496 · Cited by 924
Analyzing the Amazon Mechanical Turk marketplace
Ipeirotis, P. G. (2010) · XRDS: Crossroads, The ACM Magazine for Students · doi:10.1145/1869086.1869094 · Cited by 879
The Influence of Goal-Directed and Experiential Activities on Online Flow Experiences
Novak, T. P., Hoffman, D. L., & Duhachek, A. (2003) · Journal of Consumer Psychology · doi:10.1207/153276603768344744 · Cited by 745
Libby, R., & Luft, J. L. (1993) · Accounting Organizations and Society · doi:10.1016/0361-3682(93)90040-d · Cited by 709
Speier, C., Valacich, J. S., & Vessey, I. (1999) · Decision Sciences · doi:10.1111/j.1540-5915.1999.tb01613.x · Cited by 701
Agency research in managerial accounting: A second look
Baiman, S. (1990) · Accounting, Organizations and Society · doi:10.1016/0361-3682(90)90023-n · Cited by 679
Processing Fluency and Investors’ Reactions to Disclosure Readability
RENNEKAMP, K. (2012) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2012.00460.x · Cited by 660
Work Interrupted: A Closer Look at the Role of Interruptions in Organizational Life
Jett, Q. R., & George, J. M. (2003) · The Academy of Management Review · doi:10.2307/30040736 · Cited by 650
Isen, A. M., & Reeve, J. (2005) · Motivation and Emotion · doi:10.1007/s11031-006-9019-8 · Cited by 638
The labor economics of paid crowdsourcing
Horton, J. J., & Chilton, L. B. (2010) · Proceedings of the 11th ACM conference on Electronic commerce · doi:10.1145/1807342.1807376 · Cited by 601
Skadberg, Y. X., & Kimmel, J. R. (2004) · Computers in Human Behavior · doi:10.1016/s0747-5632(03)00050-5 · Cited by 595
The sound of silence in prisoner's dilemma and dictator games
Bohnet, I., & Frey, B. S. (1999) · Journal of Economic Behavior & Organization · doi:10.1016/s0167-2681(98)00121-8 · Cited by 487
Perspectives on experimental research in managerial accounting
Sprinkle, G. B. (2003) · Accounting, Organizations and Society · doi:10.1016/s0361-3682(01)00058-7 · Cited by 453
Honesty in Managerial Reporting
Evans, J. H., Hannan, R. L., Krishnan, R., & Moser, D. V. (2001) · The Accounting Review · doi:10.2308/accr.2001.76.4.537 · Cited by 441
Are M.B.A. Students a Good Proxy for Nonprofessional Investors?
Elliott, W. B., Hodge, F. D., Kennedy, J. J., & Pronk, M. (2007) · The Accounting Review · doi:10.2308/accr.2007.82.1.139 · Cited by 405
Bonner, S. E., Hastie, R., Sprinkle, G. B., & Young, S. M. (2000) · Journal of Management Accounting Research · doi:10.2308/jmar.2000.12.1.19 · Cited by 401
Cooperation and Contagion in Web-Based, Networked Public Goods Experiments
Suri, S., & Watts, D. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016836 · Cited by 354
Brandon, D. M., Long, J. H., Loraas, T. M., Mueller-Phillips, J., & Vansant, B. (2014) · Behavioral Research in Accounting · doi:10.2308/bria-50651 · Cited by 315
The Effect of Information Systems on Honesty in Managerial Reporting: A Behavioral Perspective*
Hannan, R. L., Rankin, F. W., & Towry, K. L. (2006) · Contemporary Accounting Research · doi:10.1506/8274-j871-2jtt-5210 · Cited by 250
Valley, K. L., Moag, J., & Bazerman, M. H. (1998) · Journal of Economic Behavior & Organization · doi:10.1016/s0167-2681(97)00054-1 · Cited by 240
Pre-Decision Information and Participative Management Control Systems
Baiman, S., & III, J. H. E. (1983) · Journal of Accounting Research · doi:10.2307/2490780 · Cited by 231
When the Muse Takes It All: A Model for the Experience of Timelessness in Organizations
Mainemelis, C. (2001) · The Academy of Management Review · doi:10.2307/3560241 · Cited by 225
Does monitoring decrease work effort?
Dickinson, D., & Villeval, M.-C. (2008) · Games and Economic Behavior · doi:10.1016/j.geb.2007.08.004 · Cited by 219
Agency theory, performance evaluation, and the hypothetical construct of intrinsic motivation
Kunz, A. H., & Pfaff, D. (2002) · Accounting, Organizations and Society · doi:10.1016/s0361-3682(01)00031-9 · Cited by 218
Napier, B. J., & Ferris, G. R. (1993) · Human Resource Management Review · doi:10.1016/1053-4822(93)90004-n · Cited by 210
Brazel, J. F., Agoglia, C. P., & Hatfield, R. C. (2004) · The Accounting Review · doi:10.2308/accr.2004.79.4.949 · Cited by 209
Clifford, S., & Jerit, J. (2014) · Journal of Experimental Political Science · doi:10.1017/xps.2014.5 · Cited by 206
Measuring and Motivating Quantity, Creativity, or Both
KACHELMEIER, S. J., REICHERT, B. E., & WILLIAMSON, M. G. (2008) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2008.00277.x · Cited by 190
Peecher, M. E., & Solomon, I. (2001) · International Journal of Auditing · doi:10.1111/1099-1123.00335 · Cited by 174
Reward System Design and Group Creativity: An Experimental Investigation
Chen, C. X., Williamson, M. G., & Zhou, F. H. (2012) · The Accounting Review · doi:10.2308/accr-50232 · Cited by 165
Asymmetry of Pre-Decision Information and Managerial Accounting
Penno, M. C. (1984) · Journal of Accounting Research · doi:10.2307/2490707 · Cited by 150
Can Wages Buy Honesty? The Relationship Between Relative Wages and Employee Theft
Chen, C. X., & Sandino, T. (2012) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2012.00456.x · Cited by 147
Negotiated Transfer Pricing: Is Fairness Easier Said than Done?
Kachelmeier, S. J., & Towry, K. L. (2002) · The Accounting Review · doi:10.2308/accr.2002.77.3.571 · Cited by 138
How Control System Design Influences Performance Misreporting
Maas, V. S., & van Rinsum, M. (2011) · Journal of Accounting Research · doi:10.1111/1475-679x.12025 · Cited by 137
Kachelmeier, S. J., & Williamson, M. G. (2010) · The Accounting Review · doi:10.2308/accr.2010.85.5.1669 · Cited by 131
The Robustness of Honesty Effects on Budget Proposals when the Superior has Rejection Authority
Douthit, J. D., & Stevens, D. E. (2014) · The Accounting Review · doi:10.2308/accr-50937 · Cited by 130
The Effects of Norms on Investor Reactions to Derivative Use
Koonce, L. L., Miller, J., & Winchel, J. (2014) · Contemporary Accounting Research · doi:10.1111/1911-3846.12118 · Cited by 123
Festré, A., & Garrouste, P. (2014) · Journal of Economic Surveys · doi:10.1111/joes.12059 · Cited by 109
Deciphering the Role of RND Efflux Transporters in Burkholderia cenocepacia
Bazzini, S., Udine, C., Sass, A. M., Pasca, M. R., Longo, F., Emiliani, G., Fondi, M., Perrin, E., Decorosi, F., Viti, C., et al. (2011) · PLoS ONE · doi:10.1371/journal.pone.0018902 · Cited by 106
Contracting on Contemporaneous versus Forward‐Looking Measures: An Experimental Investigation*
Farrell, A. M., Kadous, K., & Towry, K. L. (2008) · Contemporary Accounting Research · doi:10.1506/car.25.3.5 · Cited by 104
Self-citationLost in Translation: The Effects of Incentive Compensation on Strategy Surrogation
Choi, J., Hecht, G., & Tayler, W. B. (2012) · The Accounting Review · doi:10.2308/accr-10273 · Cited by 96
Strategy Selection, Surrogation, and Strategic Performance Measurement Systems
Choi, J., Hecht, G., & Tayler, W. B. (2012) · Journal of Accounting Research · doi:10.1111/j.1475-679x.2012.00465.x · Cited by 95
Rennekamp, K. M., Rupar, K., & Seybert, N. (2012) · The Accounting Review · doi:10.2308/accr-50879 · Cited by 79
Grenier, J. H., Pomeroy, B., & Stern, M. T. (2014) · Contemporary Accounting Research · doi:10.1111/1911-3846.12092 · Cited by 75
Self-citationThe effect of rankings on honesty in budget reporting
Brown, J. L., Fisher, J. G., Sooy, M. T., & Sprinkle, G. B. (2014) · Accounting Organizations and Society · doi:10.1016/j.aos.2014.03.001 · Cited by 69
Honesty in an Online Labor Market
Suri, S., Goldstein, D. G., & Mason, W. (2013) · Cited by 69
The Effects of Performance-Contingent Financial Incentives in Online Labor Markets
Yin, M., Chen, Y., & Sun, Y. (2013) · Proceedings of the AAAI Conference on Artificial Intelligence · doi:10.1609/aaai.v27i1.8461 · Cited by 50
Who is the Average Individual Investor? Numerical Skills and Implications for Accounting ResearchPreprint
Krische, S. D. (2014) · SSRN Electronic Journal · doi:10.2139/ssrn.2426570 · Cited by 31
The Effects of Sharing the Consequences of Tax Evasion and Regret Salience on Taxpayer Compliance
Brink, W. D. (2014) · Scholar Commons (University of South Carolina) · Cited by 1
Cited by
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
Chandler, J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01273-7 · Cited by 668
MTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330
Brasel, K., Doxey, M. M., Grenier, J. H., & Reffett, A. (2016) · The Accounting Review · doi:10.2308/accr-51380 · Cited by 313
Self-citationChandler, J. J., & Paolacci, G. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617698203 · Cited by 277
Metadata
- Authors
- Farrell, A. M., Grenier, J. H., & Leiby, J.
- Venue
- The Accounting Review
- Publisher
- American Accounting Association
- Published
- 2016-03-01
- Volume
- 92
- Issue
- 1
- Pages
- 93-114
- ISSN
- 0001-4826 (ISSN-L)
- 1558-7967
- OpenAlex
- W3123257142
- Cited by
- 288
- References
- 80
- FWCI
- 49.15
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.