The DOI matches its Crossref record; 122 of the 122 references it cites were found in a registry or an index.
Crowdsourcing Consumer Research
Open accessAbstract
Data collection in consumer research has progressively moved away from traditional samples (e.g., university undergraduates) and toward Internet samples. In the last complete volume of the Journal of Consumer Research (June 2015–April 2016), 43% of behavioral studies were conducted on the crowdsourcing website Amazon Mechanical Turk (MTurk). The option to crowdsource empirical investigations has great efficiency benefits for both individual researchers and the field, but it also poses new challenges and questions for how research should be designed, conducted, analyzed, and evaluated. We assess the evidence on the reliability of crowdsourced populations and the conditions under which crowdsourcing is a valid strategy for data collection. Based on this evidence, we propose specific guidelines for researchers to conduct high-quality research via crowdsourcing. We hope this tutorial will strengthen the community’s scrutiny on data collection practices and move the field toward better and more valid crowdsourcing of consumer research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The weirdest people in the world?
Henrich, J., Heine, S. J., & Norenzayan, A. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x0999152x · Cited by 12,641
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
WEIRD languages have misled us, too
Majid, A., & Levinson, S. C. (2010) · Behavioral and Brain Sciences · doi:10.1017/s0140525x1000018x · Cited by 4,076
Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk
Berinsky, A. J., Huber, G. A., & Lenz, G. S. (2012) · Political Analysis · doi:10.1093/pan/mpr057 · Cited by 4,008
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Self-citationInstructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Gosling, S. D., Vazire, S., Srivastava, S., & John, O. P. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.93 · Cited by 2,932
Data collection in a flat world: the strengths and weaknesses of Mechanical Turk samples
Goodman, J. K., Cryder, C. E., & Cheema, A. (2013) · Journal of Behavioral Decision Making · doi:10.1002/bdm.1753 · Cited by 2,242 · cited 4×
“…a fair wage, and whereas most evidence suggests that pay rates do not affect data quality in a typical consumer study (Burhmester et al. 2011; Goodman et al. 2013; Mason and Watts 2010; Litman et al. 2015; but see Fort et al. 2011 for an exception), adequate payments are in the researchers’…”
BackgroundResultInfluentialInside the Turk: Understanding Mechanical Turk as a Participant Pool
Paolacci, G., & Chandler, J. (2014) · Current Directions in Psychological Science · doi:10.1177/0963721414531598 · Cited by 2,136
Self-citationTurkPrime.com: A versatile crowdsourcing data acquisition platform for the behavioral sciences
Litman, L., Robinson, J., & Abberbock, T. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0727-z · Cited by 1,982
oTree—An open-source platform for laboratory, online, and field experiments
Chen, D. L., Schonger, M., & Wickens, C. (2016) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2015.12.001 · Cited by 1,889
Influentials, Networks, and Public Opinion Formation
Watts, D. J., & Dodds, P. S. (2007) · Journal of Consumer Research · doi:10.1086/518527 · Cited by 1,875
Designing Research for Application
Calder, B. J., Phillips, L. W., & Tybout, A. M. (1981) · Journal of Consumer Research · doi:10.1086/208856 · Cited by 1,806
Hauser, D. J., & Schwarz, N. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0578-z · Cited by 1,739
Casler, K., Bickel, L., & Hackett, E. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.05.009 · Cited by 1,662
Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2007) · Perspectives on Psychological Science · doi:10.1111/j.1745-6916.2007.00051.x · Cited by 1,629
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · cited 2×
“To maximize quality, researchers should make full use of quality filters (e.g., on MTurk, re-cruiting workers with approval ratings superior to 95%; Peer et al. 2014).”
BackgroundEvaluating amazon’s mechanical turk as a tool for experimental behavioral research
Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0057410 · Cited by 1,617
Talking about tactile experiences
Obrist, M., Seah, S. A., & Subramanian, S. (2013) · Proceedings of the SIGCHI Conference on Human Factors in Computing Systems · doi:10.1145/2470654.2466220 · Cited by 1,613
Petty, R. E., & Cacioppo, J. T. (1979) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.37.10.1915 · Cited by 1,529
The Online Laboratory: Conducting Experiments in a Real Labor Market∗Preprint
Zeckhauser, R. J. (2016) · Cited by 1,529
Peterson, R. A. (2001) · Journal of Consumer Research · doi:10.1086/323732 · Cited by 1,258
Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing
Cosley, D., Forte, A., Ciolfi, L., & McDonald, D. W. (2015) · Cited by 1,245
Using Mechanical Turk to Study Clinical Populations
Shapiro, D. N., Chandler, J., & Mueller, P. A. (2013) · Clinical Psychological Science · doi:10.1177/2167702612469015 · Cited by 1,139
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
An Agenda for Purely Confirmatory Research
Wagenmakers, E.-J., Wetzels, R., Borsboom, D., van der Maas, H. L. J., & Kievit, R. A. (2012) · Perspectives on Psychological Science · doi:10.1177/1745691612463078 · Cited by 1,042
The viability of crowdsourcing for survey research
Behrend, T. S., Sharek, D. J., Meade, A. W., & Wiebe, E. N. (2011) · Behavior Research Methods · doi:10.3758/s13428-011-0081-0 · Cited by 1,018
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950
Self-citationKraut, R., Olson, J., Banaji, M., Bruckman, A., Cohen, J., & Couper, M. (2004) · American Psychologist · doi:10.1037/0003-066x.59.2.105 · Cited by 924
Huff, C., & Tingley, D. (2015) · Research & Politics · doi:10.1177/2053168015604648 · Cited by 903
Amazon Mechanical Turk: Gold Mine or Coal Mine?
Fort, K., Adda, G., & Cohen, K. B. (2011) · Computational Linguistics · doi:10.1162/coli_a_00057 · Cited by 761
Small Telescopes: Detectability and the Evaluation of Replication Results
Simonsohn, U. (2015) · Psychological Science · doi:10.1177/0956797614567341 · Cited by 742
Social heuristics shape intuitive cooperation
Rand, D. G., Peysakhovich, A., Kraft-Todd, G. T., Newman, G. E., Wurzbacher, O., Nowak, M. A., & Greene, J. D. (2014) · Nature Communications · doi:10.1038/ncomms4677 · Cited by 720
p -Curve and Effect Size: Correcting for Publication Bias Using Only Significant Results
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Perspectives on Psychological Science · doi:10.1177/1745691614553988 · Cited by 717 · cited 1×
“Issues such as selective reporting, file drawer, and preregistration have been recently receiving the attention they deserve (Moore 2016; Simonsohn, Nelson, and Simmons 2014; Van’t Veer and Giner-Sorolla 2016; Wagenmakers et al. 2012).”
BackgroundRand, D. G. (2012) · Journal of Theoretical Biology · doi:10.1016/j.jtbi.2011.03.004 · Cited by 683
The demographic and political composition of Mechanical Turk samples
Levay, K. E., Freese, J., & Druckman, J. N. (2016) · Sage Open · doi:10.1177/2158244016636433 · Cited by 676 · cited 1×
“…model-based poststratification can be used to statistically adjust the estimates obtained from nonrepresentative samples (Battaglia, Hoaglin, and Frankel 2013; Park, Gelman, and Bafumi 2004), and may be applied to MTurk samples (Goel, Obeng, and Rothschild 2015; Levay, Freese, and Druckman 2016).”
BackgroundThe Concept of External Validity
Calder, B. J., Phillips, L. W., & Tybout, A. M. (1982) · Journal of Consumer Research · doi:10.1086/208920 · Cited by 613
Investigating an alternate form of the cognitive reflection test
Thomson, K. S., & Oppenheimer, D. M. (2016) · Judgment and Decision Making · doi:10.1017/s1930297500007622 · Cited by 609
The labor economics of paid crowdsourcing
Horton, J. J., & Chilton, L. B. (2010) · Proceedings of the 11th ACM conference on Electronic commerce · doi:10.1145/1807342.1807376 · Cited by 601
Weinberg, J., Freese, J., & McElhattan, D. (2014) · Sociological Science · doi:10.15195/v1.a19 · Cited by 582
Political diversity will improve social psychological science
Duarte, J. L., Crawford, J. T., Stern, C., Haidt, J., Jussim, L., & Tetlock, P. E. (2015) · Behavioral and Brain Sciences · doi:10.1017/s0140525x14000430 · Cited by 563
Financial incentives and the "performance of crowds"
Mason, W., & Watts, D. J. (2009) · Proceedings of the ACM SIGKDD Workshop on Human Computation · doi:10.1145/1600150.1600175 · Cited by 520
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
(2006) · doi:10.1145/1124772 · Cited by 517
Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls
Park, D. K., Gelman, A., & Bafumi, J. (2004) · Political Analysis · doi:10.1093/pan/mph024 · Cited by 480 · cited 1×
“…model-based poststratification can be used to statistically adjust the estimates obtained from nonrepresentative samples (Battaglia, Hoaglin, and Frankel 2013; Park, Gelman, and Bafumi 2004), and may be applied to MTurk samples (Goel, Obeng, and Rothschild 2015; Levay, Freese, and Druckman 2016).”
BackgroundOn the External Validity of Experiments in Consumer Research
Lynch, Jr. (1982) · Journal of Consumer Research · doi:10.1086/208919 · Cited by 469
Zhou, H., & Fishbach, A. (2016) · Journal of Personality and Social Psychology · doi:10.1037/pspa0000056 · Cited by 459
Internet Research in Psychology
Gosling, S. D., & Mason, W. (2015) · Annual Review of Psychology · doi:10.1146/annurev-psych-010814-015321 · Cited by 458
Pre-registration in social psychology—A discussion and suggested template
van 't Veer, A. E., & Giner-Sorolla, R. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2016.03.004 · Cited by 439
Discovery-Oriented Consumer Research
Wells, W. D. (1993) · Journal of Consumer Research · doi:10.1086/209318 · Cited by 399
Breaking monotony with meaning: Motivation in crowdsourcing markets
Chandler, D., & Kapelner, A. (2013) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2013.03.003 · Cited by 398
Cooperation and Contagion in Web-Based, Networked Public Goods Experiments
Suri, S., & Watts, D. J. (2011) · PLoS ONE · doi:10.1371/journal.pone.0016836 · Cited by 354
Cooperation and contagion in web-based, networked public goods experiments
Suri, S., & Watts, D. J. (2011) · ACM SIGecom Exchanges · doi:10.1145/1998549.1998550 · Cited by 318
The importance of assessing clinical phenomena in Mechanical Turk research.
Arditte, K. A., Çek, D., Shaw, A. M., & Timpano, K. R. (2016) · Psychological Assessment · doi:10.1037/pas0000217 · Cited by 311
The average laboratory samples a population of 7,300 Amazon Mechanical Turk workers
Stewart, N., Ungemach, C., Harris, A. J. L., Bartels, D. M., Newell, B. R., Paolacci, G., & Chandler, J. (2015) · Judgment and Decision Making · doi:10.1017/s1930297500005611 · Cited by 304 · cited 1×
“Rather, at any point in time there might be a few tens of thousands workers available, and in any quarter the average laboratory may be sampling from a population of less than 10,000 (Fort, Adda, and Cohen 2011; Stewart et al. 2015).”
BackgroundSelf-citationFerber, R. (1977) · Journal of Consumer Research · doi:10.1086/208679 · Cited by 301
The Internet as Psychological Laboratory
Skitka, L. J., & Sargis, E. G. (2006) · Annual Review of Psychology · doi:10.1146/annurev.psych.57.102904.190048 · Cited by 293
Chandler, J. J., & Paolacci, G. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617698203 · Cited by 277
Self-citationLitman, L., Robinson, J., & Rosenzweig, C. (2015) · Behavior Research Methods · doi:10.3758/s13428-014-0483-x · Cited by 275
Lynch, J. G. (1999) · Journal of the Academy of Marketing Science · doi:10.1177/0092070399273007 · Cited by 271
Practical Considerations in Raking Survey Data
Battaglia, M. P., Hoaglin, D. C., & Frankel, M. R. (2009) · Survey Practice · doi:10.29115/sp-2009-0019 · Cited by 231
The seven sins of consumer psychology
Pham, M. T. (2013) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2013.07.004 · Cited by 224
Learning and the diffusion of regime contention in the Arab Spring
Bamert, J., Gilardi, F., & Wasserfallen, F. (2015) · Research & Politics · doi:10.1177/2053168015593306 · Cited by 213
Clifford, S., & Jerit, J. (2014) · Journal of Experimental Political Science · doi:10.1017/xps.2014.5 · Cited by 206
Psychological research in the internet age: The quality of web-based data
Ramsey, S. R., Thompson, K. L., McKenzie, M., & Rosenbaum, A. (2016) · Computers in Human Behavior · doi:10.1016/j.chb.2015.12.049 · Cited by 203
Cooperation and assortativity with dynamic partner updating
Wang, J., Suri, S., & Watts, D. J. (2012) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.1120867109 · Cited by 198
Using Nonnaive Participants Can Reduce Effect Sizes
Chandler, J., Paolacci, G., Peer, E., Mueller, P., & Ratliff, K. A. (2015) · Psychological Science · doi:10.1177/0956797615585115 · Cited by 185
External Validity and the Research Process: A Comment on the Calder/Lynch Dialogue
McGrath, J. E., & Brinberg, D. (1983) · Journal of Consumer Research · doi:10.1086/208951 · Cited by 182
Emotional reactions to losing explain gender differences in entering a risky lottery
Eriksson, K., & Simpson, B. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500001030 · Cited by 172
Task search in a human computation market
Chilton, L. B., Horton, J. J., Miller, R. C., & Azenkot, S. (2010) · Proceedings of the ACM SIGKDD Workshop on Human Computation · doi:10.1145/1837885.1837889 · Cited by 171
Assessing the reliability of the M5-120 on Amazon’s mechanical Turk
Holden, C. J., Dennie, T., & Hicks, A. D. (2013) · Computers in Human Behavior · doi:10.1016/j.chb.2013.02.020 · Cited by 169
It’s a Trap! Instructional Manipulation Checks Prompt Systematic Thinking on “Tricky” Tasks
Hauser, D. J., & Schwarz, N. (2015) · Sage Open · doi:10.1177/2158244015584617 · Cited by 168
Calder, B. J., Phillips, L. W., & Tybout, A. M. (1983) · Journal of Consumer Research · doi:10.1086/208950 · Cited by 164
How Childhood Advertising Exposure Can Create Biased Product Evaluations That Persist into Adulthood
Connell, P. M., Brucks, M. L., & Nielsen, J. H. (2014) · Journal of Consumer Research · doi:10.1086/675218 · Cited by 156
Banerjee, A., & Duflo, E. (2017) · North-Holland eBooks · Cited by 156
Negative affect intensity influences drinking to cope through facets of emotion dysregulation
Veilleux, J. C., Skinner, K. D., Reese, E. D., & Shaver, J. A. (2013) · Personality and Individual Differences · doi:10.1016/j.paid.2013.11.012 · Cited by 149
Petty, R. E., & Cacioppo, J. T. (1996) · Journal of Marketing Research · doi:10.1177/002224379603300101 · Cited by 148
Feitosa, J., Joseph, D. L., & Newman, D. A. (2014) · Personality and Individual Differences · doi:10.1016/j.paid.2014.11.017 · Cited by 145
Callison-Burch, C., & Association for Computing Machinery-Digital Library (2010) · Cited by 144
Bohannon, J. (2011) · Science · doi:10.1126/science.334.6054.307 · Cited by 140
The public stigma of mental illness means a difference between you and me
Corrigan, P. W., Bink, A. B., Fokuo, J. K., & Schmidt, A. (2015) · Psychiatry Research · doi:10.1016/j.psychres.2014.12.047 · Cited by 132
The Role of External Validity in Theoretical Research
Lynch, J. G. (1983) · Journal of Consumer Research · doi:10.1086/208949 · Cited by 126
Participants at Your Fingertips
Johnson, D. R., & Borden, L. A. (2012) · Teaching of Psychology · doi:10.1177/0098628312456615 · Cited by 119
Using Nonnaive Participants Can Reduce Effect SizesPreprint
Chandler, J., Paolacci, G., Péer, E., Mueller, P. A., & Ratliff, K. A. (2015) · RePEc: Research Papers in Economics · Cited by 118
Self-citationOpportunity costs of reward delays and the discounting of hypothetical money and cigarettes
Johnson, P. S., Herrmann, E. S., & Johnson, M. W. (2014) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.110 · Cited by 117
The r FaceReader: Online facial expression recognition.
y l, D. U. (2005) · Cited by 115
Gleibs, I. H. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0789-y · Cited by 115
Knowledge creation in consumer research: Multiple routes, multiple criteria
Lynch, J. G., Alba, J. W., Krishna, A., Morwitz, V. G., & Gürhan‐Canlı, Z. (2012) · Journal of Consumer Psychology · doi:10.1016/j.jcps.2012.06.004 · Cited by 111
Should Birds of a Feather Flock Together? Understanding Self-Control Decisions in Dyads
Nikolova, H. D., & Lamberton, C. P. (2014) · Journal of Consumer Research · doi:10.1086/676599 · Cited by 102
‘Subtypes’ in the Presentation of Autistic Traits in the General Adult Population
Palmer, C. J., Paton, B. K., Enticott, P. G., & Hohwy, J. (2014) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-014-2289-1 · Cited by 99
Baumeister, R. F., Vohs, K. D., & Funder, D. C. (2009) · Oxford University Press eBooks · doi:10.1093/acprof:oso/9780195377798.003.0002 · Cited by 99
Simonson, I., & Sela, A. (2010) · Journal of Consumer Research · doi:10.1086/657022 · Cited by 98
The Ant and the Grasshopper: Understanding Personal Saving Orientation of Consumers
Dholakia, U. M., Tam, L., Yoon, S., & Wong, N. (2016) · Journal of Consumer Research · doi:10.1093/jcr/ucw004 · Cited by 98
Conditioned Superstition: Desire for Control and Consumer Brand Preferences
Hamerman, E. J., & Johar, G. V. (2013) · Journal of Consumer Research · doi:10.1086/670762 · Cited by 95
Do Mechanical Turks dream of square pie charts?
Kosara, R., & Ziemkiewicz, C. (2010) · doi:10.1145/2110192.2110202 · Cited by 84
Sakaluk, J. K. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.013 · Cited by 79
CHI '09 Extended Abstracts on Human Factors in Computing Systems
Olsen, D. R., Hinckley, K., Morris, M. R., Hudson, S. E., & Greenberg, S. (2009) · doi:10.1145/1520340 · Cited by 72
An online daily diary study of alcohol use using A mazon's M echanical T urk
Boynton, M. H., & Richman, L. S. (2014) · Drug and Alcohol Review · doi:10.1111/dar.12163 · Cited by 69
Selectively Recruiting Participants from Amazon Mechanical Turk Using QualtricsPreprint
Péer, E., Paolacci, G., Chandler, J., & Mueller, P. A. (2012) · SSRN Electronic Journal · doi:10.2139/ssrn.2100631 · Cited by 67
Self-citationAlba, J. W. (2012) · Journal of Consumer Research · doi:10.1086/661230 · Cited by 65
Paay, J., Kjeldskov, J., & Skov, M. B. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675194 · Cited by 51
Celebrate or Commemorate? A Material Purchase Advantage When Honoring Special Life Events
Goodman, J. K., Malkoc, S. A., & Stephenson, B. (2016) · Journal of the Association for Consumer Research · doi:10.1086/688352 · Cited by 49
Self-citationSiegel, J. T., Navarro, M. A., & Thomson, A. L. (2015) · Social Science & Medicine · doi:10.1016/j.socscimed.2015.08.020 · Cited by 45
Moore, D. A. (2016) · American Psychologist · doi:10.1037/a0040195 · Cited by 44
The Foresight Effect: Local Optimism Motivates Consistency and Local Pessimism Motivates Variety
Yang, A. X., & Urminsky, O. (2015) · Journal of Consumer Research · doi:10.1093/jcr/ucv039 · Cited by 41
“Lordy, Lordy, Look Who’s 40!” The Journal of Consumer Research Reaches a Milestone
Rapp, J. M., & Hill, R. P. (2015) · Journal of Consumer Research · doi:10.1093/jcr/ucv011 · Cited by 41
Proceedings of the ACM SIGKDD Workshop on Human Computation
(2009) · doi:10.1145/1600150 · Cited by 32
Social Eye Tracking: Gaze Recall with Online Crowds
Cheng, S., Sun, Z., Ma, X., Forlizzi, J., Hudson, S. E., & Dey, A. (2015) · ACM Conference on Computer Supported Cooperative Work (CSCW) · doi:10.1145/2675133.2675249 · Cited by 31
Screening Amazon’s Mechanical Turk for Adults With ADHD
Wymbs, B. T., & Dawson, A. E. (2015) · Journal of Attention Disorders · doi:10.1177/1087054715597471 · Cited by 29
Is Mechanical Turk the Answer to Our Sampling Woes?
Keith, M. G., & Harms, P. D. (2016) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.130 · Cited by 27
Fergus, T. A., & Rowatt, W. C. (2014) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2014.09.003 · Cited by 26
Autism-spectrum traits predict humor styles in the general population
Eriksson, K. (2013) · Humor - International Journal of Humor Research · doi:10.1515/humor-2013-0030 · Cited by 26
Volunteering and Reemployment Status in Unemployed Emerging Adults: A Time-worthy Investment?
Konstam, V., Tomek, S. E., Celen‐Demirtas, S., & Sweeney, K. (2014) · Journal of Career Assessment · doi:10.1177/1069072714523248 · Cited by 22
Reese, E. D., & Veilleux, J. C. (2015) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntv054 · Cited by 21
Cooper, E. A., & Farid, H. (2014) · Public Understanding of Science · doi:10.1177/0963662514554354 · Cited by 21
MTurk Crowdsourcing: A Viable Method for Rapid Discovery of Arabic Nicknames?
Higgins, C., McGrath, E. J., & Moretto, L. (2010) · Cited by 13
Extracting Heart Rate from Videos of Online Participants
Muender, T., Miller, M. K., Birk, M. V., & Mandryk, R. L. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858064 · Cited by 12
DeVoe, S. E., & House, J. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.11.004 · Cited by 12
Time, money, and happiness: Does putting a price on time affect our ability to smell the roses?
Campbell, L., Connors, S., Henderson, C., Khamitov, M., & Moroz, S. (2016) · Knowledge Commons (Lakehead University) · doi:10.17613/q0nq8-w7d22 · Cited by 7
Cheating on online assessment tests: prevalence and impact on validity
Cavanagh, T. M., Kraiger, K., Gibbons, A., Henry, K., & Maynard, T. (2007) · Dominican Scholar (Dominican University of California) · doi:10.25675/3.024341 · Cited by 4
Cited by
Scientific procedures and rationales for systematic literature reviews (SPAR‐4‐SLR)
Paul, J., Lim, W. M., O’Cass, A., Hao, A. W., & Bresciani, S. (2021) · International Journal of Consumer Studies · doi:10.1111/ijcs.12695 · Cited by 2,364
Data quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Online panels in social science research: Expanding sampling methods beyond Mechanical Turk
Chandler, J., Rosenzweig, C., Moss, A. J., Robinson, J., & Litman, L. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01273-7 · Cited by 668
The value of online surveys: a look back and a look ahead
Evans, J. R., & Mathur, A. (2018) · Internet Research · doi:10.1108/intr-03-2018-0089 · Cited by 545
A Tale of Two Sample Sources: Do Results from Online Panel Data and Conventional Data Converge?
Walter, S. L., Seibert, S. E., Goering, D., & O’Boyle, E. H. (2019) · Journal of Business and Psychology · doi:10.1007/s10869-018-9552-y · Cited by 544
Metadata
- Authors
- Goodman, J. K., & Paolacci, G.
- Venue
- Journal of Consumer Research
- Publisher
- Oxford University Press (OUP)
- Published
- 2017-02-22
- Volume
- 44
- Issue
- 1
- Pages
- 196-210
- ISSN
- 0093-5301 (ISSN-L)
- 1537-5277
- OpenAlex
- W2603418839
- Cited by
- 588
- References
- 122
- FWCI
- 62.14
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.