The DOI matches its Crossref record; 50 of the 52 references it cites were found in a registry or an index.
Evaluating amazon’s mechanical turk as a tool for experimental behavioral research
Open accessAbstract
Amazon Mechanical Turk (AMT) is an online crowdsourcing service where anonymous online workers complete web-based tasks for small sums of money. The service has attracted attention from experimental psychologists interested in gathering human subject data more efficiently. However, relative to traditional laboratory studies, many aspects of the testing environment are not under the experimenter's control. In this paper, we attempt to empirically evaluate the fidelity of the AMT system for use in cognitive behavioral experiments. These types of experiment differ from simple surveys in that they require multiple trials, sustained attention from participants, comprehension of complex instructions, and millisecond accuracy for response recording and stimulus presentation. We replicate a diverse body of tasks from experimental psychology including the Stroop, Switching, Flanker, Simon, Posner Cuing, attentional blink, subliminal priming, and category learning tasks using participants recruited using AMT. While most of replications were qualitatively successful and validated the approach of collecting data anonymously online using a web-browser, others revealed disparity between laboratory results and online results. A number of important lessons were encountered in the process of conducting these replications that should be of value to other researchers.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Regarding online experiments in general, Crump et al. [32] showed that the data collected online using a web-browser seemed mostly in line with laboratory results, so long as the experiment methods were solid.”
From Adaptive trust calibration for human-AI collaboration (2020) · cites this paper as [32]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Studies of interference in serial verbal reactions
Stroop, J. R. (1935) · Journal of Experimental Psychology · doi:10.1037/h0054651 · Cited by 18,622
Amazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
Effects of noise letters upon the identification of a target letter in a nonsearch task
Eriksen, B. A., & Eriksen, C. W. (1974) · Perception & Psychophysics · doi:10.3758/bf03203267 · Cited by 7,230
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011) · Psychological Science · doi:10.1177/0956797611417632 · Cited by 6,567
Half a century of research on the Stroop effect: An integrative review.
MacLeod, C. M. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.109.2.163 · Cited by 5,500
Running experiments on Amazon Mechanical Turk
Paolacci, G., Chandler, J., & Ipeirotis, P. G. (2010) · Judgment and Decision Making · doi:10.1017/s1930297500002205 · Cited by 3,767
Instructional manipulation checks: Detecting satisficing to increase statistical power
Oppenheimer, D. M., Meyvis, T., & Davidenko, N. (2009) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2009.03.009 · Cited by 3,493
Conducting behavioral research on Amazon’s Mechanical Turk
Mason, W., & Suri, S. (2012) · Behavior Research Methods · doi:10.3758/s13428-011-0124-6 · Cited by 3,172
Monsell, S. (2003) · Trends in Cognitive Sciences · doi:10.1016/s1364-6613(03)00028-7 · Cited by 2,984
Components of visual orienting
Posner, M. I. (1984) · Attention and performance · Cited by 2,980
Flagged first, then by citations
2 of the 52 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Ipeirotis PG (2010) Demographics of Mechanical Turk (Tech. Rep. No. CeDER-10-01). New York: New York University. Available: http://hdl.handle.net/2451/29585. Accessed 2012 Jun 1.
Cited by
Sorted by citations
Items 101–200 of 1,000
Saunders, B., Milyavskaya, M., Etz, A. J., Randles, D., & Inzlicht, M. (2018) · Collabra Psychology · doi:10.1525/collabra.134 · Cited by 155
Organizational Conspiracy Beliefs: Implications for Leadership Styles and Employee Outcomes
van Prooijen, J., & de Vries, R. E. (2015) · Journal of Business and Psychology · doi:10.1007/s10869-015-9428-3 · Cited by 136
Do learning rates adapt to the distribution of rewards?
Gershman, S. J. (2015) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0790-3 · Cited by 134
Crowdsourcing for Cognitive Science – The Utility of Smartphones
Brown, H. R., Zeidman, P., Smittenaar, P., Adams, R. A., McNab, F., Rutledge, R. B., & Dolan, R. J. (2014) · PLoS ONE · doi:10.1371/journal.pone.0100662 · Cited by 129
Quantifying Phishing Susceptibility for Detection and Behavior Decisions
Canfield, C., Fischhoff, B., & Davis, A. L. (2016) · Human Factors The Journal of the Human Factors and Ergonomics Society · doi:10.1177/0018720816665025 · Cited by 129
Collecting Psycholinguistic Response Time Data Using Amazon Mechanical Turk
Enochson, K., & Culbertson, J. (2015) · PLoS ONE · doi:10.1371/journal.pone.0116946 · Cited by 128
Desai, S. C., & Reimers, S. J. (2018) · Behavior Research Methods · doi:10.3758/s13428-018-1066-z · Cited by 166
Combating Rumor Spread on Social Media: The Effectiveness of Refutation and Warning
Ozturk, P., Li, H., & Sakamoto, Y. (2015) · Hawaii International Conference on System Sciences (HICSS) · doi:10.1109/hicss.2015.288 · Cited by 126
Kukucka, J., & Kassin, S. M. (2013) · Law and Human Behavior · doi:10.1037/lhb0000066 · Cited by 126
Running Behavioral Operations Experiments Using Amazon's Mechanical Turk
Lee, Y. S., Seo, Y. W., & Siemsen, E. (2018) · Production and Operations Management · doi:10.1111/poms.12841 · Cited by 128
Ward, E. J., & Scholl, B. J. (2014) · Psychonomic Bulletin & Review · doi:10.3758/s13423-014-0745-8 · Cited by 130
Comparisons of Online Recruitment Strategies for Convenience Samples
Antoun, C., Zhang, C., Conrad, F. G., & Schober, M. F. (2015) · Field Methods · doi:10.1177/1525822x15603149 · Cited by 136
Scheutz, M. J., & Arnold, T. (2016) · ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451772 · Cited by 124
Responses to Decline in Marketized Public Services: An Experimental Evaluation of Choice Overload
Jilke, S., Ryzin, G. G. V., & de Walle, S. V. (2015) · Journal of Public Administration Research and Theory · doi:10.1093/jopart/muv021 · Cited by 129
Subtlex-pl: subtitle-based word frequency estimates for Polish
Mandera, P., Keuleers, E., Wodniecka, Z., & Brysbaert, M. (2014) · Behavior Research Methods · doi:10.3758/s13428-014-0489-4 · Cited by 120
Internet surveillance, regulation, and chilling effects online: a comparative case study
Penney, J. W. (2017) · Internet Policy Review · doi:10.14763/2017.2.692 · Cited by 122
Online branding: Development of hotel branding through interactivity theory
Barreda, A. A., Bilgihan, A., Nusair, K., & Okumuş, F. (2016) · Tourism Management · doi:10.1016/j.tourman.2016.06.007 · Cited by 119
Hart, C. (2017) · Discourse & Society · doi:10.1177/0957926517734663 · Cited by 119
Moral foundations underlying behavioral compliance during the COVID-19 pandemic
Chan, E. Y. (2020) · Personality and Individual Differences · doi:10.1016/j.paid.2020.110463 · Cited by 127
Roulin, N. (2015) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.24 · Cited by 115
Opportunity costs of reward delays and the discounting of hypothetical money and cigarettes
Johnson, P. S., Herrmann, E. S., & Johnson, M. W. (2014) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.110 · Cited by 116
Lovett, M., Bajaba, S. M., Lovett, M., & Simmering, M. J. (2017) · Applied Psychology · doi:10.1111/apps.12124 · Cited by 125
Transdiagnostic Phenotyping Reveals a Host of Metacognitive Deficits Implicated in Compulsivity
Seow, T. X. F., & Gillan, C. M. (2020) · Scientific Reports · doi:10.1038/s41598-020-59646-4 · Cited by 117
Usable Security: History, Themes, and Challenges
Garfinkel, S., & Lipford, H. R. (2014) · Synthesis lectures on information security, privacy, and trust · doi:10.2200/s00594ed1v01y201408spt011 · Cited by 112
A Technical Guide to Using Amazon's Mechanical Turk in Behavioral Accounting Research
Buchheit, S., Doxey, M. M., Pollard, T., & Stinson, S. R. (2017) · Behavioral Research in Accounting · doi:10.2308/bria-51977 · Cited by 115
Taking Psychiatry Research Online
Gillan, C. M., & Daw, N. D. (2016) · Neuron · doi:10.1016/j.neuron.2016.06.002 · Cited by 117
Social impacts in social media: An examination of perceived truthfulness and sharing of information
Li, H., & Sakamoto, Y. (2014) · Computers in Human Behavior · doi:10.1016/j.chb.2014.08.009 · Cited by 117
Gilman, T. L., Shaheen, R., Nylocks, K. M., Halachoff, D. J., Chapman, J., Flynn, J. J., Matt, L. M., & Coifman, K. G. (2017) · Behavior Research Methods · doi:10.3758/s13428-016-0842-x · Cited by 128
Duffy, R. D., Velez, B. L., England, J. W., Autin, K. L., Douglass, R. P., Allan, B. A., & Blustein, D. L. (2018) · Journal of Counseling Psychology · doi:10.1037/cou0000247 · Cited by 109
Menard, P., Warkentin, M. E., & Lowry, P. B. (2018) · Computers & Security · doi:10.1016/j.cose.2018.01.020 · Cited by 106
Racial/Ethnic Perceptions from Hispanic Names: Selecting Names to Test for Discrimination
Gaddis, S. M. (2017) · Socius Sociological Research for a Dynamic World · doi:10.1177/2378023117737193 · Cited by 106
Cooper, S. R., Gonthier, C., Barch, D. M., & Braver, T. S. (2017) · Frontiers in Psychology · doi:10.3389/fpsyg.2017.01482 · Cited by 112
Applicants’ Fairness Perceptions of Algorithm-Driven Hiring Procedures
Lavanchy, M., Reichert, P., Narayanan, J., & Savani, K. (2023) · Journal of Business Ethics · doi:10.1007/s10551-022-05320-w · Cited by 145
Comparison of Object Recognition Behavior in Human and Monkey
Rajalingham, R., Schmidt, K., & DiCarlo, J. J. (2015) · Journal of Neuroscience · doi:10.1523/jneurosci.0573-15.2015 · Cited by 109
Manifesto for a new (computational) cognitive revolution
Griffiths, T. L. (2014) · Cognition · doi:10.1016/j.cognition.2014.11.026 · Cited by 104
The relation between smartphone use and everyday inattention.
Marty-Dugas, J., Ralph, B. C. W., Oakman, J. M., & Smilek, D. (2017) · Psychology of Consciousness Theory Research and Practice · doi:10.1037/cns0000131 · Cited by 120
Engaging People on Climate Change: The Role of Emotional Responses
Wong‐Parodi, G., & Feygina, I. (2021) · Environmental Communication · doi:10.1080/17524032.2020.1871051 · Cited by 103
Mitra, T., Hutto, C. J., & Gilbert, E. (2015) · Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems · doi:10.1145/2702123.2702553 · Cited by 102
Event segmentation improves event memory up to one month later.
Flores, S. E., Bailey, H. R., Eisenberg, M. L., & Zacks, J. M. (2017) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0000367 · Cited by 114
Hart, C. (2020) · Journal of Language and Politics · doi:10.1075/jlp.20032.har · Cited by 116
Time spent online: Latent profile analyses of emerging adults' social media use
Scott, C. F., Bay‐Cheng, L. Y., Prince, M. A., Nochajski, T. H., & Collins, R. L. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.05.026 · Cited by 108
Exploring the relationship between video game expertise and fluid intelligence
Kokkinakis, A. V., Cowling, P., Drachen, A., & Wade, A. R. (2017) · PLoS ONE · doi:10.1371/journal.pone.0186621 · Cited by 107
Al‐Natour, S., Cavusoglu, H., Benbasat, I., & Aleem, U. (2020) · Information Systems Research · doi:10.1287/isre.2020.0931 · Cited by 113
Crowdsourcing Government: Lessons from Multiple Disciplines
Liu, H. K. (2017) · Public Administration Review · doi:10.1111/puar.12808 · Cited by 98
Leveraging position bias to improve peer recommendation
Lerman, K., & Hogg, T. (2014) · PLoS ONE · doi:10.1371/journal.pone.0098914 · Cited by 96
How should sugar-sweetened beverage health warnings be designed? A randomized experiment
Grummon, A. H., Hall, M. G., Taillie, L. S., & Brewer, N. T. (2019) · Preventive Medicine · doi:10.1016/j.ypmed.2019.02.010 · Cited by 106
Rodd, J. M. (2023) · Journal of Memory and Language · doi:10.1016/j.jml.2023.104472 · Cited by 117
An integrative account of constraints on cross-situational learning
Yurovsky, D., & Frank, M. C. (2015) · Cognition · doi:10.1016/j.cognition.2015.07.013 · Cited by 97
Compassion for climate change victims and support for mitigation policy
Lu, H., & Schuldt, J. P. (2016) · Journal of Environmental Psychology · doi:10.1016/j.jenvp.2016.01.007 · Cited by 97
Can we turn shirkers into workers?
Berinsky, A. J., Margolis, M. F., & Sances, M. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.09.010 · Cited by 96
Hart, C. (2016) · Applied Linguistics · doi:10.1093/applin/amw017 · Cited by 96
A Gamification Framework for Cognitive Assessment and Cognitive Training: Qualitative Study
Khaleghi, A., Aghaei, Z., & Mahdavi, M. A. (2021) · JMIR Serious Games · doi:10.2196/21900 · Cited by 110
When is it biased?: assessing the representativeness of twitter's streaming API
Morstatter, F., Pfeffer, J., & Liu, H. (2014) · Proceedings of the 23rd International Conference on World Wide Web · doi:10.1145/2567948.2576952 · Cited by 90
Online crowdsourcing for efficient rating of speech: A validation study
Byun, T. M., Halpin, P. F., & Szeredi, D. (2014) · Journal of Communication Disorders · doi:10.1016/j.jcomdis.2014.11.003 · Cited by 95
Malle, B. F., Scheutz, M., Forlizzi, J., & Voiklis, J. (2016) · 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI) · doi:10.1109/hri.2016.7451743 · Cited by 107
Wolf, L. J., zu Ermgassen, S., Balmford, A. P., White, M. P., & Weinstein, N. (2017) · PLoS ONE · doi:10.1371/journal.pone.0170225 · Cited by 103
Gleibs, I. H. (2016) · Behavior Research Methods · doi:10.3758/s13428-016-0789-y · Cited by 93
Hazée, S., Vaerenbergh, Y. V., Delcourt, C., & Warlop, L. (2019) · Journal of Service Research · doi:10.1177/1094670519838622 · Cited by 93
Pollet, T. V., & Saxton, T. K. (2019) · Evolutionary Psychological Science · doi:10.1007/s40806-019-00192-2 · Cited by 89
Virtual reality studies outside the laboratory
Mottelson, A., & Hornbæk, K. (2017) · ACM Symposium on Virtual Reality Software and Technology (VRST) · doi:10.1145/3139131.3139141 · Cited by 90
Online versus offline: The Web as a medium for response time data collection
Chetverikov, A., & Upravitelev, P. (2015) · Behavior Research Methods · doi:10.3758/s13428-015-0632-x · Cited by 79
Effort, luck, and voting for redistribution
Lefgren, L. J., Sims, D., & Stoddard, O. B. (2016) · Journal of Public Economics · doi:10.1016/j.jpubeco.2016.08.012 · Cited by 90
Sustained representation of perspectival shape
Morales, J., Bax, A., & Firestone, C. (2020) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2000715117 · Cited by 93
Smartphones and the Neuroscience of Mental Health
Gillan, C. M., & Rutledge, R. B. (2021) · Annual Review of Neuroscience · doi:10.1146/annurev-neuro-101220-014053 · Cited by 95
Çobanoğlu, C., Çavuşoğlu, M., & Türktarhan, G. (2021) · Journal of Global Business Insights · doi:10.5038/2640-6489.6.1.1177 · Cited by 98
King, S. P., & Bryant, F. B. (2016) · Journal of Organizational Behavior · doi:10.1002/job.2118 · Cited by 108
Participant Nonnaiveté and the reproducibility of cognitive psychology
Zwaan, R. A., Pecher, D., Paolacci, G., Bouwmeester, S., Verkoeijen, P. P. J. L., Katinka Dijkstra, B., & Zeelenberg, R. (2017) · Psychonomic Bulletin & Review · doi:10.3758/s13423-017-1348-y · Cited by 89
Quality versus quantity of social ties in experimental cooperative networks
Shirado, H., Fu, F., Fowler, J. H., & Christakis, N. A. (2013) · Nature Communications · doi:10.1038/ncomms3814 · Cited by 88
Forwood, S., Ahern, A. L., Marteau, T. M., & Jebb, S. A. (2015) · International Journal of Behavioral Nutrition and Physical Activity · doi:10.1186/s12966-015-0241-1 · Cited by 88
Lu, H. (2015) · Science Communication · doi:10.1177/1075547015619173 · Cited by 93
Using Tablets to Collect Data From Young Children
Frank, M. C., Sugarman, E., Horowitz, A. C., Lewis, M. L., & Yurovsky, D. (2016) · Journal of Cognition and Development · doi:10.1080/15248372.2015.1061528 · Cited by 87
Examining aging sexual stigma attitudes among adults by gender, age, and generational status
Syme, M. L., & Cohn, T. J. (2015) · Aging & Mental Health · doi:10.1080/13607863.2015.1012044 · Cited by 80
Individual Differences in Learning Social and Non-Social Network Structures
Tompson, S., Kahn, A. E., Falk, E., Vettel, J., & Bassett, D. (2017) · Journal of Experimental Psychology. Learning, Memory and Cognition · doi:10.1037/xlm0000580 · Cited by 67
A primer on running human behavioural experiments online
Grootswagers, T. (2020) · Behavior Research Methods · doi:10.3758/s13428-020-01395-3 · Cited by 86
Social Support Predicts Hearing Aid Satisfaction
Singh, G., Lau, S.-T., & Pichora-Fuller, M. K. (2015) · Ear and Hearing · doi:10.1097/aud.0000000000000182 · Cited by 86
Pfattheicher, S., & Schindler, S. (2016) · PLoS ONE · doi:10.1371/journal.pone.0153419 · Cited by 84
Time reordered: Causal perception guides the interpretation of temporal order
Bechlivanidis, C., & Lagnado, D. A. (2015) · Cognition · doi:10.1016/j.cognition.2015.09.001 · Cited by 87
Wang, Q. J., Woods, A. T., & Spence, C. J. (2015) · i-Perception · doi:10.1177/2041669515622001 · Cited by 85
Cipora, K., Soltanlou, M., Reips, U., & Nuerk, H. (2019) · Behavior Research Methods · doi:10.3758/s13428-019-01213-5 · Cited by 83
Humans can decipher adversarial images
Zhou, Z., & Firestone, C. (2019) · RePEc: Research Papers in Economics · Cited by 87
Reflexive intergroup bias in third-party punishment.
Yudkin, D. A., Rothmund, T., Twardawski, M., Thalla, N. L., & Bavel, J. J. V. (2016) · Journal of Experimental Psychology General · doi:10.1037/xge0000190 · Cited by 100
Auditory interfaces in automated driving: an international survey
Bazilinskyy, P., & de Winter, J. (2015) · PeerJ Computer Science · doi:10.7717/peerj-cs.13 · Cited by 83
On cognitive preferences and the plausibility of rule-based models
Fürnkranz, J., Kliegr, T., & Paulheim, H. (2019) · Machine Learning · doi:10.1007/s10994-019-05856-5 · Cited by 96
Davidson, C. A., Hoffman, L., & Spaulding, W. D. (2016) · Psychiatry Research · doi:10.1016/j.psychres.2016.01.053 · Cited by 81
Survey Satisficing Inflates Reliability and Validity Measures
Hamby, T., & Taylor, W. (2016) · Educational and Psychological Measurement · doi:10.1177/0013164415627349 · Cited by 94
Mueller, M. L., Dunlosky, J. T., & Tauber, S. K. (2015) · Quarterly Journal of Experimental Psychology · doi:10.1080/17470218.2015.1058404 · Cited by 80
Chivers, L. L., Hand, D. J., Priest, J. S., & Higgins, S. T. (2016) · Preventive Medicine · doi:10.1016/j.ypmed.2016.07.029 · Cited by 77
Behavioral economic substitutability of e-cigarettes, tobacco cigarettes, and nicotine gum
Johnson, M. W., Johnson, P. S., Rass, O., & Pacek, L. R. (2017) · Journal of Psychopharmacology · doi:10.1177/0269881117711921 · Cited by 76
The Past, Present, and Future of Experimental Methods in the Social Sciences
Mize, T. D., & Manago, B. (2022) · Social Science Research · doi:10.1016/j.ssresearch.2022.102799 · Cited by 76
Menard, P., & Bott, G. J. (2024) · Information Systems Journal · doi:10.1111/isj.12544 · Cited by 97
Torfs, K., Vancleef, K., Lafosse, C., Wagemans, J. P., & de‐Wit, L. (2013) · Behavior Research Methods · doi:10.3758/s13428-013-0382-6 · Cited by 71
A Comparison of Actual and Perceived Sexual Risk Among Older Adults
Syme, M. L., Cohn, T. J., & Barnack‐Tavlaris, J. L. (2016) · The Journal of Sex Research · doi:10.1080/00224499.2015.1124379 · Cited by 75
Moving outside the lab: The viability of conducting sensorimotor learning studies online
Tsay, J. S., Lee, A., Ivry, R. B., & Avraham, G. (2021) · Neurons Behavior Data analysis and Theory · doi:10.51628/001c.26985 · Cited by 73
Woo, S. E., Keith, M. G., & Thornton, M. A. (2015) · Industrial and Organizational Psychology · doi:10.1017/iop.2015.21 · Cited by 75
A panel for lemons? Positivity bias, reputation systems and data quality on MTurk
Matherly, T. (2019) · European Journal of Marketing · doi:10.1108/ejm-07-2017-0491 · Cited by 79
Sex differences in the Simon task help to interpret sex differences in selective attention
Stoet, G. (2016) · Psychological Research · doi:10.1007/s00426-016-0763-4 · Cited by 77
Online cognition: factors facilitating reliable online neuropsychological test results
Feenstra, H. E. M., Vermeulen, I., Murre, J. M. J., & Schagen, S. B. (2016) · The Clinical Neuropsychologist · doi:10.1080/13854046.2016.1190405 · Cited by 70
Humans can decipher adversarial images
Zhou, Z., & Firestone, C. (2019) · Nature Communications · doi:10.1038/s41467-019-08931-6 · Cited by 121
ARDIS: a Swedish historical handwritten digit dataset
Kusetoğulları, H., Yavariabdi, A., Cheddad, A., Grahn, H., & Hall, J. (2019) · Neural Computing and Applications · doi:10.1007/s00521-019-04163-3 · Cited by 73
Lie prevalence, lie characteristics and strategies of self-reported good liars
Verigin, B. L., Meijer, E. H., Bogaard, G., & Vrij, A. (2019) · PLoS ONE · doi:10.1371/journal.pone.0225566 · Cited by 88
Metadata
- Authors
- Crump, M. J. C., McDonnell, J. V., & Gureckis, T. M.
- Affiliations
- City University of New York (US)
- New York University (US)
- Corresponding author
- Crump, Matthew J. C.
- Venue
- PLoS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2013-03-13
- Volume
- 8
- Issue
- 3
- Pages
- e57410
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC3596391
- OpenAlex
- W2080211902
- Semantic Scholar
- d59101076dd96b6b39f58c3d3e3747e5cf5f98ab
- Cited by
- 1,617
- References
- 52
- Citations by year
- 2026: 25 · 2025: 63 · 2024: 83 · 2023: 108 · 2022: 162 · 2021: 171 · 2020: 164 · 2019: 138 · 2018: 164 · 2017: 155 · 2016: 146 · 2015: 129 · 2014: 86 · 2013: 20
- FWCI
- 192.44
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Locations
- PLoS ONE (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- CUNY Academic Works (City University of New York) (submittedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Life Sciences › Neuroscience › Cognitive Neuroscience › Neural and Behavioral Psychology Studies
- Social Sciences › Economics, Econometrics and Finance › Economics and Econometrics › Experimental Behavioral Economics Studies
- Keywords
- Amazon Mechanical Turk, online crowdsourcing, replication, Stroop task, attentional blink, subliminal priming, category learning, Simon task, flanker task, stimulus presentation
- MeSH terms
- Attention, Cues, Humans, Learning, Reaction Time, Internet, Behavioral Research, Learning Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
