Cited by 106Mobile Crowdsensing and CrowdsourcingNames, Identity, and Discrimination ResearchSurvey Methodology and Nonresponse

The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.

Turking in the time of COVID

Open access

Arechar, Antonio A., Rand, David G.·2021·Behavior Research Methods

DOI 10.3758/s13428-021-01588-4PMC PMC8103881

Open this paper in CoChat — free account Open PDF

Abstract

On March 16, 2020, the US Government introduced strict social distancing protocols for the United States in an effort to stem the spread of the COVID-19 pandemic. This had an immediate major effect on the job market, with millions of Americans forced to find alternative ways to make a living from home. As online labor markets like Amazon Mechanical Turk (MTurk) play a major role in social science research, concerns have been raised that the pandemic may be reducing the diversity of subjects participating in experiments. Here, we investigate this possibility empirically. Specifically, we look at 15,539 responses gathered in 23 studies run on MTurk between February and July 2020, examining the distribution of gender, age, ethnicity, political preference, and analytic cognitive style. We find notable changes on some of the measures following the imposition of nationwide social distancing: participants are more likely to be less reflective (as measured by the Cognitive Reflection Test), and somewhat less likely to be white, Democrats (traditionally over-represented on MTurk), and experienced with MTurk. Most of these differences are explained by an influx of new participants into the MTurk subject pool who are more diverse and representative - but also less attentive - than previous MTurkers.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Integrity signals

Registry

Crossref · 10.3758/s13428-021-01588-4

Cites

15 of 15 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256
Fighting COVID-19 Misinformation on Social Media: Experimental Evidence for a Scalable Accuracy-Nudge Intervention
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G., & Rand, D. G. (2020) · Psychological Science · doi:10.1177/0956797620939054 · Cited by 1,715
Self-citation
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618
The Online Laboratory: Conducting Experiments in a Real Labor Market∗Preprint
Zeckhauser, R. J. (2016) · Cited by 1,454
Separating the Shirkers from the Workers? Making Sure Respondents Pay Attention on Self‐Administered Surveys
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014) · American Journal of Political Science · doi:10.1111/ajps.12081 · Cited by 951
Nonnaïveté among Amazon Mechanical Turk workers: Consequences and solutions for behavioral researchers
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950
Correction: Corrigendum: The oldest North American pachycephalosaurid and the hidden diversity of small-bodied ornithischian dinosaurs
Evans, D. C., Schott, R. K., Larson, D. W., Brown, C. M., & Ryan, M. J. (2014) · Nature Communications · doi:10.1038/ncomms4289 · Cited by 942
Intuitive Cooperation and the Social Heuristics Hypothesis: Evidence from 15 Time Constraint Studies
Rand, D. G., Peysakhovich, A., Kraft‐Todd, G., Newman, G. E., Wurzbacher, O., Nowak, M. A., & Greene, J. D. (2013) · Nature Communications · doi:10.1038/ncomms4677 · Cited by 772
Self-citation
The Online Laboratory: Conducting Experiments in a Real Labor Market
Horton, J. J., Rand, D. G., & Zeckhauser, R. (2010) · National Bureau of Economic Research · doi:10.3386/w15961 · Cited by 402
Self-citation
Tapped out or barely tapped? Recommendations for how to harness the vast and largely unused potential of the Mechanical Turk participant pool
Robinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226394 · Cited by 203

Show all 14 references

Sorted by citations

Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0279720 · Cited by 1,532
Cognitive Challenges in Human–Artificial Intelligence Collaboration: Investigating the Path Toward Productive Delegation
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021) · Information Systems Research · doi:10.1287/isre.2021.1079 · Cited by 625
Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation
Pennycook, G., & Rand, D. G. (2022) · Nature Communications · doi:10.1038/s41467-022-30073-5 · Cited by 270
Self-citation
The Generalizability of Online Experiments Conducted During the COVID-19 Pandemic
Peyton, K., Huber, G. A., & Coppock, A. (2021) · Journal of Experimental Political Science · doi:10.1017/xps.2021.17 · Cited by 174
Comparing attentional disengagement between prolific and MTurk samples
Albert, D. A., & Smilek, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-46048-5 · Cited by 166
Is it ethical to use Mechanical Turk for behavioral research? Relevant data from a representative survey of MTurk participants and wages
Moss, A. J., Rosenzweig, C., Robinson, J., Jaffe, S. N., & Litman, L. (2023) · Behavior Research Methods · doi:10.3758/s13428-022-02005-0 · Cited by 129
A Note on Increases in Inattentive Online Survey-Takers Since 2020
Ternovski, J., Orr, L., Kalla, J., & Aronow, P. M. (2022) · Journal of Quantitative Description Digital Media · doi:10.51685/jqd.2022.002 · Cited by 103
Applications of Artificial Intelligence, Machine Learning, Big Data and the Internet of Things to the COVID-19 Pandemic: A Scientometric Review Using Text Mining
Rodríguez‐Rodríguez, I., Rodríguez, J., Shirvanizadeh, N., Ortíz, A., & Pardo-Quiles, D. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168578 · Cited by 82
The Generalizability of Online Experiments Conducted During The COVID-19 PandemicPreprint
Peyton, K., Huber, G. A., & Coppock, A. (2020) · doi:10.31235/osf.io/s45yg · Cited by 69
Mass support for proposals to reshape policing depends on the implications for crime and safety
Vaughn, P. E., Peyton, K., & Huber, G. A. (2022) · Criminology & Public Policy · doi:10.1111/1745-9133.12572 · Cited by 64
Validated names for experimental studies on race and ethnicity
Crabtree, C. D., Kim, J. Y., Gaddis, S. M., Holbein, J. B., Guage, C., & Marx, W. W. (2023) · Scientific Data · doi:10.1038/s41597-023-01947-0 · Cited by 64
Orthorexia nervosa vs. healthy orthorexia: relationships with disordered eating, eating behavior, and healthy lifestyle choices
Zickgraf, H. F., & Barrada, J. R. (2021) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-021-01263-9 · Cited by 60
Teach Me to Explain: A Review of Datasets for Explainable NLP.Preprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · Cited by 57
When fairness is flawed: Effects of false balance reporting and weight-of-evidence statements on beliefs and perceptions of climate change.
Imundo, M. N., & Rapp, D. N. (2021) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2021.10.002 · Cited by 49
Untitled work
(2022) · doi:10.1145/3491102.3501957 · Cited by 47
Emotion may predict susceptibility to fake news but emotion regulation does not seem to help
Bagó, B., Rosenzweig, L. R., Berinsky, A. J., & Rand, D. G. (2022) · Cognition & Emotion · doi:10.1080/02699931.2022.2090318 · Cited by 53
Self-citation
Social identity and support for defunding the police in the aftermath of George Floyd’s murder
Jackson, J., Fine, A., Bradford, B., & Trinkner, R. J. (2023) · Group Processes & Intergroup Relations · doi:10.1177/13684302221128230 · Cited by 47
The Burden for High-Quality Online Data Collection Lies With Researchers, Not Recruitment Platforms
Cuskley, C. F., & Sulik, J. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916241242734 · Cited by 48
Linkages between Psychedelics and Meditation in a Population-Based Sample in the United States
Simonsson, O., & Goldberg, S. B. (2022) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2021.2022816 · Cited by 41
Centering race in procedural justice theory: Structural racism and the under- and overpolicing of Black communities.
Jackson, J., McKay, T. E., Cheliotis, L. K., Bradford, B., Fine, A., & Trinkner, R. J. (2021) · Law and Human Behavior · doi:10.1037/lhb0000524 · Cited by 54
Psychology's WEIRD Problems
de Oliveira, G. S., & Baggs, E. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009303538 · Cited by 57
Calibration of cognitive tests to address the reliability paradox for decision-conflict tasks
Kucina, T., Wells, L., Lewis, I., de Salas, K., Kohl, A. T., Palmer, M. A., Sauer, J., Matzke, D., Aidman, E., & Heathcote, A. (2023) · Nature Communications · doi:10.1038/s41467-023-37777-2 · Cited by 28
Increasing Organ Donor Registrations with Behavioral Interventions: A Field Experiment
Robitaille, N., Mažar, N., Tsai, C. I., Haviv, A. M., & Hardy, E. (2021) · Journal of Marketing · doi:10.1177/0022242921990070 · Cited by 37
Your health vs. my liberty: Philosophical beliefs dominated reflection and identifiable victim effects when predicting public health recommendation compliance during the COVID-19 pandemic
Byrd, N., & Białek, M. (2021) · Cognition · doi:10.1016/j.cognition.2021.104649 · Cited by 32
Self-reported changes in physical activity, sedentary behavior, and screen time among informal caregivers during the COVID-19 pandemic
Greaney, M. L., Kunicki, Z. J., Drohan, M. M., Ward‐Ritacco, C. L., Riebe, D. A., & Cohen, S. A. (2021) · BMC Public Health · doi:10.1186/s12889-021-11294-7 · Cited by 29
Running online experiments using web-conferencing software
Li, J., Leider, S., Beil, D. R., & Duenyas, I. (2021) · Journal of the Economic Science Association · doi:10.1007/s40881-021-00112-w · Cited by 27
Love thy (partisan) neighbor: Brief befriending meditation reduces affective polarization
Simonsson, O., Narayanan, J., & Marks, J. (2021) · Group Processes & Intergroup Relations · doi:10.1177/13684302211020108 · Cited by 35
Is the juice worth the squeeze? Learning the marginal value of mental effort over time.
Otto, A. R., Braem, S., Silvetti, M., & Vassena, E. (2022) · Journal of Experimental Psychology General · doi:10.1037/xge0001208 · Cited by 29
When Truth Trumps Facts: Studies on Partisan Moral Flexibility in American Politics
Kim, M. J., Hahl, O., Poskanzer, E. J., & Zuckerman, E. W. (2024) · American Journal of Sociology · doi:10.1086/730763 · Cited by 25
Work experience on algorithm-based platforms: The bright and dark sides of turking
Orhan, M. A., Khelladi, I., Castellano, S., & Singh, S. K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121907 · Cited by 28
Confirmatory measurement modeling and longitudinal invariance of the CompACT-15: A short-form assessment of psychological flexibility.
Hsu, T., Hoffman, L., & Thomas, E. B. K. (2023) · Psychological Assessment · doi:10.1037/pas0001214 · Cited by 23
Labour realities at Amazon and COVID-19: obstacles and collective possibilities for its warehouse workers and MTurk workers
Kassem, S. (2022) · Global Political Economy · doi:10.1332/uvgt2823 · Cited by 20
Existing terminology related to antimicrobial resistance fails to evoke risk perceptions and be remembered
Krockow, E. M., Cheng, K. O., Maltby, J., & McElroy, E. (2023) · Communications Medicine · doi:10.1038/s43856-023-00379-6 · Cited by 19
Why do Experiments Fail? Six Practical Suggestions for Successful Online Experiments
Kim, J. (2023) · Journal of Travel Research · doi:10.1177/00472875231209983 · Cited by 16
Facial trustworthiness predicts ingroup inclusion decisions
Tracy, R. E., Wilson, J. P., Slepian, M. L., & Young, S. G. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104047 · Cited by 15
Common knowledge promotes cooperation in the threshold public goods game by reducing uncertainty
Deutchman, P., Amir, D., Jordan, M. R., & McAuliffe, K. (2021) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2021.12.003 · Cited by 14
Attention! Do We Really Need Attention Checks?
Roth, Y., & Yakobi, O. (2024) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2377 · Cited by 18
When “meaningless” means more: biographic resonance and audience appreciation of popular entertainment
Bonus, J. A., Watts, J., & Francemone, C. J. (2022) · Journal of Communication · doi:10.1093/joc/jqac028 · Cited by 15
Social media and hiring: a survey experiment on discrimination based on online social class cues
Galos, D. R. (2023) · European Sociological Review · doi:10.1093/esr/jcad012 · Cited by 15
Cognitive challenges in human-AI collaboration: Investigating the path towards productive delegationPreprint
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2019) · SSRN Electronic Journal · doi:10.2139/ssrn.3368813 · Cited by 17
Digital Literacy and Vulnerability to Misinformation
Ali, A., & Qazi, I. A. (2022) · Journal of Quantitative Description Digital Media · doi:10.51685/jqd.2022.025 · Cited by 12
Risk attitudes: The central tendency bias
Akbari, K., Eigruber, M., & Vetschera, R. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100042 · Cited by 17
Shelter-in-place orders, loneliness, and collaborative behavior
Babin, J. J., Foray, M., & Hussey, A. J. (2021) · Economics & Human Biology · doi:10.1016/j.ehb.2021.101056 · Cited by 12
Did the COVID-19 pandemic help or hurt Donald Trump’s political fortunes?
Hart, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247664 · Cited by 11
Conducting interactive experiments on Toloka
Chapkovski, P. (2023) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2023.100790 · Cited by 11
Worth the Risk? Greater Acceptance of Instrumental Harm Befalling Men than Women
Graso, M., Reynolds, T., & Aquino, K. (2023) · Archives of Sexual Behavior · doi:10.1007/s10508-023-02571-0 · Cited by 10
“Moreover, our data were collected during the earlier days of COVID-19, which could have influenced the composition or motivations of our samples (Arechar & Rand, 2021).”
Method
Crowdsourcing in Plastic and Reconstructive Surgery: A Systematic Review with Reporting Recommendations
Villavisanis, D. F., Blum, J. D., Cheung, L., Cho, D., Bartlett, S. P. P., Swanson, J. W., & Taylor, J. A. (2023) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000010232 · Cited by 10
People can use the placement of objects to infer communicative goals
Lopez-Brau, M., & Jara‐Ettinger, J. (2020) · Cognition · doi:10.1016/j.cognition.2023.105524 · Cited by 10
Transformation of social relationships in COVID-19 America: Remote communication may amplify political echo chambers
Lee, B., Lee, K., & Hartmann, B. (2023) · Science Advances · doi:10.1126/sciadv.adi1540 · Cited by 11
Reforming the Police: Examining the Effect of Message Framing on Police Reform Policy Preferences
Dunbar, A., & Hanink, P. A. (2023) · Criminal Justice Policy Review · doi:10.1177/08874034231211259 · Cited by 10
Does auditor assurance of client prosocial activities affect subsequent reporter-auditor negotiations?
Douthit, J. D., Kachelmeier, S. J., & Landuyt, B. W. V. (2024) · Accounting Organizations and Society · doi:10.1016/j.aos.2024.101550 · Cited by 9
Eating disorder related research using Amazon Mechanical Turk ( MTurk ): Friend or foe?: Commentary on Burnette et al. (2021)
Vogel, M., Krüger, J., & Junne, F. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23675 · Cited by 9
Sleep Quality Among Informal Caregivers During the COVID-19 Pandemic: A Cross-Sectional Study
Greaney, M. L., Kunicki, Z. J., Drohan, M. M., Nash, C. C., & Cohen, S. A. (2022) · Gerontology and Geriatric Medicine · doi:10.1177/23337214211057387 · Cited by 11
Macroeconomic expectations, central bank communication, and background uncertainty: A COVID-19 laboratory experiment
Petersen, L., & Rholes, R. (2022) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2022.104460 · Cited by 9
Minimal impact of consolidation on learned switch-readiness.
Bejjani, C., Siqi-Liu, A., & Egner, T. (2021) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0001074 · Cited by 10
A spatial version of the Stroop task for examining proactive and reactive control independently from non-conflict processes
Spinelli, G., & Lupker, S. (2024) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-024-02892-9 · Cited by 7
Love Thy (Partisan) Neighbor: Brief Befriending Meditation Reduces Affective PolarizationPreprint
Simonsson, O., Narayanan, J., & Marks, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3674051 · Cited by 8
The Lay Public's Knowledge of the Most Common Acute Coronary Syndrome Symptoms Experienced by Women and Men
Blakeman, J. R., Eckhardt, A. L., & Kim, M. (2022) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0000000000000931 · Cited by 7
Emotional Intelligence and Collaborative Dynamics in Industry 5.0 for Human-Machine Interactions
Singh, S., Rai, M., Pandey, J. K., & Saxena, A. K. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-6806-0.ch010 · Cited by 5
Over-reliance on aesthetics? The appearance-reveals-character lay theory increases consumers’ devaluation of unattractive produce
Madan, S., Savani, K., & Johar, G. V. (2025) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2025.06.004 · Cited by 5
Punitiveness toward social distancing deviance in the COVID-19 pandemic: Findings from two national experiments
Lee, H., Pickett, J. T., Graham, A., Cullen, F. T., Jonson, C. L., Haner, M., & Sloan, M. M. (2024) · Journal of Experimental Criminology · doi:10.1007/s11292-024-09610-3 · Cited by 5
How has the COVID-19 pandemic shaped behavior in crowdsourcing? The role of online labor market training
Mourelatos, E., Simonen, J., Hosio, S. J., Likhobaba, D., & Ustalov, D. (2024) · Journal of Business Economics · doi:10.1007/s11573-024-01196-6 · Cited by 5
The original sin of crowd work for human subjects research
Xia, H. (2022) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-12-2021-0126 · Cited by 6
Emotion may predict susceptibility to fake news but emotion regulation does not seem to helpPreprint
Bagó, B., Rosenzweig, L. R., Berinsky, A. J., & Rand, D. G. (2021) · doi:10.31234/osf.io/hm9vu · Cited by 5
Self-citation
Was retrospective change measurement conducted with Covid-19 containment inconsistent? Comparing prospective and retrospective change measures using data from a national survey on substance use and addictive behaviors
Dupuis, M., Studer, J., Wicki, M., Marmet, S., & Gmel, G. E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286597 · Cited by 4
Good Cop, Bad Cop, Black Cop, White Cop: Unraveling Public Perceptions of Police Use of Force
Dunbar, A., Hanink, P. A., & Kyle, B. (2024) · Crime & Delinquency · doi:10.1177/00111287241268298 · Cited by 4
An Exploration of Sedentary Behavior, Physical Activity, and Quality of Life During the COVID-19 Outbreak
Huang, C., Huang, W., & Yen, H. (2023) · International Journal of Public Health · doi:10.3389/ijph.2023.1605585 · Cited by 4
Beyond Pandering: Investment Project Quality, Voter Support, and the Use of Investment Incentives
Jud, S. (2023) · Business and Politics · doi:10.1017/bap.2023.20 · Cited by 3
Fairness preferences revisited
Zhang, Y., Hoffmann, M., Sara, R., & Eckel, C. C. (2024) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2024.04.033 · Cited by 3
Categorizing a Face and Facing a Category: The Constructive Impacts of Ambiguity and Uncertainty in Racial Categorization
Levy, A., Nguyen, C. Q., Slepian, M. L., Gaither, S. E., Pauker, K., & Dovidio, J. F. (2022) · Personality and Social Psychology Bulletin · doi:10.1177/01461672221084537 · Cited by 3
When Text is Not Enough: The Processing of Text and Illustration for Emerging Medical Technologies
Huang, Q. E., & Peng, W. (2022) · Communication Studies · doi:10.1080/10510974.2022.2074490 · Cited by 3
Dialect on Trial: An Experimental Examination of Raciolinguistic Ideologies and Character Judgments
Dunbar, A., King, S., & Vaughn, C. R. (2024) · Race and Justice · doi:10.1177/21533687241258772 · Cited by 3
Use of crowdsourced images for determining 2D:4D and relationship to pro-environmental variables
Pickering, G. J., Anger, N., Baird, J., Dale, G., & Tattersall, G. J. (2022) · acta ethologica · doi:10.1007/s10211-022-00401-5 · Cited by 3
Exploring the Potential of Online Webcam-based Eye Tracking in Decision-Making Research and Influence Factors on Data QualityPreprint
Schneegans, T., Bachman, M. D., Huettel, S. A., & Heekeren, H. R. (2021) · doi:10.31234/osf.io/zm3us · Cited by 4
The Duality of Empowerment and Marginalization in Microtask Crowdsourcing: A Replication
Hadavi, Y., Villafranca, E., & Petter, S. (2022) · AIS Transactions on Replication Research · doi:10.17705/1atrr.00073 · Cited by 2
People can understand IPCC visuals and are not influenced by colors
Battocletti, V., Romano, A., & Sotis, C. (2023) · Environmental Research Letters · doi:10.1088/1748-9326/acfb95 · Cited by 2
Public support for counterterrorism efforts using probabilistic computing technologies to decipher terrorist communication on the internet
Reimer, T. O., & Johnson, N. (2022) · Current Psychology · doi:10.1007/s12144-022-02753-4 · Cited by 2
Story stimuli for instantiating true and false beliefs about the world
Salovich, N. A., Imundo, M. N., & Rapp, D. N. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01904-6 · Cited by 2
Narrow Bracketing in Work ChoicesPreprint
Fallucchi, F., & Kaufmann, M. (2021) · arXiv (Cornell University) · Cited by 2
Lives or Livelihoods? Perceived Tradeoffs and Public Demand for Non-Pharmaceutical InterventionsPreprint
Settele, S., & Shupe, C. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3589891 · Cited by 2
Experiences and Perceptions of Default NudgesPreprint
Michaelsen, P. (2021) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/3jm6a · Cited by 2
Do decision makers have subjective probabilities? An experimental testPreprint
Ronayne, D., Veneziani, R., & Zame, W. R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4143504 · Cited by 2
Attention Checks in Decisions From Experience: The Case of Checking ExperimentsPreprint
Roth, Y., & Yakobi, O. (2022) · doi:10.31234/osf.io/63qht · Cited by 2
Reliability and Validity of the Brief Attention and Mood Scale of 7 Items (BAMS-7): A Self-Administered, Online AssessmentPreprint
Madore, K. P., Osman, A. M., Kerlan, K. R., & Schafer, R. J. (2024) · medRxiv · doi:10.1101/2024.04.05.24305401 · Cited by 2
Public opinion on compensating different types of wrongful convictions
Wu, S. (2023) · doi:10.54014/959r-5b0w · Cited by 2
Time-Horizon and Timescale Effects in the Quasi-Hyperbolic Discounting Model
Acland, D. J. (2025) · Review of Behavioral Economics · doi:10.1561/105.00000211 · Cited by 1
Why Efficient Leaders Risk Appearing as Jerks: People Expect Authority Figures to Accept Organizational Sacrificial Harm, but View Doing so as Cold and Immoral
Brandt, E., Lam, J. K. H., & Conway, P. J. (2025) · Journal of Applied Social Psychology · doi:10.1111/jasp.70018 · Cited by 1
How Reflection Enhances Task Factuality in the Use of Large Language Models
Tarafdar, M., Adam, M., & Nguyen, L. T. (2026) · Journal of Management Information Systems · doi:10.1080/07421222.2026.2692283 · Cited by 1
Tragedy of the Commons in Crowd Work-Based Research
Xia, H. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3626493 · Cited by 1
A Phenomenological Study of the Experiences of Front Liner Police Officers Infected with Covid-19
Saguran, J. B. (2023) · International Journal of Science and Management Studies (IJSMS) · doi:10.51386/25815946/ijsms-v6i4p123 · Cited by 1
Recruiting Participants With Programming Skills: A Comparison of Four Crowdsourcing Platforms and a CS Student Mailing List
Tahaei, M., & Vaniea, K. E. (2021) · University of Edinburgh · doi:10.7488/ds/3184 · Cited by 1
Transformation of social relationships in COVID-19 America: Remote communication may amplify political echo chambersPreprint
Lee, B., Lee, K., & Hartmann, B. S. (2023) · SocArXiv (OSF Preprints) · doi:10.31235/osf.io/5b3a4 · Cited by 2
Why Efficient Leaders Come Across as Jerks: A Moral Dilemma Perspective on Sacrificial Decisions in Organizational SettingsPreprint
Brandt, E., & Conway, P. J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4607975 · Cited by 1
Does Auditor Assurance of Client Prosocial Activities Affect Auditor-Client Negotiations?Preprint
Douthit, J. D., Kachelmeier, S. J., & Landuyt, B. W. V. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3571812
Intranasal Use of Prescription Stimulants Among Adults Aged 18 to 30: Results From A Crowdsourcing Platform
Vosburg, S. K., Faraone, S. V., Riley, E. N., Whitaker, T., Kardish, J., Baker, D., Kollins, S. H., & Rush, C. R. (2022) · Journal of Attention Disorders · doi:10.1177/10870547221112948
Trial by ideology: Ideological differences in responses to errors in determining guilt in the United States
Mallinas, S. R., Kievit, D. L., & Plant, E. A. (2023) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2023.104489
Carvings in stone: design research for public health investigations in the age of COVID-19
Mages, M. A., Chehade, E. C., Carr, S. J., Kim, M., Mello, S., Thomas, M., & Yang, Y. (2023) · Design for Health · doi:10.1080/24735132.2023.2231167
Tecnologías digitales disruptivas aplicadas a la gestión de la pandemia por COVID-19: Un análisis a través de la producción científica
Rodríguez‐Rodríguez, I., & Rodríguez, J. (2022) · UMA Editorial eBooks · doi:10.24310/mumaedmumaed.111
Revisiting ‘Growth and Inequality in Public Good Provision’—Reproducing and Generalizing Through Inconvenient Online ExperimentationPreprint
Roggenkamp, H. (2024) · doi:10.31219/osf.io/6rn97
Three essays on migration
Sureka, S. K. (2022) · doi:10.31274/td-20240329-803

Authors: Arechar, A. A., & Rand, D. G.
Affiliations: Centro de Investigación y Docencia Económicas (MX)
Massachusetts Institute of Technology (US)
Corresponding author: Arechar, Antonio A.
ORCID: Arechar, Antonio A.
Rand, David G.
Venue: Behavior Research Methods
Publisher: Springer Science and Business Media LLC
Published: 2021-05-07
Volume: 53
Issue: 6
Pages: 2591-2595
ISSN: 1554-351X (ISSN-L)
1554-3528
DOI: 10.3758/s13428-021-01588-4
PMC: PMC8103881
OpenAlex: W3157385754
Cited by: 105
References: 14
Citations by year: 2026: 1 · 2025: 8 · 2024: 14 · 2023: 29 · 2022: 27 · 2021: 19
FWCI: 19.04
Type: journal article
Language: en
Open access: bronze
doi.org
Licence: www.springer.com/tdm (tdm)
www.springer.com/tdm (vor)
Locations: Behavior Research Methods (publishedVersion)
PubMed Central (publishedVersion)
PubMed (publishedVersion)
DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
OSF Preprints (OSF Preprints) (submittedVersion)
PsyArXiv (OSF Preprints) (submittedVersion)
PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
Topics: Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
Keywords: Amazon Mechanical Turk, COVID-19 pandemic, social distancing, digital labour markets, participant diversity, demographic representation, cognitive reflection, analytic cognitive style, gender, ethnicity, political preferences
MeSH terms: COVID-19, SARS-CoV-2, Attention, Humans, Social Sciences, United States, Pandemics

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Turking in the time of COVID

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Self-check

Citations

Cites

Cited by

Related

Metadata

Record

About these checks