Turking in the time of COVID
Open accessAbstract
On March 16, 2020, the US Government introduced strict social distancing protocols for the United States in an effort to stem the spread of the COVID-19 pandemic. This had an immediate major effect on the job market, with millions of Americans forced to find alternative ways to make a living from home. As online labor markets like Amazon Mechanical Turk (MTurk) play a major role in social science research, concerns have been raised that the pandemic may be reducing the diversity of subjects participating in experiments. Here, we investigate this possibility empirically. Specifically, we look at 15,539 responses gathered in 23 studies run on MTurk between February and July 2020, examining the distribution of gender, age, ethnicity, political preference, and analytic cognitive style. We find notable changes on some of the measures following the imposition of nationwide social distancing: participants are more likely to be less reflective (as measured by the Cognitive Reflection Test), and somewhat less likely to be white, Democrats (traditionally over-represented on MTurk), and experienced with MTurk. Most of these differences are explained by an influx of new participants into the MTurk subject pool who are more diverse and representative - but also less attentive - than previous MTurkers.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participants on MTurk are quite a bit more attentive than participants on Lucid.
“This is likely due in large part to participants on MTurk being quite a bit more attentive than participants on Lucid50”
From Accuracy prompts are a replicable and generalizable approach for reducing the spread of misinformation (2022) · cites this paper as [50]
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Cognitive Reflection and Decision Making
Frederick, S. (2005) · Journal of Economic Perspectives · doi:10.1257/089533005775196732 · Cited by 5,256 · see citing works
Pennycook, G., McPhetres, J., Zhang, Y., Lu, J. G., & Rand, D. G. (2020) · Psychological Science · doi:10.1177/0956797620939054 · Cited by 1,715 · see citing works
Self-citationReputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618 · see citing works
The Online Laboratory: Conducting Experiments in a Real Labor Market∗Preprint
Zeckhauser, R. J. (2016) · Cited by 1,454 · see citing works
Berinsky, A. J., Margolis, M. F., & Sances, M. W. (2014) · American Journal of Political Science · doi:10.1111/ajps.12081 · Cited by 951 · see citing works
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950 · see citing works
Evans, D. C., Schott, R. K., Larson, D. W., Brown, C. M., & Ryan, M. J. (2014) · Nature Communications · doi:10.1038/ncomms4289 · Cited by 942 · see citing works
Intuitive Cooperation and the Social Heuristics Hypothesis: Evidence from 15 Time Constraint Studies
Rand, D. G., Peysakhovich, A., Kraft‐Todd, G., Newman, G. E., Wurzbacher, O., Nowak, M. A., & Greene, J. D. (2013) · Nature Communications · doi:10.1038/ncomms4677 · Cited by 730 · see citing works
Self-citationThe Online Laboratory: Conducting Experiments in a Real Labor Market
Horton, J. J., Rand, D. G., & Zeckhauser, R. (2010) · National Bureau of Economic Research · doi:10.3386/w15961 · Cited by 402 · see citing works
Self-citationRobinson, J., Rosenzweig, C., Moss, A. J., & Litman, L. (2019) · PLoS ONE · doi:10.1371/journal.pone.0226394 · Cited by 203 · see citing works
Participant carelessness and fraud: Consequences for clinical research and potential solutions.
Chandler, J., Sisso, I., & Shapiro, D. N. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000479 · Cited by 124 · see citing works
No Participant Left Behind: Conducting Science During COVID-19
Lourenco, S. F., & Tasimi, A. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.05.003 · Cited by 111 · see citing works
Demographic Stability on Mechanical Turk Despite COVID-19
Moss, A. J., Rosenzweig, C., Robinson, J., & Litman, L. (2020) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2020.05.014 · Cited by 87 · see citing works
The persistence of pay inequality: The gender pay gap in an anonymous online labor market
Litman, L., Robinson, J., Rosen, Z., Rosenzweig, C., Waxman, J., & Bates, L. M. (2020) · PLoS ONE · doi:10.1371/journal.pone.0229383 · Cited by 74 · see citing works
The online laboratory: conducting experiments in a real labor market
Horton, J. J., Rand, D. G., & Zeckhauser, R. (2011) · Experimental Economics · doi:10.1007/s10683-011-9273-9 · Cited by 9 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0279720 · Cited by 1,532 · see citing works
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2021) · Information systems research · doi:10.1287/isre.2021.1079 · Cited by 474 · see citing works
Pennycook, G., & Rand, D. G. (2022) · Nature Communications · doi:10.1038/s41467-022-30073-5 · Cited by 270 · see citing works
Self-citationThe Generalizability of Online Experiments Conducted During the COVID-19 Pandemic
Peyton, K., Huber, G. A., & Coppock, A. (2021) · Journal of Experimental Political Science · doi:10.1017/xps.2021.17 · Cited by 174 · see citing works
Comparing attentional disengagement between prolific and MTurk samples
Albert, D. A., & Smilek, D. (2023) · Scientific Reports · doi:10.1038/s41598-023-46048-5 · Cited by 166 · see citing works
Moss, A. J., Rosenzweig, C., Robinson, J., Jaffe, S. N., & Litman, L. (2023) · Behavior Research Methods · doi:10.3758/s13428-022-02005-0 · Cited by 109 · see citing works
A Note on Increases in Inattentive Online Survey-Takers Since 2020
Ternovski, J., Orr, L., Kalla, J., & Aronow, P. M. (2022) · Journal of Quantitative Description Digital Media · doi:10.51685/jqd.2022.002 · Cited by 103 · see citing works
Rodríguez‐Rodríguez, I., Rodríguez, J., Shirvanizadeh, N., Ortíz, A., & Pardo-Quiles, D. (2021) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph18168578 · Cited by 77 · see citing works
The Generalizability of Online Experiments Conducted During The COVID-19 PandemicPreprint
Peyton, K., Huber, G. A., & Coppock, A. (2020) · doi:10.31235/osf.io/s45yg · Cited by 69 · see citing works
Mass support for proposals to reshape policing depends on the implications for crime and safety
Vaughn, P. E., Peyton, K., & Huber, G. A. (2022) · Criminology & Public Policy · doi:10.1111/1745-9133.12572 · Cited by 64 · see citing works
Validated names for experimental studies on race and ethnicity
Crabtree, C. D., Kim, J. Y., Gaddis, S. M., Holbein, J. B., Guage, C., & Marx, W. W. (2023) · Scientific Data · doi:10.1038/s41597-023-01947-0 · Cited by 64 · see citing works
Zickgraf, H. F., & Barrada, J. R. (2021) · Eating and Weight Disorders - Studies on Anorexia Bulimia and Obesity · doi:10.1007/s40519-021-01263-9 · Cited by 58 · see citing works
Teach Me to Explain: A Review of Datasets for Explainable NLP.Preprint
Wiegreffe, S., & Marasović, A. (2021) · arXiv (Cornell University) · Cited by 57 · see citing works
Imundo, M. N., & Rapp, D. N. (2021) · Journal of Applied Research in Memory and Cognition · doi:10.1016/j.jarmac.2021.10.002 · Cited by 49 · see citing works
(2022) · doi:10.1145/3491102.3501957 · Cited by 47 · see citing works
Emotion may predict susceptibility to fake news but emotion regulation does not seem to help
Bagó, B., Rosenzweig, L. R., Berinsky, A. J., & Rand, D. G. (2022) · Cognition & Emotion · doi:10.1080/02699931.2022.2090318 · Cited by 46 · see citing works
Self-citationSocial identity and support for defunding the police in the aftermath of George Floyd’s murder
Jackson, J., Fine, A., Bradford, B., & Trinkner, R. J. (2023) · Group Processes & Intergroup Relations · doi:10.1177/13684302221128230 · Cited by 47 · see citing works
The Burden for High-Quality Online Data Collection Lies With Researchers, Not Recruitment Platforms
Cuskley, C. F., & Sulik, J. (2024) · Perspectives on Psychological Science · doi:10.1177/17456916241242734 · Cited by 48 · see citing works
Linkages between Psychedelics and Meditation in a Population-Based Sample in the United States
Simonsson, O., & Goldberg, S. B. (2022) · Journal of Psychoactive Drugs · doi:10.1080/02791072.2021.2022816 · Cited by 41 · see citing works
Jackson, J., McKay, T. E., Cheliotis, L. K., Bradford, B., Fine, A., & Trinkner, R. J. (2021) · Law and Human Behavior · doi:10.1037/lhb0000524 · Cited by 50 · see citing works
de Oliveira, G. S., & Baggs, E. (2023) · Cambridge University Press eBooks · doi:10.1017/9781009303538 · Cited by 47 · see citing works
Calibration of cognitive tests to address the reliability paradox for decision-conflict tasks
Kucina, T., Wells, L., Lewis, I., de Salas, K., Kohl, A. T., Palmer, M. A., Sauer, J., Matzke, D., Aidman, E., & Heathcote, A. (2023) · Nature Communications · doi:10.1038/s41467-023-37777-2 · Cited by 28 · see citing works
Increasing Organ Donor Registrations with Behavioral Interventions: A Field Experiment
Robitaille, N., Mažar, N., Tsai, C. I., Haviv, A. M., & Hardy, E. (2021) · Journal of Marketing · doi:10.1177/0022242921990070 · Cited by 37 · see citing works
Byrd, N., & Białek, M. (2021) · Cognition · doi:10.1016/j.cognition.2021.104649 · Cited by 32 · see citing works
Greaney, M. L., Kunicki, Z. J., Drohan, M. M., Ward‐Ritacco, C. L., Riebe, D. A., & Cohen, S. A. (2021) · BMC Public Health · doi:10.1186/s12889-021-11294-7 · Cited by 28 · see citing works
Running online experiments using web-conferencing software
Li, J., Leider, S., Beil, D. R., & Duenyas, I. (2021) · Journal of the Economic Science Association · doi:10.1007/s40881-021-00112-w · Cited by 27 · see citing works
Love thy (partisan) neighbor: Brief befriending meditation reduces affective polarization
Simonsson, O., Narayanan, J., & Marks, J. (2021) · Group Processes & Intergroup Relations · doi:10.1177/13684302211020108 · Cited by 28 · see citing works
Is the juice worth the squeeze? Learning the marginal value of mental effort over time.
Otto, A. R., Braem, S., Silvetti, M., & Vassena, E. (2022) · Journal of Experimental Psychology General · doi:10.1037/xge0001208 · Cited by 25 · see citing works
When Truth Trumps Facts: Studies on Partisan Moral Flexibility in American Politics
Kim, M. J., Hahl, O., Poskanzer, E. J., & Zuckerman, E. W. (2024) · American Journal of Sociology · doi:10.1086/730763 · Cited by 24 · see citing works
Work experience on algorithm-based platforms: The bright and dark sides of turking
Orhan, M. A., Khelladi, I., Castellano, S., & Singh, S. K. (2022) · Technological Forecasting and Social Change · doi:10.1016/j.techfore.2022.121907 · Cited by 21 · see citing works
Hsu, T., Hoffman, L., & Thomas, E. B. K. (2023) · Psychological Assessment · doi:10.1037/pas0001214 · Cited by 21 · see citing works
Kassem, S. (2022) · Global Political Economy · doi:10.1332/uvgt2823 · Cited by 20 · see citing works
Krockow, E. M., Cheng, K. O., Maltby, J., & McElroy, E. (2023) · Communications Medicine · doi:10.1038/s43856-023-00379-6 · Cited by 19 · see citing works
Why do Experiments Fail? Six Practical Suggestions for Successful Online Experiments
Kim, J. (2023) · Journal of Travel Research · doi:10.1177/00472875231209983 · Cited by 16 · see citing works
Facial trustworthiness predicts ingroup inclusion decisions
Tracy, R. E., Wilson, J. P., Slepian, M. L., & Young, S. G. (2020) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2020.104047 · Cited by 15 · see citing works
Common knowledge promotes cooperation in the threshold public goods game by reducing uncertainty
Deutchman, P., Amir, D., Jordan, M. R., & McAuliffe, K. (2021) · Evolution and Human Behavior · doi:10.1016/j.evolhumbehav.2021.12.003 · Cited by 14 · see citing works
Attention! Do We Really Need Attention Checks?
Roth, Y., & Yakobi, O. (2024) · Journal of Behavioral Decision Making · doi:10.1002/bdm.2377 · Cited by 16 · see citing works
Bonus, J. A., Watts, J., & Francemone, C. J. (2022) · Journal of Communication · doi:10.1093/joc/jqac028 · Cited by 13 · see citing works
Social media and hiring: a survey experiment on discrimination based on online social class cues
Galos, D. R. (2023) · European Sociological Review · doi:10.1093/esr/jcad012 · Cited by 13 · see citing works
Fügener, A., Grahl, J., Gupta, A., & Ketter, W. (2019) · Social Science Research Network · doi:10.2139/ssrn.3368813 · Cited by 30 · see citing works
Digital Literacy and Vulnerability to Misinformation
Ali, A., & Qazi, I. A. (2022) · Journal of Quantitative Description Digital Media · doi:10.51685/jqd.2022.025 · Cited by 12 · see citing works
Risk attitudes: The central tendency bias
Akbari, K., Eigruber, M., & Vetschera, R. (2023) · EURO Journal on Decision Processes · doi:10.1016/j.ejdp.2023.100042 · Cited by 11 · see citing works
Shelter-in-place orders, loneliness, and collaborative behavior
Babin, J. J., Foray, M., & Hussey, A. J. (2021) · Economics & Human Biology · doi:10.1016/j.ehb.2021.101056 · Cited by 11 · see citing works
Did the COVID-19 pandemic help or hurt Donald Trump’s political fortunes?
Hart, J. (2021) · PLoS ONE · doi:10.1371/journal.pone.0247664 · Cited by 11 · see citing works
Conducting interactive experiments on Toloka
Chapkovski, P. (2023) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2023.100790 · Cited by 11 · see citing works
Worth the Risk? Greater Acceptance of Instrumental Harm Befalling Men than Women
Graso, M., Reynolds, T., & Aquino, K. (2023) · Archives of Sexual Behavior · doi:10.1007/s10508-023-02571-0 · Cited by 10 · see citing works
“Moreover, our data were collected during the earlier days of COVID-19, which could have influenced the composition or motivations of our samples (Arechar & Rand, 2021).”
MethodVillavisanis, D. F., Blum, J. D., Cheung, L., Cho, D., Bartlett, S. P. P., Swanson, J. W., & Taylor, J. A. (2023) · Plastic & Reconstructive Surgery · doi:10.1097/prs.0000000000010232 · Cited by 10 · see citing works
People can use the placement of objects to infer communicative goals
Lopez-Brau, M., & Jara‐Ettinger, J. (2020) · Cognition · doi:10.1016/j.cognition.2023.105524 · Cited by 10 · see citing works
Lee, B., Lee, K., & Hartmann, B. (2023) · Science Advances · doi:10.1126/sciadv.adi1540 · Cited by 10 · see citing works
Reforming the Police: Examining the Effect of Message Framing on Police Reform Policy Preferences
Dunbar, A., & Hanink, P. A. (2023) · Criminal Justice Policy Review · doi:10.1177/08874034231211259 · Cited by 9 · see citing works
Douthit, J. D., Kachelmeier, S. J., & Landuyt, B. W. V. (2024) · Accounting Organizations and Society · doi:10.1016/j.aos.2024.101550 · Cited by 9 · see citing works
Vogel, M., Krüger, J., & Junne, F. (2022) · International Journal of Eating Disorders · doi:10.1002/eat.23675 · Cited by 9 · see citing works
Sleep Quality Among Informal Caregivers During the COVID-19 Pandemic: A Cross-Sectional Study
Greaney, M. L., Kunicki, Z. J., Drohan, M. M., Nash, C. C., & Cohen, S. A. (2022) · Gerontology and Geriatric Medicine · doi:10.1177/23337214211057387 · Cited by 9 · see citing works
Petersen, L., & Rholes, R. (2022) · Journal of Economic Dynamics and Control · doi:10.1016/j.jedc.2022.104460 · Cited by 9 · see citing works
Minimal impact of consolidation on learned switch-readiness.
Bejjani, C., Siqi-Liu, A., & Egner, T. (2021) · Journal of Experimental Psychology Learning Memory and Cognition · doi:10.1037/xlm0001074 · Cited by 9 · see citing works
Spinelli, G., & Lupker, S. (2024) · Attention, Perception, & Psychophysics · doi:10.3758/s13414-024-02892-9 · Cited by 7 · see citing works
Love Thy (Partisan) Neighbor: Brief Befriending Meditation Reduces Affective PolarizationPreprint
Simonsson, O., Narayanan, J., & Marks, J. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3674051 · Cited by 8 · see citing works
Blakeman, J. R., Eckhardt, A. L., & Kim, M. (2022) · The Journal of Cardiovascular Nursing · doi:10.1097/jcn.0000000000000931 · Cited by 7 · see citing works
Emotional Intelligence and Collaborative Dynamics in Industry 5.0 for Human-Machine Interactions
Singh, S., Rai, M., Pandey, J. K., & Saxena, A. K. (2024) · Advances in computational intelligence and robotics book series · doi:10.4018/979-8-3693-6806-0.ch010 · Cited by 5 · see citing works
Madan, S., Savani, K., & Johar, G. V. (2025) · International Journal of Research in Marketing · doi:10.1016/j.ijresmar.2025.06.004 · Cited by 5 · see citing works
Lee, H., Pickett, J. T., Graham, A., Cullen, F. T., Jonson, C. L., Haner, M., & Sloan, M. M. (2024) · Journal of Experimental Criminology · doi:10.1007/s11292-024-09610-3 · Cited by 5 · see citing works
Mourelatos, E., Simonen, J., Hosio, S. J., Likhobaba, D., & Ustalov, D. (2024) · Journal of Business Economics · doi:10.1007/s11573-024-01196-6 · Cited by 5 · see citing works
The original sin of crowd work for human subjects research
Xia, H. (2022) · Journal of Information Communication and Ethics in Society · doi:10.1108/jices-12-2021-0126 · Cited by 6 · see citing works
Emotion may predict susceptibility to fake news but emotion regulation does not seem to helpPreprint
Bagó, B., Rosenzweig, L. R., Berinsky, A. J., & Rand, D. G. (2021) · doi:10.31234/osf.io/hm9vu · Cited by 5 · see citing works
Self-citationDupuis, M., Studer, J., Wicki, M., Marmet, S., & Gmel, G. E. (2023) · PLoS ONE · doi:10.1371/journal.pone.0286597 · Cited by 4 · see citing works
Good Cop, Bad Cop, Black Cop, White Cop: Unraveling Public Perceptions of Police Use of Force
Dunbar, A., Hanink, P. A., & Kyle, B. (2024) · Crime & Delinquency · doi:10.1177/00111287241268298 · Cited by 4 · see citing works
Huang, C., Huang, W., & Yen, H. (2023) · International Journal of Public Health · doi:10.3389/ijph.2023.1605585 · Cited by 4 · see citing works
Beyond Pandering: Investment Project Quality, Voter Support, and the Use of Investment Incentives
Jud, S. (2023) · Business and Politics · doi:10.1017/bap.2023.20 · Cited by 3 · see citing works
Fairness preferences revisited
Zhang, Y., Hoffmann, M., Sara, R., & Eckel, C. C. (2024) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2024.04.033 · Cited by 3 · see citing works
Levy, A., Nguyen, C. Q., Slepian, M. L., Gaither, S. E., Pauker, K., & Dovidio, J. F. (2022) · Personality and Social Psychology Bulletin · doi:10.1177/01461672221084537 · Cited by 3 · see citing works
When Text is Not Enough: The Processing of Text and Illustration for Emerging Medical Technologies
Huang, Q. E., & Peng, W. (2022) · Communication Studies · doi:10.1080/10510974.2022.2074490 · Cited by 3 · see citing works
Dialect on Trial: An Experimental Examination of Raciolinguistic Ideologies and Character Judgments
Dunbar, A., King, S., & Vaughn, C. R. (2024) · Race and Justice · doi:10.1177/21533687241258772 · Cited by 3 · see citing works
Use of crowdsourced images for determining 2D:4D and relationship to pro-environmental variables
Pickering, G. J., Anger, N., Baird, J., Dale, G., & Tattersall, G. J. (2022) · acta ethologica · doi:10.1007/s10211-022-00401-5 · Cited by 3 · see citing works
Schneegans, T., Bachman, M. D., Huettel, S. A., & Heekeren, H. R. (2021) · doi:10.31234/osf.io/zm3us · Cited by 4 · see citing works
The Duality of Empowerment and Marginalization in Microtask Crowdsourcing: A Replication
Hadavi, Y., Villafranca, E., & Petter, S. (2022) · AIS Transactions on Replication Research · doi:10.17705/1atrr.00073 · Cited by 2 · see citing works
People can understand IPCC visuals and are not influenced by colors
Battocletti, V., Romano, A., & Sotis, C. (2023) · Environmental Research Letters · doi:10.1088/1748-9326/acfb95 · Cited by 2 · see citing works
Reimer, T. O., & Johnson, N. (2022) · Current Psychology · doi:10.1007/s12144-022-02753-4 · Cited by 2 · see citing works
Story stimuli for instantiating true and false beliefs about the world
Salovich, N. A., Imundo, M. N., & Rapp, D. N. (2022) · Behavior Research Methods · doi:10.3758/s13428-022-01904-6 · Cited by 2 · see citing works
Narrow Bracketing in Work ChoicesPreprint
Fallucchi, F., & Kaufmann, M. (2021) · arXiv (Cornell University) · Cited by 2 · see citing works
Lives or Livelihoods? Perceived Tradeoffs and Public Demand for Non-Pharmaceutical InterventionsPreprint
Settele, S., & Shupe, C. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3589891 · Cited by 2 · see citing works
Experiences and Perceptions of Default NudgesPreprint
Michaelsen, P. (2021) · doi:10.31234/osf.io/3jm6a · Cited by 2 · see citing works
Do decision makers have subjective probabilities? An experimental testPreprint
Ronayne, D., Veneziani, R., & Zame, W. R. (2022) · SSRN Electronic Journal · doi:10.2139/ssrn.4143504 · Cited by 2 · see citing works
Attention Checks in Decisions From Experience: The Case of Checking ExperimentsPreprint
Roth, Y., & Yakobi, O. (2022) · doi:10.31234/osf.io/63qht · Cited by 2 · see citing works
Madore, K. P., Osman, A. M., Kerlan, K. R., & Schafer, R. J. (2024) · medRxiv · doi:10.1101/2024.04.05.24305401 · Cited by 2 · see citing works
Public opinion on compensating different types of wrongful convictions
Wu, S. (2023) · doi:10.54014/959r-5b0w · Cited by 2 · see citing works
Time-Horizon and Timescale Effects in the Quasi-Hyperbolic Discounting Model
Acland, D. J. (2025) · Review of Behavioral Economics · doi:10.1561/105.00000211 · Cited by 1 · see citing works
Brandt, E., Lam, J. K. H., & Conway, P. J. (2025) · Journal of Applied Social Psychology · doi:10.1111/jasp.70018 · Cited by 1 · see citing works
How Reflection Enhances Task Factuality in the Use of Large Language Models
Tarafdar, M., Adam, M., & Nguyen, L. T. (2026) · Journal of Management Information Systems · doi:10.1080/07421222.2026.2692283 · Cited by 1 · see citing works
Tragedy of the Commons in Crowd Work-Based Research
Xia, H. (2023) · ACM Journal on Responsible Computing · doi:10.1145/3626493 · Cited by 1 · see citing works
A Phenomenological Study of the Experiences of Front Liner Police Officers Infected with Covid-19
Saguran, J. B. (2023) · International Journal of Science and Management Studies (IJSMS) · doi:10.51386/25815946/ijsms-v6i4p123 · Cited by 1 · see citing works
Tahaei, M., & Vaniea, K. E. (2021) · University of Edinburgh · doi:10.7488/ds/3184 · Cited by 1 · see citing works
Lee, B., Lee, K., & Hartmann, B. S. (2023) · doi:10.31235/osf.io/5b3a4 · Cited by 2 · see citing works
Brandt, E., & Conway, P. J. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4607975 · Cited by 1 · see citing works
Does Auditor Assurance of Client Prosocial Activities Affect Auditor-Client Negotiations?Preprint
Douthit, J. D., Kachelmeier, S. J., & Landuyt, B. W. V. (2020) · SSRN Electronic Journal · doi:10.2139/ssrn.3571812
Vosburg, S. K., Faraone, S. V., Riley, E. N., Whitaker, T., Kardish, J., Baker, D., Kollins, S. H., & Rush, C. R. (2022) · Journal of Attention Disorders · doi:10.1177/10870547221112948
Mallinas, S. R., Kievit, D. L., & Plant, E. A. (2023) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2023.104489
Carvings in stone: design research for public health investigations in the age of COVID-19
Mages, M. A., Chehade, E. C., Carr, S. J., Kim, M., Mello, S., Thomas, M., & Yang, Y. (2023) · Design for Health · doi:10.1080/24735132.2023.2231167
Rodríguez‐Rodríguez, I., & Rodríguez, J. (2022) · UMA Editorial eBooks · doi:10.24310/mumaedmumaed.111
Sureka, S. K. (2022) · doi:10.31274/td-20240329-803
- Authors
- Arechar, A. A., & Rand, D. G.
- Affiliations
- Centro de Investigación y Docencia Económicas (MX)
- Massachusetts Institute of Technology (US)
- Corresponding author
- Arechar, Antonio A.
- Venue
- Behavior Research Methods
- Published
- 2021-05-07
- Volume
- 53
- Issue
- 6
- Pages
- 2591-2595
- ISSN
- 1554-351X (ISSN-L)
- 1554-3528
- PMC
- PMC8103881
- OpenAlex
- W3157385754
- Cited by
- 105
- References
- 14
- Citations by year
- 2026: 1 · 2025: 8 · 2024: 14 · 2023: 29 · 2022: 27 · 2021: 19 · 2020: 6 · 2019: 1
- FWCI
- 19.04
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Behavior Research Methods (publishedVersion)
- PubMed Central (publishedVersion)
- PubMed (publishedVersion)
- DSpace@MIT (Massachusetts Institute of Technology) (submittedVersion, cc-by-nc-sa)
- OSF Preprints (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion)
- PsyArXiv (OSF Preprints) (submittedVersion, cc-by)
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Names, Identity, and Discrimination Research
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- Amazon Mechanical Turk, COVID-19 pandemic, social distancing, digital labour markets, participant diversity, demographic representation, cognitive reflection, analytic cognitive style, gender, ethnicity, political preferences
- MeSH terms
- COVID-19, SARS-CoV-2, Attention, Humans, Social Sciences, United States, Pandemics
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
