Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher’s question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our primary concern was the potential compromise in data quality and representativeness often associated with these platforms, as highlighted in recent studies [23,35,68].”
From Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models (2024) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Combined with superior prescreening tools, transparent demographic targeting, and cost-efficiency (Douglas et al., 2023), these strengths make prolific particularly well-suited for research in educational psychology.”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prolific participants show higher attentiveness, longer and more appropriate completion times, fewer signs of disengagement, and greater response consistency (Albert and Smilek, 2023; Douglas et al., 2023).”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Douglas et al. found that the data quality of samples recruited on Prolific is higher than on other labour markets and student samples, with participants paying more attention to the tasks and providing more meaningful answers [22].”
From Cognitive Forcing for Better Decision-Making: Reducing Overreliance on AI Systems Through Partial Explanations (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Prolific and CloudResearch users consistently outperform Amazon Mechanical Turk (MTurk), Qualtrics, and SONA users on key data quality indicators (Douglas et al., 2023).”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · cited 1×
“, to account for the fact that we already “used up” all available degrees of freedom for the test of our a priori hypotheses) [43].”
BackgroundAmazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
The international personality item pool and the future of public-domain personality measures
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006) · Journal of Research in Personality · doi:10.1016/j.jrp.2005.08.007 · Cited by 3,185
(1999) · Medical Entomology and Zoology · Cited by 3,151
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 3×
“[19] observed that MTurk (the platform with the most experienced workers) had worse data quality than Prolific (one of two platforms with less experienced workers) but better data quality than CrowdFlower (the other platform with less experienced workers).”
BackgroundInfluentialData quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Detecting and Deterring Insufficient Effort Responding to Surveys
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-011-9231-8 · Cited by 1,295
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
Chmielewski, M., & Kucker, S. C. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619875149 · Cited by 1,027 · cited 3×
“, Cronbach’s alphas and internal validity between scales) [2].”
BackgroundCoppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
MTurk Research: Review and Recommendations
Aguinis, H., Villamor, I., & Ramani, R. S. (2021) · Journal of Management · doi:10.1177/0149206320969787 · Cited by 984
Research works printed but not linked: 1
- Ternovski J, Orr L. A Note on Increases in Inattentive Online Survey-Takers Since 2020. Journal of Quantitative Description: Digital Media. 2022; 2: 1– 35.
Cited by
Items 901–1,000 of 1,004
Dunn, M. S., & Occhipinti, S. (2026) · International Journal of Social Robotics · doi:10.1007/s12369-026-01412-z · Cited by 1
The Bridge of Hope: Mediating the Relationship Between Mindfulness and Loneliness
Gutiérrez, D., Poandl, M. M., & Kitching, E. (2026) · The Journal of Humanistic Counseling · doi:10.1002/johc.70050 · Cited by 1
Mourtgos, S. M., Adams, I. T., McLean, K. D., Stoughton, S. W., & Alpert, G. P. (2026) · Police Quarterly · doi:10.1177/10986111261462411 · Cited by 1
van Knippenberg, D., Mistry, S., Kirkman, B. L., Hirst, G., & Li, J. (2026) · Journal of Business and Psychology · doi:10.1007/s10869-026-10136-w · Cited by 1
Li, X., Cui, W., Olatunji, S., & Piercy, C. W. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2692012 · Cited by 1
Raubvogel, S., de la Fuente, M. M., Chicoine, M., & Kwofie, E. M. (2026) · Business Strategy and the Environment · doi:10.1002/bse.71217 · Cited by 1
Korankye, B., Yi, K., Borah, P. S., Odai, L. A., Kodua, L. T., & Amankona, D. (2026) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-12-2025-1452 · Cited by 1
Roe, B. E., Xu, L., Martin, C. K., & Apolzan, J. W. (2026) · Waste Management · doi:10.1016/j.wasman.2026.115727 · Cited by 1
AI-Driven Leadership and the Innovation Paradox: More Innovation Activity, Lower Innovation Quality
Mirčetić, V., Pavlović, Z. S., Aleksandar Ignjatović, P., Vujko, A., & Kovačević, I. (2026) · World · doi:10.3390/world7070120 · Cited by 1
Siegrist, M., Wassmann, B., Berthold, A., Hässig-Wegmann, A., Luchsinger, L., Michel, F., & Zhou, X. (2026) · Food Quality and Preference · doi:10.1016/j.foodqual.2026.106062 · Cited by 2
Intergenerational Contact and Ageist Beliefs: Perspective-Taking as a Mediator
Swanson, H. L., Dewey, E., & Sheehan, C. (2026) · The International Journal of Aging and Human Development · doi:10.1177/00914150261467665 · Cited by 1
Mirčetić, V., Aleksandar Ignjatović, P., Milojević, S., Vujko, A., & Životić, I. (2026) · Administrative Sciences · doi:10.3390/admsci16080366 · Cited by 2
Tan, W., Cruse, J., Summers, D. M., & Fritz, Z. (2026) · Health Expectations · doi:10.1111/hex.70765 · Cited by 1
Chen, X., Presno, D., Tao, J., & Chen, L. (2026) · Journal of Cancer Survivorship · doi:10.1007/s11764-026-02101-6 · Cited by 1
Health equity readiness index for digital health organizations
Wagon, L., Nißen, M., & Kowatsch, T. (2026) · International Journal for Equity in Health · doi:10.1186/s12939-026-02964-2 · Cited by 1
Beyond transactions: engagement with generative AI chatbots and retail channel choice
Assadi, A. E. (2026) · The International Review of Retail Distribution and Consumer Research · doi:10.1080/09593969.2026.2713583 · Cited by 1
Hübner, V., & Pfost, M. (2026) · Measurement Instruments for the Social Sciences · doi:10.5964/miss.20721 · Cited by 1
Loaiza-Ramírez, J. P., Reimer, T. O., Todd, P. M., Qian, X., Gazmeh, H., Hamim, O. F., & Ukkusuri, S. V. (2026) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2026.2727258 · Cited by 1
Morgan, M. A., Logan, M. W., Connealy, N. T., & Dulisse, B. C. (2026) · Journal of Contemporary Criminal Justice · doi:10.1177/10439862261485727 · Cited by 1
Krispenz, A., & Bertrams, A. (2026) · Current Psychology · doi:10.1007/s12144-026-10086-9 · Cited by 1
Guzzo, R. F. (2026) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-02-2026-0264 · Cited by 1
Suh, T., Nasta, L., Pirolo, L., & Magnanelli, B. S. (2026) · Journal of Services Marketing · doi:10.1108/jsm-07-2025-0445 · Cited by 1
Tamaki, R., & Baltiansky, D. (2024) · Journal of Emerging Investigators · doi:10.59720/23-083 · Cited by 2
Promises and pitfalls of #Relapse narratives in Destigmatization: The mediating role of emotions
Lu, H. (2024) · Addictive Behaviors Reports · doi:10.1016/j.abrep.2024.100551 · Cited by 2
Tanase, R., Algesheimer, R., & Mariani, M. S. (2026) · Nature Human Behaviour · doi:10.1038/s41562-026-02417-4 · Cited by 6
Improving data quality in online parenting research
Kaylor‐Tapscott, M. L., Tolliver‐Lynn, M. N., & Sullivan, M. A. (2024) · Infant and Child Development · doi:10.1002/icd.2525 · Cited by 1
Schlechter, P., Bryant, R. A., & Morina, N. (2024) · Child Abuse & Neglect · doi:10.1016/j.chiabu.2024.106938 · Cited by 1
How to Weaken the Endowment Effect in the Housing Market? The Role of Behavioral Interventions
Tomal, M. (2024) · Real Estate Management and Valuation · doi:10.2478/remav-2024-0038 · Cited by 1
Colby, H., Popovich, D. L., & Stovall, T. (2024) · Medical Decision Making · doi:10.1177/0272989x241275645 · Cited by 1
Gender vs. personality: The role of masculinity in explaining cognitive style
Plotkina, D., Hoffmann, A. O. I., Roger, P., & D’Hondt, C. (2024) · Journal of Behavioral and Experimental Finance · doi:10.1016/j.jbef.2024.100995 · Cited by 1
News Consumers’ Expectations of Journalists Vary by Differences in Journalistic Role Preference
Farrish, K., Munno, G., & Craig, M. (2024) · Journalism & Mass Communication Quarterly · doi:10.1177/10776990241287159 · Cited by 1
Humans and robots are nearly ethically equivalent
Jorgenson, C., Willems, J., Özkes, A., & Vanderelst, D. F. (2024) · AI and Ethics · doi:10.1007/s43681-024-00603-6 · Cited by 1
Indirect restrictions demobilize supporters of abortion rights
Connors, E. C., Ponte, A. D., & DeScioli, P. (2024) · Policy Studies Journal · doi:10.1111/psj.12575 · Cited by 1
Elphinstone, B., Walshe, J., Nicol, D., & Taylor, M. (2024) · Frontiers in Public Health · doi:10.3389/fpubh.2024.1508261 · Cited by 1
Is In-Game Purchase a Guilty Pleasure? Negative Beliefs and Treatment
Li, M., Guo, H., Lee, G.-S., & Shah, R. (2024) · Production and operations management · doi:10.1177/10591478241310221
Harney, J. (2024) · Social Policy and Administration · doi:10.1111/spol.13118 · Cited by 2
Color-Name Aware Optimization to Enhance the Perception of Transparent Overlapped Charts
Lu, K., Zhu, L., Wang, Y., Zeng, Q., Song, W., & Reda, K. (2024) · IEEE Transactions on Visualization and Computer Graphics · doi:10.1109/tvcg.2024.3520219 · Cited by 1
The effects of brief mindfulness training on monetary and In-kind donation decisions
Imbriale, P., Stavrulaki, E., & Surti, C. (2025) · Current Psychology · doi:10.1007/s12144-024-07218-4 · Cited by 1
Determinants of Economic Risk Preferences Across Adolescence
Zhang, Y., Camerer, C. F., & Tashjian, S. M. (2025) · Journal of Behavioral Decision Making · doi:10.1002/bdm.70007 · Cited by 1
Integrating Actual Human Behavior into an Agent-Based School Shooting Simulation
Kapadia, K., Yongsatianchot, N., Marsella, S., & John, R. S. (2024) · Winter Simulation Conference · doi:10.1109/wsc63780.2024.10838721 · Cited by 2
Wiradhany, W., Djalal, F. M., & de Bruin, A. B. H. (2025) · Accountability in Research · doi:10.1080/08989621.2025.2457100 · Cited by 2
Badas, A., & Schmidt, E. R. (2025) · Political Research Quarterly · doi:10.1177/10659129251321434 · Cited by 1
Little, K. K., Evans, S. C., Malik, N., & Lindahl, K. M. (2025) · LGBTQ+ Family An Interdisciplinary Journal · doi:10.1080/27703371.2025.2490128 · Cited by 1
Jiang, X., & Harvey, N. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1448153 · Cited by 1
Zhao, E. R., Colón, L. F., Araghi, K., Tuma, O., Bansal, A., Amen, T. B., Mok, J. K., Asada, T., Kaidi, A. C., Kazarian, G. S., et al. (2025) · The Spine Journal · doi:10.1016/j.spinee.2025.05.005 · Cited by 1
An exploration of exaggerated fears of identity theft in hoarding disorder
Worden, B. L., & Levy, H. C. (2025) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2025.05.043 · Cited by 1
Double, K. S., Xiao, H. H., Pinkus, R. T., Walker, S., & MacCann, C. (2025) · Journal of Happiness Studies · doi:10.1007/s10902-025-00914-3 · Cited by 1
Public Segmentation and the Impact of AI Use in E-Rulemaking
Perez, L. A., Jensen, M. L., Bessarabova, E., Talbert, N., Li, Y., & Zhu, R. (2025) · Media and Communication · doi:10.17645/mac.9550 · Cited by 1
Luo, Y., & Ng, Y.-L. (2025) · Social Media + Society · doi:10.1177/20563051251348922 · Cited by 1
Barry, B., Coish, M., Fields, D., McPhee, P., Monaghan, J., & Ternes, M. (2025) · International Journal of Forensic Mental Health · doi:10.1177/14999013251366447 · Cited by 1
Potential Interventions for Policy Support Targeting Justice-involved People with Mental Illness
Hernandez, B., & Louden, J. E. (2025) · International Journal of Forensic Mental Health · doi:10.1177/14999013251372626 · Cited by 3
SimanTov‐Nachlieli, I., Bamberger, P. A., Liu, S., Wang, M., & Chen, J. (2025) · Occupational Health Science · doi:10.1007/s41542-025-00240-3 · Cited by 1
Development of a Brief State Self-Transcendent Emotion Scale
Samantha, R. M., Sahar, Z., & Diessner, R. (2025) · Psychology Research and Practice · doi:10.37155/2972-3086-0402-3 · Cited by 2
Brown, E., & Kogan, L. R. (2025) · Human-Animal Interactions · doi:10.1079/hai.2025.0054 · Cited by 1
Testing the socio‐functional model: Does precarity cause conspiracy belief?
Marques, M. D., Williams, M. N., Clarke, E. J. R., Fida, N., Heffernan, J., Hill, S., Kerr, J. R., Ling, M., Parkinson, C., Thermos, M., et al. (2025) · British Journal of Social Psychology · doi:10.1111/bjso.70037 · Cited by 1
Reimer, T. O., Oh, J., Loaiza-Ramírez, J. P., & Barber, H. (2024) · Sustainability · doi:10.3390/su16062540 · Cited by 1
Moral judgements on human and autonomous drivers’ decisions in unavoidable collisions scenarios
Bovesi, A., Calabretto, A., Stroppa, A., Adamo, G. M., Canepa, F., & Guidi, S. (2024) · European Conference on Cognitive Ergonomics · doi:10.1145/3673805.3673827 · Cited by 1
EXPLORING RELAPSE IN THE HUMAN LABORATORY: NOTES FROM A SYMPOSIUM
Craig, A. R., Kimball, R. T., Saini, V., Smith, S. W., & Sullivan, W. E. (2025) · Experimental Analysis of Human Behavior Bulletin · doi:10.14244/eahb.v35ispecialissue.10 · Cited by 1
Allocative SkillPreprint
Caplin, A., Deming, D. J., Leth‐Petersen, S., & Weidmann, B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4594737 · Cited by 5
Salerno, J. M., Bean, S., Duran, N. D., Wulff, A. N., & Kassin, S. M. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/b2xq4 · Cited by 1
Salerno, J. M., Bean, S., Duran, N. D., Wulff, A. N., & Kassin, S. M. (2023) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/87q5h · Cited by 1
Numeracy, Consistency, and the Allais Paradox
McGinity, S. (2023) · ISU Red - Research and eData (Illinois State University) · doi:10.30707/etd2023.20240124055107676812.999991 · Cited by 1
News-Trade Integration and Investor BehaviorPreprint
Cade, N. L., Croom, J., & Kamrath, B. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4709548 · Cited by 1
Sociopolitical Stance-taking and Labor Market Sorting: Evidence from ExperimentsPreprint
Hurst, R., & Lee, S. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4846170 · Cited by 3
Bayesian Factor Mixture Modeling with Response Time for Detecting Careless RespondentsPreprint
Zhang, L., Ulitzsch, E., & Domingue, B. W. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/qc9jb · Cited by 2
The licensing effect in gambling choice: A daily diary studyPreprint
Wu, R., & Clark, L. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/u5hdf · Cited by 2
Disclosure Presentation Attributes, Generative AI Output, and Investor JudgmentsPreprint
Croom, J., Gale, B., & Grant, S. M. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.5040309 · Cited by 3
Cumulative Fee Neglect and Brokerage ChoicePreprint
Eaton, G. W., Malliaris, S. G., & Puertas, M. I. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5112507 · Cited by 1
The Political Economy of Green Investing: Insights from the 2024 U.S. ElectionPreprint
Ceccarelli, M., Ramelli, S., Vasileva, A., & Wagner, A. F. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5198512 · Cited by 7
The licensing effect in gambling choice: A daily diary studyPreprint
Wu, R., & Clark, L. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/u5hdf_v1 · Cited by 1
Affective Framing of Environmental News Headlines Influences Engagement, Donations, and MemoryPreprint
Sinclair, A. H., Cosme, D., Tartak, J. C., & Falk, E. B. (2025) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/pzkfd_v3 · Cited by 1
Motivations to connect with like-minded audiences increase partisan sharing on social mediaPreprint
Marie, A., & Petersen, M. B. (2025) · OSF Preprints (OSF Preprints) · doi:10.31219/osf.io/nmg9h_v4 · Cited by 1
Partisan Ingroup-Outgroup Bias in Investment DecisionsPreprint
Choi, Y. J., Chu, Y., Tan, C. M., & Zhu, R. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5813102 · Cited by 2
Gittner, K. B., Matheny, L. M., Balkcom, G. D., Duncan, K., Lewis, A. E., Down, J., & Truong, A. (2026) · Survey Practice · doi:10.29115/sp-2026-0017 · Cited by 2
The Impact of Trauma-Informed Meditation on Posttraumatic Growth: A Randomized Controlled TrialPreprint
Poandl, M. M., Gutiérrez, D., & Dorais, S. (2026) · Research Square · doi:10.21203/rs.3.rs-9657939/v1 · Cited by 1
Testing the testing effect on prolific: when retrieval practice fails to boost learning
Sigayret, K., Parmentier, J.-F., & Silvestre, F. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1727423
Birnbaum, A., & Haney‐Caron, E. (2023) · Journal of Ethnicity in Criminal Justice · doi:10.1080/15377938.2023.2207171
Economic Decision-Making Skill Predicts Income in Two CountriesPreprint
Caplin, A., Deming, D. J., Leth‐Petersen, S., & Weidmann, B. (2023) · SSRN Electronic Journal · doi:10.2139/ssrn.4567680 · Cited by 6
Intention of MOOCs Adoption, Completion and Continued Use
Alzahrani, M., Karimi, F., Bharathy, G. K., & Prasad, M. (2023) · Lecture notes in computer science · doi:10.1007/978-981-99-8385-8_1 · Cited by 1
Hypothetical nudges provide directional but noisy estimates of real behavior changePreprint
Gandhi, L., Kiyawat, A., Camerer, C. F., & Watts, D. J. (2023) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/c7mkf
A New Perspective on Ai Diffusion: Modeling Differences in Usage Across 26 Recommender FeaturesPreprint
Engström, E., Vartanova, I., Johansson, J. V., Persson, M. S., & Strimling, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4688546
McDougall, J. S. (2024) · doi:10.52295/dcc.6649 · Cited by 1
Stepfamily relationships and attachment in young adulthood
Egginton, B. R. (2023) · doi:10.32469/10355/98754
Modeling Differences in Usage Across 26 Online Recommender Features in the U.SPreprint
Engström, E., Vartanova, I., Johansson, J. V., Persson, M. S., & Strimling, P. (2024) · SSRN Electronic Journal · doi:10.2139/ssrn.4791136
Motivated reasoning in Technology Workers
Feath, K. (2023) · San José State University ScholarWorks (San Jose State University) · doi:10.31979/etd.2svm-b2yv
Human-Centered AI at Work: Common Ground in Theories and Methods
(2024) · Frontiers research topics · doi:10.3389/978-2-8325-4840-0
Tao, J., Brown‐Devlin, N., & Forbes, A. (2024) · International Journal of Sport Communication · doi:10.1123/ijsc.2024-0030 · Cited by 2
Estimating Percentages and Free Recall of Categorized Lists in Education Related TasksPreprint
Wright, D. B. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/cgp73
Bok, S., Shum, J., & Lee, M. (2024) · Cogent Psychology · doi:10.1080/23311908.2024.2359827 · Cited by 1
Fed Up: The Determinants of Public Opposition to the U.S. Federal Reserve
Anson, I. G. (2024) · Political Research Quarterly · doi:10.1177/10659129241265703 · Cited by 1
(Title withdrawn - for peer review)Preprint
Kim, J. J., Joel, S., Gonzales, A. M., Murphy, B. A., Perez, J. C., Kaufman, V. A., Bradbury, T. N., Eastwick, P. W., & Karney, B. R. (2024) · PsyArXiv (OSF Preprints) · doi:10.31234/osf.io/n8dpj
Herbener, A. B., Bohn, A., & Pfattheicher, S. (2024) · Collabra Psychology · doi:10.1525/collabra.121246
Kamps, J. C. C., Birk, M. V., Vriens, C., Wallner, G., & Bernhaupt, R. (2024) · Mensch und Computer (MuC) · doi:10.1145/3670653.3670675
Thomas, H. A., Ellis, J. D., & Grekin, E. R. (2024) · Assessment · doi:10.1177/10731911241273352
Anchoring of political attitudes
Broscious, C., Halladay, B., & Landsman, R. (2024) · Economics Letters · doi:10.1016/j.econlet.2024.111944
Police evaluation of evidence: statistical format and evidence type
Cabell, J. J., Livingston, T. N., & Yang, Y. (2024) · Law Probability and Risk · doi:10.1093/lpr/mgae010 · Cited by 2
The effectiveness of compassionate leadership across cultures: a moderated mediation model
Horak, S., Park, J. G., Worm, V. D., & Jakobsen, M. (2026) · Personnel Review · doi:10.1108/pr-10-2024-0953
Grashuis, J. (2024) · Journal of International Food & Agribusiness Marketing · doi:10.1080/08974438.2024.2411550
Eisele, L., & Apruzzese, G. (2024) · Annual Symposium on Computer-Human Interaction in Play · doi:10.1145/3665463.3678817 · Cited by 1
Metadata
- Authors
- Douglas, B. D., Ewell, P. J., & Brauer, M.
- Affiliations
- University of Wisconsin–Madison (US)
- Kenyon College (US)
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-03-14
- Volume
- 18
- Issue
- 3
- Pages
- e0279720
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10013894
- OpenAlex
- W4324129606
- Cited by
- 1,532
- References
- 44
- Citations by year
- 2026: 558 · 2025: 571 · 2024: 329 · 2023: 69
- FWCI
- 347.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab, <https://brite.wisc.edu/>
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- online data collection, data quality, crowdsourcing platforms, Amazon Mechanical Turk, Prolific, Qualtrics, Sona, attention checks, bot detection, online surveys
- MeSH terms
- Data Accuracy, Humans, Surveys and Questionnaires, Research Subjects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.