Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher’s question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our primary concern was the potential compromise in data quality and representativeness often associated with these platforms, as highlighted in recent studies [23,35,68].”
From Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models (2024) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Combined with superior prescreening tools, transparent demographic targeting, and cost-efficiency (Douglas et al., 2023), these strengths make prolific particularly well-suited for research in educational psychology.”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prolific participants show higher attentiveness, longer and more appropriate completion times, fewer signs of disengagement, and greater response consistency (Albert and Smilek, 2023; Douglas et al., 2023).”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Douglas et al. found that the data quality of samples recruited on Prolific is higher than on other labour markets and student samples, with participants paying more attention to the tasks and providing more meaningful answers [22].”
From Cognitive Forcing for Better Decision-Making: Reducing Overreliance on AI Systems Through Partial Explanations (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Prolific and CloudResearch users consistently outperform Amazon Mechanical Turk (MTurk), Qualtrics, and SONA users on key data quality indicators (Douglas et al., 2023).”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · cited 1×
“, to account for the fact that we already “used up” all available degrees of freedom for the test of our a priori hypotheses) [43].”
BackgroundAmazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
The international personality item pool and the future of public-domain personality measures
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006) · Journal of Research in Personality · doi:10.1016/j.jrp.2005.08.007 · Cited by 3,185
(1999) · Medical Entomology and Zoology · Cited by 3,151
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 3×
“[19] observed that MTurk (the platform with the most experienced workers) had worse data quality than Prolific (one of two platforms with less experienced workers) but better data quality than CrowdFlower (the other platform with less experienced workers).”
BackgroundInfluentialData quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Detecting and Deterring Insufficient Effort Responding to Surveys
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-011-9231-8 · Cited by 1,295
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
Chmielewski, M., & Kucker, S. C. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619875149 · Cited by 1,027 · cited 3×
“, Cronbach’s alphas and internal validity between scales) [2].”
BackgroundCoppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
MTurk Research: Review and Recommendations
Aguinis, H., Villamor, I., & Ramani, R. S. (2021) · Journal of Management · doi:10.1177/0149206320969787 · Cited by 984
Research works printed but not linked: 1
- Ternovski J, Orr L. A Note on Increases in Inattentive Online Survey-Takers Since 2020. Journal of Quantitative Description: Digital Media. 2022; 2: 1– 35.
Cited by
Items 301–400 of 1,004
The upside of uncertainty: how counterfeit risk in secondary markets influences primary luxury sales
Hayes, A. S., & Pellandini‐Simányi, L. (2025) · Journal of Business Research · doi:10.1016/j.jbusres.2025.115698 · Cited by 13
Xu, S., Gong, Z. H., Li, Y., Zhao, Y., & Huang, S. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2560522 · Cited by 9
Semantic memory network resilience in aging: The role of abstract and semantically diverse words
Cosgrove, A. L., Beaty, R. E., Sandberg, C., Kenett, Y. N., & Diaz, M. T. (2025) · Cognition · doi:10.1016/j.cognition.2025.106339 · Cited by 10
Lee, Y., Oh, S., Kim, T., & Bang, H. (2025) · Journal of Marketing Management · doi:10.1080/0267257x.2025.2596021 · Cited by 10
Rahman, M. J., & Liang, H. (2026) · International Journal of Information Management · doi:10.1016/j.ijinfomgt.2026.103060 · Cited by 8
Kleinert, S., & Urbig, D. (2026) · Entrepreneurship Theory and Practice · doi:10.1177/10422587261430320 · Cited by 13
Nguyen, P., Watson, G. P., Barnes, D., Agrawal, S., Schuster, A. M., & Cotten, S. R. (2026) · Journal of Management & Organization · doi:10.1017/jmo.2026.10090 · Cited by 15
Goal-directed remembering in older adults
Hennessee, J. P., Schorn, J. M., Walsh, C. R., Castel, A. D., & Knowlton, B. J. (2023) · Aging Neuropsychology and Cognition · doi:10.1080/13825585.2023.2282223 · Cited by 7
Zakharin, M., & Bates, T. C. (2023) · Personality and Individual Differences · doi:10.1016/j.paid.2023.112502 · Cited by 7
Spicer, S. G., Close, J., Nicklin, L. L., Uther, M., Whalley, B., Fullwood, C., Parke, J., Lloyd, J., & Lloyd, H. M. (2024) · Royal Society Open Science · doi:10.1098/rsos.231046 · Cited by 14
People’s preferred balance between politicians, citizens, and experts in policy-making decisions
Haesevoets, T., Verschuere, B., & Roets, A. (2024) · Policy Studies · doi:10.1080/01442872.2024.2323673 · Cited by 12
Arsenault, A. C., Kreps, S. E., Snider, K. L. G., & Canetti, D. (2024) · Journal of Peace Research · doi:10.1177/00223433241233960 · Cited by 7
Support for redistribution is shaped by motives of egalitarian division and coercive redistribution
Lin, C.-A., & Bates, T. C. (2024) · Personality and Individual Differences · doi:10.1016/j.paid.2024.112684 · Cited by 8
Ward, S. B., Nardella, S., Hamilton, K. W., & Walsh, K. (2024) · The Journal of Sex Research · doi:10.1080/00224499.2024.2352555 · Cited by 8
Tobacco Images Choice and its Association With Craving and Dependence in People Who Smoke Cigarettes
Solinas, M., Chauvet, C., Lafay‐Chebassier, C., Vanderkam, P., Barillot, L., Moeller, S. J., Goldstein, R. Z., Noël, X., Jaafari, N., & Chatard, A. (2024) · Nicotine & Tobacco Research · doi:10.1093/ntr/ntae138 · Cited by 8
Educators’ perceptions of expectancy, value, and cost for supporting student emotions
Peterson, E. G., & Zengilowski, A. (2024) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2024.102294 · Cited by 7
Louis, S. S., & Petersen, N. (2024) · Justice Quarterly · doi:10.1080/07418825.2024.2361719 · Cited by 7
Comparing and modeling the use of online recommender systems
Engström, E., Vartanova, I., Johansson, J. V., Persson, M. S., & Strimling, P. (2024) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2024.100449 · Cited by 11
Virtual reality vs traditional previews: the role of self-construal in customer decision-making
Balaji, M. S., Jiang, Y., Dwivedi, Y. K., Behl, A., & Jain, K. (2024) · International Journal of Contemporary Hospitality Management · doi:10.1108/ijchm-10-2023-1578 · Cited by 9
Quantifying data quality after removing respondents who fail data quality checks
Taplin, R. (2024) · Current Issues in Tourism · doi:10.1080/13683500.2024.2378611 · Cited by 11
Thwaites, P., Vandeweerd, N., & Paquot, M. (2024) · Applied Linguistics · doi:10.1093/applin/amae048 · Cited by 12
Can we enhance prosocial behavior? Using post-ride feedback to improve micromobility interactions
Scott-Sharoni, S. T., Mehrotra, S., Salubre, K., Song, M., Misu, T., & Akash, K. (2024) · International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI) · doi:10.1145/3640792.3675739 · Cited by 8
Schroeders, U., Loos, A., Wiedemann, S., & Jankowsky, K. (2024) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000855 · Cited by 7
You Do You[Tube]!The multifaceted roles of online video viewing in the lives of U.S. children
Bonus, J. A., Dore, R. A., Hassinger‐Das, B., Wilson, J. M., O'Hara, E., & Francemone, C. J. (2024) · Early Childhood Research Quarterly · doi:10.1016/j.ecresq.2024.10.001 · Cited by 9
Slow Tourism and Well-Being: A Transformational Learning Perspective
Vo‐Thanh, T., Luong, V. H., Manthiou, A., & Petr, C. (2024) · Tourism Planning & Development · doi:10.1080/21568316.2024.2443187 · Cited by 12
Smrke, U., Rehberger, A., Plohl, N., & Mlakar, I. (2025) · Applied Sciences · doi:10.3390/app15041937 · Cited by 14
Rathjens, B., Zhang, L., & Cha, J. (2025) · Journal of Hospitality and Tourism Technology · doi:10.1108/jhtt-11-2023-0348 · Cited by 9
From Niche to Mainstream: US Consumer Trends and Preferences for Elderberry Products
Cai, Z., Stubblefield, K., Thomas, A. L., & Aguilar, F. X. (2024) · HortScience · doi:10.21273/hortsci18180-24 · Cited by 9
The relation between clutter and visual fatigue in children with cerebral visual impairment
Hove, C. V., Damiano, C., & Itzhak, N. B. (2025) · Ophthalmic and Physiological Optics · doi:10.1111/opo.13447 · Cited by 9
Ngo, M. S. M., & Badri, S. K. Z. (2025) · Global Business and Organizational Excellence · doi:10.1002/joe.22293 · Cited by 6
Vargiu, C., Nai, A., & Garzia, D. (2025) · The Journal of Politics · doi:10.1086/735437 · Cited by 9
Integrating prior knowledge and multiple texts: expanding the Documents Model Framework
List, A. (2025) · Reading and Writing · doi:10.1007/s11145-025-10641-z · Cited by 8
Moreno-Gata, S., Rodríguez-Torres, R., Pérez, A. R., & Betancor, V. (2025) · PLoS ONE · doi:10.1371/journal.pone.0322212 · Cited by 10
Zhang, X., & Zhou, Z. (2025) · Journal of Product & Brand Management · doi:10.1108/jpbm-07-2023-4588 · Cited by 8
Martin, D., Hawkins, R. M., Gee, T., Turnock, L. A., & Lazuras, L. (2025) · Performance Enhancement & Health · doi:10.1016/j.peh.2025.100353 · Cited by 10
Making (Enough for) Love: The Association of Income and Relationship Readiness
Peetz, J., & MacDonald, G. (2025) · Journal of Marriage and the Family · doi:10.1111/jomf.70000 · Cited by 12
Kuhl, U., & Bush, A. (2025) · Communications in computer and information science · doi:10.1007/978-3-032-08333-3_12 · Cited by 12
When trust turns digital: Public support for online crime reporting
Jackson, J., Bradford, B., Chan, A., & Lee, Y. (2025) · Journal of Experimental Criminology · doi:10.1007/s11292-025-09713-5 · Cited by 17
Sayah, F. A., Short, H., Ramos-Goñi, J. M., Viney, R. C., Lubetkin, E. I., Janssen, M. F., Johnson, J. A., the EQ-DAPHNIE Project Team, Johnson, J. G., Janssen, M. F., et al. (2025) · Quality of Life Research · doi:10.1007/s11136-025-04074-y · Cited by 8
Spurrier, H., Kamineni, R., Dissanayake, M., & Lindsay, N. (2025) · Journal of Small Business Management · doi:10.1080/00472778.2025.2579191 · Cited by 9
Ndone, J., Zheng, Q., Niu, R., Jin, Y., & Duffy, M. E. (2025) · Public Relations Review · doi:10.1016/j.pubrev.2025.102651 · Cited by 7
Associations between chronotype and psychiatric symptoms across the adult lifespan
Axelsson, J., & Balter, L. J. T. (2025) · Translational Psychiatry · doi:10.1038/s41398-025-03782-w · Cited by 6
Citizen‐Centric Approaches to AI in Government Programs: Lessons From Experimental Studies
Johnson, D. (2025) · Public Administration Review · doi:10.1111/puar.70077 · Cited by 7
Spicer, A., Andrade, M., Weiss‐Cohen, L., van Baal, S. T., Torrance, J., Xiao, L. Y., & Newall, P. W. S. (2026) · Behavioural Public Policy · doi:10.1017/bpp.2026.10034 · Cited by 6
Asher, M. W., Gold, G., Chen, E., & Carvalho, P. F. (2026) · Advances in Methods and Practices in Psychological Science · doi:10.1177/25152459261424723 · Cited by 9
Food Appreciation Scale Development and Dimensionality Assessment
Cosgrove, K., & Wharton, C. M. (2023) · International Journal of Environmental Research and Public Health · doi:10.3390/ijerph20146345 · Cited by 6
Social Imagery and Judicial Legitimacy: Evidence From Evangelical Christians
Badas, A., & Schmidt, E. R. (2023) · Political Research Quarterly · doi:10.1177/10659129231197207 · Cited by 7
Close, J., Spicer, S. G., Nicklin, L. L., Uther, M., Whalley, B., Fullwood, C., Stiff, C., Parke, J., Lloyd, J., & Lloyd, H. M. (2023) · Royal Society Open Science · doi:10.1098/rsos.231045 · Cited by 13
Degrees of difference: Do college credentials earned behind bars improve labor market outcomes?
Ballou, A. (2024) · Criminology · doi:10.1111/1745-9125.12364 · Cited by 9
How couples think about money: Types of money motives and relationship satisfaction
Peetz, J., & Joseph, M. (2024) · Personal Relationships · doi:10.1111/pere.12539 · Cited by 8
Sawyer, J. S. (2024) · Death Studies · doi:10.1080/07481187.2024.2349933 · Cited by 6
REAL ESTATE BROKERAGE AND ENDOWMENT EFFECTS IN THE HOUSING MARKET: EVIDENCE FROM POLAND
Tomal, M. (2024) · International Journal of Strategic Property Management · doi:10.3846/ijspm.2024.21456 · Cited by 7
Ulitzsch, E., Buchholz, J., Shin, H. J., Bertling, J., & Lüdtke, O. (2024) · Large-scale Assessments in Education · doi:10.1186/s40536-024-00205-y · Cited by 9
Kay, C. S. (2024) · Behavior Research Methods · doi:10.3758/s13428-024-02452-x · Cited by 7
Baker, L. N., & Fuglestad, A. J. (2024) · Appetite · doi:10.1016/j.appet.2024.107608 · Cited by 10
Johnson, D., & Neshkova, M. I. (2024) · The American Review of Public Administration · doi:10.1177/02750740241268263 · Cited by 11
Judges Behaving Badly: Judicial Misconduct and a Threat to Rights
Baker, M. N., & Canelo, K. S. (2024) · Journal of Experimental Political Science · doi:10.1017/xps.2024.8 · Cited by 6
Schneider, J. M., Schmidt, K. A., Bohrer, K., & Merk, S. (2024) · Zeitschrift für Psychologie · doi:10.1027/2151-2604/a000573 · Cited by 10
Jin, H., Hayward, W. G., & Cheung, O. S. (2024) · Journal of Vision · doi:10.1167/jov.24.11.13 · Cited by 6
Perceiving Low Social Mobility Induces Zero-Sum Beliefs About Social Hierarchies
Wang, X., Zhuo, Y., Chen, W., Du, H., & Chen, Z. (2024) · Personality and Social Psychology Bulletin · doi:10.1177/01461672241290390 · Cited by 10
Green, D. J.-A., Barnes, T. A., & Klein, N. D. (2024) · Scientific Reports · doi:10.1038/s41598-024-76934-5 · Cited by 8
Evans, E. H., Ridley, B. J., Cornelissen, P. L., Kramer, R. S. S., Araújo‐Soares, V., & Tovee, M. J. (2024) · British Journal of Health Psychology · doi:10.1111/bjhp.12765 · Cited by 7
Denning, D. M., & Brown, T. A. (2024) · Affective Science · doi:10.1007/s42761-024-00290-w · Cited by 6
Examining Cognitions in an Anger Episode with and Without Physical Aggression
DiBlasi, T., Keenan, K., Pisano, C., Tumbarello, G., Engel, V., Vincent, B., & Loiacono, A. (2024) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10563-0 · Cited by 7
The Dark Triad and cyberloafing: unveiling the shadowy nexus
Cohen, A. G., & Özsoy, E. (2025) · Management Research Review · doi:10.1108/mrr-12-2023-0908 · Cited by 9
Validation of an online imitation-inhibition task
Westfal, M., Cracco, E., Crusius, J., & Genschow, O. (2025) · Behavior Research Methods · doi:10.3758/s13428-024-02557-3 · Cited by 10
Eerie edibles: Realism and food neophobia predict an uncanny valley in AI-generated food images
Diel, A., Lalgi, T. J., Teufel, M., Bäuerle, A., & MacDorman, K. F. (2025) · Appetite · doi:10.1016/j.appet.2025.107926 · Cited by 8
Benchmarking Music Generation Models and Metrics via Human Preference Studies
Grötschla, F., Solak, A., Lanzendörfer, L. A., & Wattenhofer, R. P. (2025) · IEEE International Conference on Acoustics Speech and Signal Processing · doi:10.1109/icassp49660.2025.10887745 · Cited by 24
Is Deliberate Control of Behavior Rare? A Test of the Automaticity Dominance Perspective
Miles, A., Khan, S., & Samim, Y. (2025) · Socius Sociological Research for a Dynamic World · doi:10.1177/23780231251325087 · Cited by 6
Standen, E. C., Rothman, A. J., & Mann, T. L. (2025) · Annals of Behavioral Medicine · doi:10.1093/abm/kaaf018 · Cited by 8
Thwaites, P., Jadoulle, P., & Paquot, M. (2025) · Assessing Writing · doi:10.1016/j.asw.2025.100937 · Cited by 7
Breiner, C. E., McQuaid, G. A., Wallace, G. L., & Zickgraf, H. F. (2025) · Autism Research · doi:10.1002/aur.70039 · Cited by 6
Monaghan, C., Shou, Y., Mewton, P., Quayle, A., & Dawel, A. (2025) · Assessment · doi:10.1177/10731911251333664 · Cited by 9
Grace, K. T., Gupta, J., Fay, K. E., Altay, T., Kanselaar, S., & Miller, E. P. (2025) · Journal of Midwifery & Women s Health · doi:10.1111/jmwh.13758 · Cited by 6
Kennedy, S. L., Cornacchi, S. D., Kaur, M. N., Morrison, S. D., Rae, C. S., Johnson, N., Khatchadourian, K., Armstrong, K., Marinković, M. D., Sequeira, G. M., et al. (2025) · Transgender Health · doi:10.1089/trgh.2024.0203 · Cited by 6
Working where we want: The role of work arrangement fit in work-related and personal wellbeing
Schweitzer, L., Smith, C. J., Lyons, S. T., Henchey, A., & Kostuchuk, J. (2025) · Journal of Management & Organization · doi:10.1017/jmo.2025.10024 · Cited by 8
Reyes, M. A. (2023) · Personal Relationships · doi:10.1111/pere.12516 · Cited by 6
Bastiani, O., Russell, A. M. T., & Newall, P. W. S. (2023) · Journal of Gambling Studies · doi:10.1007/s10899-023-10261-8 · Cited by 6
Millionaire Justices and Attitudes Towards the Supreme Court
Badas, A., & Justus, B. (2023) · Political Behavior · doi:10.1007/s11109-023-09905-7 · Cited by 7
The grey side of procurement: Measuring the prevalence of questionable purchasing practices
Chen, F., van Dalen, J., & Wynstra, F. (2024) · Journal of Purchasing and Supply Management · doi:10.1016/j.pursup.2024.100922 · Cited by 14
Adu, P., Popoola, T., Collings, S. C., Aspin, C., Medvedev, O. N., & Simpson, C. R. (2024) · Current Psychology · doi:10.1007/s12144-024-05877-x · Cited by 8
Economic Decision-Making Skill Predicts Income in Two Countries
Caplin, A., Deming, D. J., Leth‐Petersen, S., & Weidmann, B. (2023) · National Bureau of Economic Research · doi:10.3386/w31674 · Cited by 19
Transparency Fallacy: Perceived Fairness in Algorithmic Management
Mirbabaie, M., Langer, M., Rieskamp, J., & Hofeditz, L. (2026) · Business & Information Systems Engineering · doi:10.1007/s12599-025-00963-1 · Cited by 17
Psychological shadows: The Dark Tetrad and fraudulent shopping practices
Cohen, A. G. (2025) · Personality and Individual Differences · doi:10.1016/j.paid.2025.113103 · Cited by 8
Ndone, J., & Abdallah, J. C. (2025) · Communication & Sport · doi:10.1177/21674795251323193 · Cited by 5
Özkan, Ö., Rowe, R., Norman, P. E., Day, M. R., & Poulter, D. R. (2025) · Transportation Research Part F Traffic Psychology and Behaviour · doi:10.1016/j.trf.2025.03.003 · Cited by 6
Assessing construct reliability through open-ended survey response analysis
Koralesky, K. E., von Keyserlingk, M. A. G., & Weary, D. M. (2025) · PLoS ONE · doi:10.1371/journal.pone.0320570 · Cited by 5
Do Hot Flashes Get the Cold Shoulder? Menopausal Symptoms and Disclosure Influence Leader Ratings
Grandey, A. A., Burke, V., Zeytun, D., Frasca, T. J., & Matsick, J. L. (2025) · Journal of Business and Psychology · doi:10.1007/s10869-025-10039-2 · Cited by 7
Stinglhamber, F., Ohana, M., & Demoulin, S. (2025) · Journal of Business Research · doi:10.1016/j.jbusres.2025.115642 · Cited by 8
Noble, A. J., Lane, S., New, R. P., Cope, H., Foley, C., Williams, H. L., Sztriha, L., Powell, G. A., Reuber, M., & Marson, A. G. (2025) · Epilepsia · doi:10.1111/epi.18624 · Cited by 5
Brandt, G., Pahlenkemper, M., Juergensen, V., Lehe, M. S., Laskowski, N. M., Halbeisen, G., & Paslakis, G. (2025) · Scientific Reports · doi:10.1038/s41598-025-19246-6 · Cited by 6
Derbyshire, J., Dhami, M. K., Belton, I., Önkal, D., & Aven, T. (2025) · Risk Analysis · doi:10.1111/risa.70112 · Cited by 6
Improving access to gender-affirming care: a mixed-methods exploration of the role of telehealth
Duffy, C. M. C., Wall, C. S. J., Cull, S. L., Otieno, J. M., Schult, D., Mardigian, F. R., Cardinale-McGinnis, G., Easter, R., & Benotsch, E. G. (2025) · Annals of Behavioral Medicine · doi:10.1093/abm/kaaf074 · Cited by 5
Beyond algorithm aversion: The impact of psychological readiness on algorithmic advice
Kim, H. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108824 · Cited by 12
Huynh, M.-T. (2025) · Information Systems Frontiers · doi:10.1007/s10796-025-10647-6 · Cited by 12
Defensive reactions to a meat reduction intervention
Graça, J., Fanter, E., Hopwood, C. J., & Hoeks, J. C. J. (2025) · Appetite · doi:10.1016/j.appet.2025.108354 · Cited by 6
Zhang, X., Li, J., Shao, X., & Han, H. (2025) · Tourism Management · doi:10.1016/j.tourman.2025.105331 · Cited by 6
Why AI robot chefs backfire: Consumer responses to robot-themed appeals
Baek, T. H., Kim, M., & Bang, H. (2025) · Journal of Hospitality and Tourism Management · doi:10.1016/j.jhtm.2025.101358 · Cited by 6
Hu, Z., Chi, C. G.-Q., & Luo, J. M. (2025) · Journal of Hospitality Marketing & Management · doi:10.1080/19368623.2025.2594109 · Cited by 5
Craig, M. J. A., Choi, M., Zhu, Y., Hassan, T., Noi, S. M., & Silva, D. E. (2026) · Social Media + Society · doi:10.1177/20563051261417301 · Cited by 6
Metadata
- Authors
- Douglas, B. D., Ewell, P. J., & Brauer, M.
- Affiliations
- University of Wisconsin–Madison (US)
- Kenyon College (US)
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-03-14
- Volume
- 18
- Issue
- 3
- Pages
- e0279720
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10013894
- OpenAlex
- W4324129606
- Cited by
- 1,532
- References
- 44
- Citations by year
- 2026: 558 · 2025: 571 · 2024: 329 · 2023: 69
- FWCI
- 347.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab, <https://brite.wisc.edu/>
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- online data collection, data quality, crowdsourcing platforms, Amazon Mechanical Turk, Prolific, Qualtrics, Sona, attention checks, bot detection, online surveys
- MeSH terms
- Data Accuracy, Humans, Surveys and Questionnaires, Research Subjects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.