Data quality in online human-subjects research: Comparisons between MTurk, Prolific, CloudResearch, Qualtrics, and SONA
Open accessThe DOI matches its Crossref record; 45 of the 45 references it cites were found in a registry or an index.
Abstract
With the proliferation of online data collection in human-subjects research, concerns have been raised over the presence of inattentive survey participants and non-human respondents (bots). We compared the quality of the data collected through five commonly used platforms. Data quality was indicated by the percentage of participants who meaningfully respond to the researcher’s question (high quality) versus those who only contribute noise (low quality). We found that compared to MTurk, Qualtrics, or an undergraduate student sample (i.e., SONA), participants on Prolific and CloudResearch were more likely to pass various attention checks, provide meaningful answers, follow instructions, remember previously presented information, have a unique IP address and geolocation, and work slowly enough to be able to read all the items. We divided the samples into high- and low-quality respondents and computed the cost we paid per high-quality respondent. Prolific ($1.90) and CloudResearch ($2.00) were cheaper than MTurk ($4.36) and Qualtrics ($8.17). SONA cost $0.00, yet took the longest to collect the data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Our primary concern was the potential compromise in data quality and representativeness often associated with these platforms, as highlighted in recent studies [23,35,68].”
From Shaping Human-AI Collaboration: Varied Scaffolding Levels in Co-writing with Language Models (2024) · cites this paper as [23]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Combined with superior prescreening tools, transparent demographic targeting, and cost-efficiency (Douglas et al., 2023), these strengths make prolific particularly well-suited for research in educational psychology.”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Prolific participants show higher attentiveness, longer and more appropriate completion times, fewer signs of disengagement, and greater response consistency (Albert and Smilek, 2023; Douglas et al., 2023).”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Douglas et al. found that the data quality of samples recruited on Prolific is higher than on other labour markets and student samples, with participants paying more attention to the tasks and providing more meaningful answers [22].”
From Cognitive Forcing for Better Decision-Making: Reducing Overreliance on AI Systems Through Partial Explanations (2025) · cites this paper as [22]
Experimental — accuracy not yet measuredThe cited source says what the claim attributes to it.
In the citing paper
“Prolific and CloudResearch users consistently outperform Amazon Mechanical Turk (MTurk), Qualtrics, and SONA users on key data quality indicators (Douglas et al., 2023).”
From Testing the testing effect on prolific: when retrieval practice fails to boost learning (2026) · cites this paper as Douglas et al., 2023
1 more claim about this paper
Create a free account to see every claim other papers make about this one.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Simple Sequentially Rejective Multiple Test Procedure
(1979) · Scandinavian Journal of Statistics · doi:10.2307/4615733 · Cited by 21,526 · cited 1×
“, to account for the fact that we already “used up” all available degrees of freedom for the test of our a priori hypotheses) [43].”
BackgroundAmazon's Mechanical Turk: A New Source of Inexpensive, Yet High-Quality, Data?
Buhrmester, M., Kwang, T., & Gosling, S. D. (2011) · Perspectives on Psychological Science · doi:10.1177/1745691610393980 · Cited by 9,996
The international personality item pool and the future of public-domain personality measures
Goldberg, L. R., Johnson, J. A., Eber, H. W., Hogan, R., Ashton, M. C., Cloninger, C. R., & Gough, H. G. (2006) · Journal of Research in Personality · doi:10.1016/j.jrp.2005.08.007 · Cited by 3,185
(1999) · Medical Entomology and Zoology · Cited by 3,151
Beyond the Turk: Alternative platforms for crowdsourcing behavioral research
Peer, E., Brandimarte, L., Samat, S., & Acquisti, A. (2017) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2017.01.006 · Cited by 2,957 · cited 3×
“[19] observed that MTurk (the platform with the most experienced workers) had worse data quality than Prolific (one of two platforms with less experienced workers) but better data quality than CrowdFlower (the other platform with less experienced workers).”
BackgroundInfluentialData quality of platforms and panels for online behavioral research
Peer, E., Rothschild, D., Gordon, A., Evernden, Z., & Damer, E. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01694-3 · Cited by 1,718
Detecting and Deterring Insufficient Effort Responding to Surveys
Huang, J. L., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2012) · Journal of Business and Psychology · doi:10.1007/s10869-011-9231-8 · Cited by 1,295
An MTurk Crisis? Shifts in Data Quality and the Impact on Study Results
Chmielewski, M., & Kucker, S. C. (2020) · Social Psychological and Personality Science · doi:10.1177/1948550619875149 · Cited by 1,027 · cited 3×
“, Cronbach’s alphas and internal validity between scales) [2].”
BackgroundCoppock, A., & McClellan, O. A. (2019) · Research & Politics · doi:10.1177/2053168018822174 · Cited by 987
MTurk Research: Review and Recommendations
Aguinis, H., Villamor, I., & Ramani, R. S. (2021) · Journal of Management · doi:10.1177/0149206320969787 · Cited by 984
Research works printed but not linked: 1
- Ternovski J, Orr L. A Note on Increases in Inattentive Online Survey-Takers Since 2020. Journal of Quantitative Description: Digital Media. 2022; 2: 1– 35.
Cited by
Items 201–300 of 1,004
Grassini, S., Thorp, S. O., Cipriani, E., Mannari, V., & Sevic, A. (2025) · Computers in Human Behavior Reports · doi:10.1016/j.chbr.2025.100834 · Cited by 14
McLean, C. P., Chen, Z. Z., Song, R., Le, J., Fielding, J., & Sharp, G. (2024) · Journal of Eating Disorders · doi:10.1186/s40337-024-00964-7 · Cited by 13
Grashuis, J. (2024) · Journal of Foodservice Business Research · doi:10.1080/15378020.2024.2303534 · Cited by 13
Empathic Disequilibrium as a Predictor of Nonsuicidal Self-Injury in Autistic and Nonautistic People
Moseley, R. L., Shalev, I., Gregory, N. J., & Uzefovsky, F. (2024) · Autism in Adulthood · doi:10.1089/aut.2023.0134 · Cited by 13
Perrig, S. A. C., Scharowski, N., Brühlmann, F., von Felten, N., Opwis, K., & Aeschbach, L. F. (2024) · Proceedings of the CHI Conference on Human Factors in Computing Systems · doi:10.1145/3613904.3642270 · Cited by 10
Zhang, B. (2024) · Health Communication · doi:10.1080/10410236.2024.2348236 · Cited by 11
Huang, J. (., & Murthy, U. S. (2024) · International Journal of Accounting Information Systems · doi:10.1016/j.accinf.2024.100696 · Cited by 16
Dewitte, L., Olaru, G., Lewis, N. A., Allemand, M., & Hill, P. L. (2024) · The Journal of Positive Psychology · doi:10.1080/17439760.2024.2433045 · Cited by 12
Prolific: Crowdsourcing Academic Online Research
Péer, E. (2024) · Cambridge University Press eBooks · doi:10.1017/9781009000796.005 · Cited by 18
Antisemitism as a dark-ego vehicle
Bertrams, A., & Krispenz, A. (2024) · Current Psychology · doi:10.1007/s12144-024-07120-z · Cited by 10
Online Reflective Essays: A Guide for Qualitative Researchers
Grigorovicius, M. (2025) · International Journal of Qualitative Methods · doi:10.1177/16094069251321256 · Cited by 11
Development and validation of the trust in AI scale (TAIS)
Wischnewski, M., Doebler, P., & Krämer, N. C. (2026) · Journal of Psychology and AI · doi:10.1080/29974100.2026.2715475 · Cited by 19
Rutter, L. A., Edinger, A., Lorenzo‐Luaces, L., ten Thij, M., Valdez, D., & Bollen, J. (2025) · Cognitive Therapy and Research · doi:10.1007/s10608-025-10580-7 · Cited by 15
Assessing the quality and reliability of the Amazon Mechanical Turk (MTurk) data in 2024
Shimoni, H., & Axelrod, V. (2025) · Royal Society Open Science · doi:10.1098/rsos.250361 · Cited by 16
Wang, B., Shibo, B. W., & Kafle, J. (2025) · Journal of Health Communication · doi:10.1080/10810730.2025.2540864 · Cited by 12
Bowden, J., Eshaghi, M. S., Afshardoost, M., & Rahman, S. M. (2025) · Journal of Consumer Marketing · doi:10.1108/jcm-10-2024-7298 · Cited by 12
Nadeem, W., Ashraf, A. R., & Hultman, M. (2025) · European Journal of Marketing · doi:10.1108/ejm-07-2024-0572 · Cited by 12
Starke, S., & Ludviga, I. (2025) · Sustainability · doi:10.3390/su17209184 · Cited by 17
Bakker, A. B., Caracuzzo, E., Breevaart, K., Urbini, F., & Callea, A. (2025) · European Management Journal · doi:10.1016/j.emj.2025.10.009 · Cited by 11
Herbenick, D., Fu, T., Chen, X., Ali, S., Stanojević, I. S., Hensel, D. J., Wright, P. J., Peterson, Z. D., Harezlak, J., & Fortenberry, J. D. (2025) · Archives of Sexual Behavior · doi:10.1007/s10508-025-03245-9 · Cited by 12
Zhu, J., & Molnár, A. (2026) · Computers in Human Behavior · doi:10.1016/j.chb.2026.108929 · Cited by 18
Auzepy, A., Bannier, C. E., & Gärtner, F. (2026) · International Review of Economics & Finance · doi:10.1016/j.iref.2026.105773 · Cited by 13
Borderline personality disorder features are associated with inflexible social interpretations
Duda, J. M., Fineberg, S. K., Deng, W., Ma, Y., Everaert, J., Cannon, T. D., & Joormann, J. (2023) · Journal of Affective Disorders · doi:10.1016/j.jad.2023.12.036 · Cited by 11
Esfand, S. M., Null, K. E., Duda, J. M., de Leeuw, J. R., & Pizzagalli, D. A. (2024) · Journal of Affective Disorders · doi:10.1016/j.jad.2024.01.133 · Cited by 13
Widespread, bipartisan aversion exists to neighbors owning AR-15s or storing guns insecurely
Sola, J. L., & Pickett, J. T. (2024) · Proceedings of the National Academy of Sciences · doi:10.1073/pnas.2311825121 · Cited by 12
How social media and flexible work arrangements harden salespeople to abusive supervision
Dugan, R. G., Edelblum, A. B., Kalra, A., Lee, N. Y., & Yoon, S. (2024) · Industrial Marketing Management · doi:10.1016/j.indmarman.2024.07.008 · Cited by 11
Towards the Terminator Economy: Assessing Job Exposure to AI Through LLMs
Colombo, E., Mercorio, F., Mezzanzanica, M., & Serino, A. (2025) · International Joint Conference on Artificial Intelligence · doi:10.24963/ijcai.2025/1066 · Cited by 18
Fantasy Worlds, Real-Life Impact: The Benefits of RPGs for Transgender Identity Exploration
Wert, S. V., & Howansky, K. (2024) · Journal of Homosexuality · doi:10.1080/00918369.2024.2320242 · Cited by 11
Symbolic metaprogram search improves learning efficiency and explains rule learning in humans
Rule, J., Piantadosi, S. T., Cropper, A., Ellis, K., Nye, M., & Tenenbaum, J. B. (2024) · Nature Communications · doi:10.1038/s41467-024-50966-x · Cited by 44
Rose, B., Kuiper, M. E., Folmer, C. R., & van Rooij, B. (2024) · Crime Science · doi:10.1186/s40163-024-00240-8 · Cited by 10
Gaither, B. M., Kim, S., & Austin, L. (2025) · Public Relations Review · doi:10.1016/j.pubrev.2024.102536 · Cited by 10
Talebi, A., Mukherjee, S., Gera, N., Kaur, K., & Das, G. (2025) · Journal of Product Innovation Management · doi:10.1111/jpim.12774 · Cited by 18
Development and preliminary validation of the comprehensive emotional eating scale (CEES)
Hooper, M. A., Dixit, U., Ahlich, E. M., & Zickgraf, H. F. (2025) · Appetite · doi:10.1016/j.appet.2025.107969 · Cited by 10
Dahlawi, G. A., Badawi, N. S., & Salam, M. A. (2025) · Administrative Sciences · doi:10.3390/admsci15040127 · Cited by 12
An open-source reproducible chess robot for human-robot interaction research
Zhang, R., de Winter, J., Dodou, D., Seyffert, H. C., & Eisma, Y. B. (2025) · Frontiers in Robotics and AI · doi:10.3389/frobt.2025.1436674 · Cited by 14
Meme advertising for luxury brands: effects on perceived funniness and sharing intention
Baek, T. H., Park, J., & Kim, J. S. (2025) · International Journal of Advertising · doi:10.1080/02650487.2025.2510062 · Cited by 13
Price, A., Sumner, P., & Powell, G. (2025) · Vision Research · doi:10.1016/j.visres.2025.108640 · Cited by 17
Motivations to connect with like-minded audiences increase partisan sharing on social media
Marie, A., & Petersen, M. B. (2025) · PNAS Nexus · doi:10.1093/pnasnexus/pgaf197 · Cited by 12
Kluge, J., Korman, B. A., Schledjewski, J., & Grosche, M. (2025) · Frontiers in Psychology · doi:10.3389/fpsyg.2025.1535727 · Cited by 14
Eslamdoust, S., Borhani, T., Dorri, R., & Karandish, M. (2025) · Discover Psychology · doi:10.1007/s44202-025-00363-2 · Cited by 11
Zhai, C., & Öztürk, İ. (2025) · Aggressive Behavior · doi:10.1002/ab.70041 · Cited by 9
Naturalistic psychedelic use and changes in depressive symptoms
Simonsson, O., Hendricks, P. S., Swords, C. M., Osika, W., & Goldberg, S. B. (2025) · Journal of Affective Disorders · doi:10.1016/j.jad.2025.119857 · Cited by 10
Carvalho, I., Loureiro, S. M. C., Ivanov, S. H., Björk, P., & Seyitoğlu, F. (2025) · Journal of Hospitality and Tourism Insights · doi:10.1108/jhti-01-2025-0126 · Cited by 13
Lee, Y.-H., & Yuan, C. W. (2025) · Telematics and Informatics · doi:10.1016/j.tele.2025.102339 · Cited by 9
Darsono, D., Hersugondo, H., Yuniawan, A., Kusumaningrum, H. P., Latan, H., & Ali, M. (2025) · Journal of economic and administrative sciences. · doi:10.1108/jeas-04-2025-0221 · Cited by 13
Under pressure: how time constraints, task complexity, and AI reliability shape human-AI interaction
Hermanns, L., & Teubner, T. (2025) · Behaviour and Information Technology · doi:10.1080/0144929x.2025.2587732 · Cited by 15
Valori, I., Fan, Y., Jung, M. M., & Fairhurst, M. T. (2024) · Scientific Reports · doi:10.1038/s41598-024-57582-1 · Cited by 14
Klassen, A. F., Pusic, A. L., Kaur, M. N., Mansouri, J., Tsangaris, E., Dayan, S. H., Klok, J. A., Armstrong, K., Santosa, K. B., Rae, C. S., et al. (2024) · Facial Plastic Surgery & Aesthetic Medicine · doi:10.1089/fpsam.2023.0204 · Cited by 9
Social interaction, support preferences, and the use of wearable health trackers
Pennington, N., & Dam, L. (2023) · Frontiers in Communication · doi:10.3389/fcomm.2023.1256452 · Cited by 10
A temporal investigation of experienced incivility and perpetrated counterproductive work behaviour
Henderson, A. A., & Matthews, R. A. (2023) · Journal of Occupational and Organizational Psychology · doi:10.1111/joop.12487 · Cited by 9
Schroeders, U., Morgenstern, M., Jankowsky, K., & Gnambs, T. (2024) · European Journal of Psychological Assessment · doi:10.1027/1015-5759/a000818 · Cited by 11
Crowdsourced data in public administration research: A review and look to the future
Stritch, J. M., Pedersen, M. J., & Pezo, I. (2024) · Public Administration Review · doi:10.1111/puar.13823 · Cited by 17
Beiza, A. A., Maneethai, D., & Madera, J. M. (2024) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2024.103756 · Cited by 12
The perceived impacts of short-term rental platforms: Comparing the United States and United Kingdom
Lutz, C., Majetić, F., Miguel, C., Perez‐Vega, R., & Jones, B. (2024) · Technology in Society · doi:10.1016/j.techsoc.2024.102586 · Cited by 9
Fast fashion consumption as a female competition strategy: implications for sustainable consumption
Font, C. M., & Font, X. (2024) · Journal of Fashion Marketing and Management · doi:10.1108/jfmm-03-2023-0063 · Cited by 13
Virtuous victimhood as a Dark Triad resource transfer strategy
Bates, T. C., Grant, C., Hobbs, L., Johnston, C. S., Moghaddam, S., & Sinclair, K. (2024) · Personality and Individual Differences · doi:10.1016/j.paid.2024.112964 · Cited by 8
Amsalem, D., Greuel, M., Liu, S., Martin, A., & Adam, M. (2025) · JMIR Public Health and Surveillance · doi:10.2196/66496 · Cited by 10
Öztürk, İ., Lehtonen, E., Madigan, R., Lee, Y. M., Aittoniemi, E., & Merat, N. (2025) · Technology in Society · doi:10.1016/j.techsoc.2025.102903 · Cited by 10
Human-centred AI vs machine-centred AI: navigating AI ethical crises in luxury retail
Li, J., & Kang, J. (2025) · International Journal of Retail & Distribution Management · doi:10.1108/ijrdm-11-2024-0600 · Cited by 12
Sun, Y., Freeman, J., Shoenberger, H., & Shen, F. (2025) · International Journal of Human-Computer Interaction · doi:10.1080/10447318.2025.2495847 · Cited by 10
A Simple Ensemble Strategy for LLM Inference: Towards More Stable Text Classification
Niimi, J. (2025) · Lecture notes in computer science · doi:10.1007/978-3-031-97144-0_17 · Cited by 18
Koller, S., Ahmetoglu, G., & Stephan, U. (2025) · Journal of Business Venturing · doi:10.1016/j.jbusvent.2025.106538 · Cited by 10
Catch me if you search: When contextual web search results affect the detection of hallucinations
Nahar, M., Lee, E.-J., Park, J. W., & Lee, D. (2025) · Computers in Human Behavior · doi:10.1016/j.chb.2025.108763 · Cited by 7
Tang, R., Fang, Y., Sun, J., Bode, L., & Vraga, E. K. (2025) · Journalism & Mass Communication Quarterly · doi:10.1177/10776990251359660 · Cited by 13
Alajhar, N. A., Bajaba, S. M., & Basahal, A. S. (2025) · The International Journal of Human Resource Management · doi:10.1080/09585192.2025.2565655 · Cited by 9
Cross, E. S., & Kappas, A. (2025) · Annual Review of Psychology · doi:10.1146/annurev-psych-040325-025951 · Cited by 16
Laupichler, M. C., Knoth, N., Schleiß, J., & Raupach, T. (2025) · British Journal of Educational Technology · doi:10.1111/bjet.70035 · Cited by 14
Almugren, I., Rai, J. S., Korayim, D., Mohanty, J., & Virmani, N. (2026) · Journal of Knowledge Management · doi:10.1108/jkm-07-2025-0957 · Cited by 9
Reminders undermine impressions of genuine gratitude.
Wang, J., Chaudhry, S. J., & Koch, A. S. (2024) · Journal of Personality and Social Psychology · doi:10.1037/pspi0000442 · Cited by 11
Humans vs. ChatGPT: Evaluating Annotation Methods for Financial Corpora
Kaikaus, J., Li, H., & Brunner, R. J. (2023) · IEEE International Conference on Big Data (BigData) · doi:10.1109/bigdata59044.2023.10386425 · Cited by 14
Racial Bias in Pediatric Pain Perception
Haas, S. M., Mullin, G. J. D., Williams, A., Reynolds, A., Tuerxuntuoheti, A., Reyes, P. G. M., & Mende‐Siedlecki, P. (2024) · Journal of Pain · doi:10.1016/j.jpain.2024.104583 · Cited by 9
Rivalry as a Contextual Factor of Gender Inequality in Network Returns
Lampronti, S. Y., Operti, E., & Sgourev, S. V. (2024) · Journal of Management Studies · doi:10.1111/joms.13121 · Cited by 8
Inoculation hesitancy: an exploration of challenges in scaling inoculation theory
Johnson, A., & Madsen, J. K. (2024) · Royal Society Open Science · doi:10.1098/rsos.231711 · Cited by 15
Numeric social-media posts engage people with climate science
Peters, E. M., Markowitz, D. M., Nadratowski, A., & Shoots‐Reinhard, B. L. (2024) · PNAS Nexus · doi:10.1093/pnasnexus/pgae250 · Cited by 8
Nicol, A. A. M., & Tóth‐Király, I. (2024) · The Journal of Sex Research · doi:10.1080/00224499.2024.2371951 · Cited by 10
Siamionava, K., Mitra, S., & Westerman, G. F. (2024) · Production and Operations Management · doi:10.1177/10591478241266524 · Cited by 8
Radcliffe, P. J., & Patihis, L. (2024) · Memory · doi:10.1080/09658211.2024.2383311 · Cited by 10
The relationship between sleep and interoception
Bynum, L. G., & Brindle, R. C. (2024) · Journal of Sleep Research · doi:10.1111/jsr.14337 · Cited by 9
Boom, L. D., & Meulenaere, K. D. (2024) · Work Aging and Retirement · doi:10.1093/workar/waae016 · Cited by 14
Mazidi, M., Grafton, B., Moulds, M. L., Newby, J. M., & MacLeod, C. (2024) · Cognitive Therapy and Research · doi:10.1007/s10608-024-10551-4 · Cited by 10
Social media addiction and borderline personality disorder: a survey study
Collins, M., & Grant, J. E. (2025) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2024.1459827 · Cited by 15
Price, A., Sumner, P., & Powell, G. (2025) · Vision Research · doi:10.1016/j.visres.2025.108610 · Cited by 15
Gustafson, C. R. (2023) · Appetite · doi:10.1016/j.appet.2023.106596 · Cited by 9
Althnayan, S., Bajaba, S. M., Alsabban, A., & Mandurah, K. (2023) · Employee Responsibilities and Rights Journal · doi:10.1007/s10672-023-09483-0 · Cited by 9
Unlocking Insights into Prolific: Research Implications, Participant Behavior and MotivationsPreprint
Aksoy, B., & Nevo, S. (2025) · SSRN Electronic Journal · doi:10.2139/ssrn.5203748 · Cited by 14
Carbon emission reductions and offsets: Consumer perceptions of firm environmental sustainability
Utgård, J., & Gaustad, T. B. (2024) · Journal of Cleaner Production · doi:10.1016/j.jclepro.2024.143300 · Cited by 9
Yeboah, A. B., Parker, J. E., Matos, L. M., Wilson, P. A., & Tomiyama, A. J. (2024) · International Journal of Eating Disorders · doi:10.1002/eat.24290 · Cited by 7
An exploratory study of victimisation and near misses in online shopping fraud
Edwards, M., Whittaker, J. M., Cross, C., & Button, M. (2024) · Global Crime · doi:10.1080/17440572.2024.2423918 · Cited by 14
Hao, H., Lee, Y. W., Sharko, M., Li, Q., & Zhang, Y. (2024) · JMIR Formative Research · doi:10.2196/60862 · Cited by 11
Lustria, M. L. A., Aliche, O., Killian, M., & He, Z. (2025) · JAMIA Open · doi:10.1093/jamiaopen/ooaf009 · Cited by 8
Gip, H., Russen, M., & Guchait, P. (2025) · International Journal of Hospitality Management · doi:10.1016/j.ijhm.2025.104204 · Cited by 14
Horne, B. D., & Nevo, D. (2025) · Scientific Reports · doi:10.1038/s41598-025-98221-7 · Cited by 12
Gendered racial microaggressions, self-silencing, and disordered eating in Black women
Parker, J. E., Enders, C. K., Tomiyama, A. J., & Hunger, J. M. (2025) · Body Image · doi:10.1016/j.bodyim.2025.101901 · Cited by 7
Ciocîrlan, A.-B., & Rowe, R. (2025) · PLoS ONE · doi:10.1371/journal.pone.0325196 · Cited by 8
Shao, C., Kim, Y., & Xu, L. Z. (2025) · Communication Reports · doi:10.1080/08934215.2025.2519253 · Cited by 12
Evidence of spillovers from (non)cooperative human-bot to human-human interactions
Harrell, A., & Traeger, M. L. (2025) · iScience · doi:10.1016/j.isci.2025.113006 · Cited by 11
Can AI-Attributed News Challenge Partisan News Selection? Evidence from a Conjoint Experiment
Zoizner, A., Matthes, J., Corbu, N., de Vreese, C. H., Esser, F., Koç-Michalska, K., Schemer, C., Theocharis, Y., & Zilinsky, J. (2025) · The International Journal of Press/Politics · doi:10.1177/19401612251342679 · Cited by 11
Audience Responses to Online Public Shaming in Online Environments: Mixed Methods Study
Chen, A., Dunn, L., Fan, W., & Agrawal, N. J. B. (2025) · Journal of Medical Internet Research · doi:10.2196/67923 · Cited by 9
Evaluating Online Data Collection Platforms Using A Simple Rule-Following Task
Suri, D., Kube, S., & Schultz, J. (2025) · Economics Letters · doi:10.1016/j.econlet.2025.112509 · Cited by 8
Evoking trust in smart voice assistants
Wetzels, R. W. H., Wetzels, M. G. M., Grewal, D., & Doek, B. (2025) · Journal of service management · doi:10.1108/josm-06-2024-0275 · Cited by 12
Metadata
- Authors
- Douglas, B. D., Ewell, P. J., & Brauer, M.
- Affiliations
- University of Wisconsin–Madison (US)
- Kenyon College (US)
- Venue
- PLOS ONE
- Publisher
- Public Library of Science (PLoS)
- Published
- 2023-03-14
- Volume
- 18
- Issue
- 3
- Pages
- e0279720
- ISSN
- 1932-6203 (ISSN-L)
- PMC
- PMC10013894
- OpenAlex
- W4324129606
- Cited by
- 1,532
- References
- 44
- Citations by year
- 2026: 558 · 2025: 571 · 2024: 329 · 2023: 69
- FWCI
- 347.46
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- Locations
- PLoS ONE (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- RePEc: Research Papers in Economics (submittedVersion)
- DOAJ (DOAJ: Directory of Open Access Journals) (submittedVersion, cc-by-sa)
- Funders
- BRITE Lab Grant: Behavioral Research Insights Through Experiments Lab, <https://brite.wisc.edu/>
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Physical Sciences › Computer Science › Information Systems › Research Data Management Practices
- Keywords
- online data collection, data quality, crowdsourcing platforms, Amazon Mechanical Turk, Prolific, Qualtrics, Sona, attention checks, bot detection, online surveys
- MeSH terms
- Data Accuracy, Humans, Surveys and Questionnaires, Research Subjects
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.