The DOI matches its Crossref record; 27 of the 27 references it cites were found in a registry or an index.
Noncompliant responding: Comparing exclusion criteria in MTurk personality research to improve data quality
Open accessOverview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The cited source doesn’t support the claim as written.
In the citing paper
“However, compared to undergraduate research pool respondents, online study participants typically provide comparable or superior data quality and research fidelity [e.g., 90].”
From Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory (2023) · cites this paper as [90]
Checked by a reasoning model — may be wrong.
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“In crowd-sourced settings such as MTurk, respondents are often incentivized to respond as attentively, compliantly, and quickly as possible to both open higher-paying opportunities and prevent being rejected from future employment [90, 100].”
From Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory (2023) · cites this paper as [90]
Experimental — accuracy not yet measuredThe source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Although some recent concerns about MTurk quality have emerged, data support that diverse validity and attention checks, such as those included in the present studies, adequately protect data quality [90–92].”
From Not so terrifying after all? A set of failed replications of the mortality salience effects of Terror Management Theory (2023) · cites this paper as [90]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Identifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865
Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
Mark, G., Fussell, S. R., Lampe, C., Schraefel, M., Hourcade, J., Appert, C., & Wigdor, D. J. (2017) · International Conference on Human Factors in Computing Systems · doi:10.1145/3025453 · Cited by 2,222
Hauser, D. J., & Schwarz, N. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0578-z · Cited by 1,739
The HEXACO-60: A Short Measure of the Major Dimensions of Personality
Ashton, M. C., & Lee, K. (2009) · Journal of Personality Assessment · doi:10.1080/00223890902935878 · Cited by 1,709
Methods for the detection of carelessly invalid responses in survey data
Curran, P. G. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.006 · Cited by 1,356
Chandler, J., Mueller, P., & Paolacci, G. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0365-7 · Cited by 950
An Evaluation of Amazon’s Mechanical Turk, Its Rapid Rise, and Its Effective Use
Buhrmester, M. D., Talaifar, S., & Gosling, S. D. (2018) · Perspectives on Psychological Science · doi:10.1177/1745691617706516 · Cited by 815
SOME EXPERIMENTAL RESULTS IN THE CORRELATION OF MENTAL ABILITIES 1
Brown, W. E. (1910) · British Journal of Psychology 1904-1920 · doi:10.1111/j.2044-8295.1910.tb00207.x · Cited by 733
Validity and Mechanical Turk: An assessment of exclusion methods and interactive experiments
Thomas, K. A., & Clifford, S. (2017) · Computers in Human Behavior · doi:10.1016/j.chb.2017.08.038 · Cited by 625
Psychometric Properties of the HEXACO-100
Lee, K., & Ashton, M. C. (2016) · Assessment · doi:10.1177/1073191116659134 · Cited by 580
Economic games on the internet: the effect of $1 stakes
Amir, O., Rand, D. G., & Gal, Y. K. (2012) · PLoS ONE · doi:10.1371/journal.pone.0031461 · Cited by 364
Lee, K., & Ashton, M. C. (2006) · Psychological Assessment · doi:10.1037/1040-3590.18.2.182 · Cited by 334
Huang, J. L. W., Bowling, N. A., Liu, M., & Li, Y. (2014) · Journal of Business and Psychology · doi:10.1007/s10869-014-9357-6 · Cited by 315
Dirty Data: The Effects of Screening Respondents Who Provide Low-Quality Data in Survey Research
DeSimone, J. A., & Harms, P. D. (2017) · Journal of Business and Psychology · doi:10.1007/s10869-017-9514-9 · Cited by 269
Some experimental results in the correlation of mental abilities.
Brown, W. G. (1910) · British Journal of Psychology · Cited by 265
Dunn, A. M., Heggestad, E. D., Shanock, L. R., & Theilgard, N. (2016) · Journal of Business and Psychology · doi:10.1007/s10869-016-9479-0 · Cited by 233
Detecting careless respondents in web-based questionnaires: Which method to use?
Niessen, A. S. M., Meijer, R. R., & Tendeiro, J. N. (2016) · Journal of Research in Personality · doi:10.1016/j.jrp.2016.04.010 · Cited by 206
Marjanovic, Z., Holden, R. R., Struthers, W., Cribbie, R. A., & Greenglass, E. R. (2014) · Personality and Individual Differences · doi:10.1016/j.paid.2014.08.021 · Cited by 122
McInnis, B. J., Cosley, D., Nam, C., & Leshed, G. (2016) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/2858036.2858539 · Cited by 119
Zhao, K., Ferguson, E., & Smillie, L. D. (2015) · Journal of Personality · doi:10.1111/jopy.12237 · Cited by 110
Curran, P. G., & Hauser, K. A. (2019) · Journal of Research in Personality · doi:10.1016/j.jrp.2019.103849 · Cited by 79
What Is Engagement? Proactivity as the Missing Link in the HEXACO Model of Personality
de Vries, R. E., Wawoe, K. W., & Holtrop, D. (2014) · Journal of Personality · doi:10.1111/jopy.12150 · Cited by 46
Detecting random responders with infrequency scales using an error-balancing threshold
Kim, D. S., McCabe, C. J., Yamasaki, B. L., Louie, K. A., & King, K. M. (2017) · Behavior Research Methods · doi:10.3758/s13428-017-0964-9 · Cited by 37
AN EVALUATION OF THE VALIDITY OF THE PERSONALITY RESEARCH FORM WITH FILIPINO UNIVERSITY STUDENTS
Fekken, G. C., Holden, R. R., Jackson, D. N., & Guthrie, G. M. (1987) · International Journal of Psychology · doi:10.1080/00207598708246781 · Cited by 21
An effect size index for comparing two independent alpha coefficients
Liu, H., & Weng, L. (2008) · British Journal of Mathematical and Statistical Psychology · doi:10.1348/000711008x315518 · Cited by 17
Marjanovic, Z., Bajkov, L., & MacDonald, J. (2018) · Psychological Reports · doi:10.1177/0033294118783917 · Cited by 11
Weathers, D., & Bardakçı, A. (2015) · Journal of Marketing Analytics · doi:10.1057/jma.2015.6 · Cited by 9
Research works printed but not linked: 1
- Diedenhofen, 2016, cocron: A web interfae and R package for the statistical comparison of Cronbach's alpha coefficients, International Journal of Internet Science, 11, 51
Cited by
Sorted by citations
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0279720 · Cited by 1,532
MTurk Research: Review and Recommendations
Aguinis, H., Villamor, I., & Ramani, R. S. (2020) · Journal of Management · doi:10.1177/0149206320969787 · Cited by 1,098
Zickar, M. J., & Keith, M. G. (2022) · Annual Review of Organizational Psychology and Organizational Behavior · doi:10.1146/annurev-orgpsych-120920-052946 · Cited by 231
Roozenbeek, J., Freeman, A. L. J., & van der Linden, S. (2021) · Psychological Science · doi:10.1177/09567976211024535 · Cited by 121
Agley, J. D., Xiao, Y., Nolan, R., & Golzarri‐Arroyo, L. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01665-8 · Cited by 118
Metadata
- Authors
- Barends, A. J., & de Vries, R. E.
- Affiliations
- Vrije Universiteit Amsterdam (NL)
- Corresponding author
- Barends, Ard J.
- Venue
- Personality and Individual Differences
- Publisher
- Elsevier BV
- Published
- 2019-06
- Volume
- 143
- Pages
- 84-89
- ISSN
- 0191-8869 (ISSN-L)
- OpenAlex
- W2915179069
- Cited by
- 124
- References
- 27
- Citations by year
- 2026: 12 · 2025: 20 · 2024: 18 · 2023: 18 · 2022: 24 · 2021: 20 · 2020: 9 · 2019: 2
- FWCI
- 14.17
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Personality and Individual Differences (publishedVersion)
- VU Research Portal (publishedVersion, other-oa)
- University of Twente Research Information (submittedVersion, other-oa)
- Data Archiving and Networked Services (DANS) (submittedVersion, other-oa)
- Funders
- VU University
- Topics
- Physical Sciences › Computer Science › Computer Science Applications › Mobile Crowdsensing and Crowdsourcing
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Keywords
- Amazon Mechanical Turk, data quality, exclusion criteria, personality assessment
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
