Participant carelessness and fraud: Consequences for clinical research and potential solutions.
The DOI matches its Crossref record; 20 of the 20 references it cites were found in a registry or an index.
Abstract
Clinical psychological research studies often require individuals with specific characteristics. The Internet can be used to recruit broadly, enabling the recruitment of rare groups such as people with specific psychological disorders. However, Internet-based research relies on participant self-report to determine eligibility, and thus, data quality depends on participant honesty. For those rare groups, even low levels of participant dishonesty can lead to a substantial proportion of fraudulent survey responses, and all studies will include careless respondents who do not pay attention to questions, do not understand them, or provide intentionally wrong responses. Poor-quality responses should be thought of as categorically different from high-quality responses. Including these responses will lead to the overestimation of the prevalence of rare groups and incorrect estimates of scale reliability, means, and correlations between constructs. We demonstrate that for these reasons, including poor-quality responses-which are usually positively skewed-will lead to several data-quality problems including spurious associations between measures. We provide recommendations about how to ensure that fraudulent participants are detected and excluded from self-report research studies. (PsycINFO Database Record (c) 2019 APA, all rights reserved).
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
Cella, D., Riley, W., Stone, A., Rothrock, N., Reeve, B., Yount, S., Amtmann, D., Bode, R., Buysse, D., Choi, S., et al. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.04.011 · Cited by 5,128
Initial reliability and validity of a new retrospective measure of child abuse and neglect
(1994) · American Journal of Psychiatry · doi:10.1176/ajp.151.8.1132 · Cited by 3,577
Reputation as a sufficient condition for data quality on Amazon Mechanical Turk
Peer, E., Vosgerau, J., & Acquisti, A. (2014) · Behavior Research Methods · doi:10.3758/s13428-013-0434-y · Cited by 1,618
Methods for the detection of carelessly invalid responses in survey data
Curran, P. G. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.006 · Cited by 1,356
Conducting Clinical Research Using Crowdsourced Convenience Samples
Chandler, J., & Shapiro, D. (2016) · Annual Review of Clinical Psychology · doi:10.1146/annurev-clinpsy-021815-093623 · Cited by 1,072
Self-citationBerinsky, A. J., Margolis, M. F., & Sances, M. W. (2014) · American Journal of Political Science · doi:10.1111/ajps.12081 · Cited by 951
Outcome Bias in Decision Evaluation
Baron, J., & Hershey, J. C. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.4.569 · Cited by 873
Caring about carelessness: Participant inattention and its effects on research
Maniaci, M. R., & Rogge, R. D. (2014) · Journal of Research in Personality · doi:10.1016/j.jrp.2013.09.008 · Cited by 852
Measuring Delusional Ideation: The 21-Item Peters et al. Delusions Inventory (PDI)
Peters, E., Joseph, S., Day, S., & Garety, P. (2004) · Schizophrenia Bulletin · doi:10.1093/oxfordjournals.schbul.a007116 · Cited by 607
Generalizing from Survey Experiments Conducted on Mechanical Turk: A Replication Approach
Coppock, A. (2019) · Political Science Research and Methods · doi:10.1017/psrm.2018.10 · Cited by 515
The Construction and Validation of an Abridged Version of the Autism-Spectrum Quotient (AQ-Short)
Hoekstra, R. A., Vinkhuyzen, A. A. E., Wheelwright, S., Bartels, M., Boomsma, D. I., Baron-Cohen, S., Posthuma, D., & van der Sluis, S. (2011) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-010-1073-0 · Cited by 506
The shape of and solutions to the MTurk quality crisis
Kennedy, R., Clifford, S., Burleigh, T., Waggoner, P. D., Jewell, R., & Winter, N. J. G. (2020) · Political Science Research and Methods · doi:10.1017/psrm.2020.6 · Cited by 506
Demetrovics, Z., Szeredi, B., & Rózsa, S. (2008) · Behavior Research Methods · doi:10.3758/brm.40.2.563 · Cited by 409
MTurk Character Misrepresentation: Assessment and Solutions
Sharpe Wessling, K., Huber, J., & Netzer, O. (2017) · Journal of Consumer Research · doi:10.1093/jcr/ucx053 · Cited by 330
Wood, D., Harms, P. D., Lowman, G. H., & DeSimone, J. A. (2017) · Social Psychological and Personality Science · doi:10.1177/1948550617703168 · Cited by 243
Robinson-Cimpian, J. P. (2014) · Educational Researcher · doi:10.3102/0013189x14534297 · Cited by 222
Loeber, R., Stouthamer-Loeber, M., & Raskin White, H. (1999) · Journal of Clinical Child Psychology · doi:10.1207/s15374424jccp280304 · Cited by 217
Miller, J. D., Crowe, M., Weiss, B., Maples-Keller, J. L., & Lynam, D. R. (2017) · Personality Disorders Theory Research and Treatment · doi:10.1037/per0000191 · Cited by 179
The registration continuum in clinical science: A guide toward transparent practices
Benning, S. D., Bachrach, R. L., Smith, E. A., Freeman, A. J., & Wright, A. G. C. (2019) · Journal of Abnormal Psychology · doi:10.1037/abn0000451 · Cited by 77
Cited by
Douglas, B. D., Ewell, P. J., & Brauer, M. (2023) · PLOS ONE · doi:10.1371/journal.pone.0279720 · Cited by 1,532
Development and Initial Validation of the Duke Misophonia Questionnaire
Rosenthal, M. Z., Anand, D., Cassiello-Robbins, C., Williams, Z. J., Guetta, R. E., Trumbull, J., & Kelley, L. D. (2021) · Frontiers in Psychology · doi:10.3389/fpsyg.2021.709928 · Cited by 117
Arechar, A. A., & Rand, D. G. (2021) · Behavior Research Methods · doi:10.3758/s13428-021-01588-4 · Cited by 105
McCarty, R. J., Downing, S. T., Daley, M. L., McNamara, J. P. H., & Guastello, A. D. (2023) · Anxiety, Stress, & Coping · doi:10.1080/10615806.2022.2075855 · Cited by 16
Metadata
- Authors
- Chandler, J., Sisso, I., & Shapiro, D.
- Venue
- Journal of Abnormal Psychology
- Published
- 2020-01
- Volume
- 129
- Issue
- 1
- Pages
- 49-55
- ISSN
- 0021-843X (ISSN-L)
- 1939-1846
- OpenAlex
- W2997588693
- Cited by
- 120
- References
- 20
- FWCI
- 24.74
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.