The DOI matches its Crossref record; 51 of the 51 references it cites were found in a registry or an index.
Identifying careless responses in survey data.
Abstract
When data are collected via anonymous Internet surveys, particularly under conditions of obligatory participation (such as with student samples), data quality can be a concern. However, little guidance exists in the published literature regarding techniques for detecting careless responses. Previously several potential approaches have been suggested for identifying careless respondents via indices computed from the data, yet almost no prior work has examined the relationships among these indicators or the types of data patterns identified by each. In 2 studies, we examined several methods for identifying careless responses, including (a) special items designed to detect careless response, (b) response consistency indices formed from responses to typical survey items, (c) multivariate outlier analysis, (d) response time, and (e) self-reported diligence. Results indicated that there are two distinct patterns of careless response (random and nonrandom) and that different indices are needed to identify these different response patterns. We also found that approximately 10%-12% of undergraduates completing a lengthy survey for course credit were identified as careless responders. In Study 2, we simulated data with known random response patterns to determine the efficacy of several indicators of careless response. We found that the nature of the data strongly influenced the efficacy of the indices to identify careless responses. Recommendations include using identified rather than anonymous responses, incorporating instructed response items before data collection, as well as computing consistency indices and multivariate outlier analysis to ensure high-quality data.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“According to Meade and Craig’s [40] recommendations, questionnaires with the same option (e.g., choice 1) throughout the whole questionnaire were excluded.”
From Effects of attractions and social attributes on peoples' usage intention and media dependence towards chatbot: The mediating role of parasocial interaction and emotional support (2025) · cites this paper as Meade and Craig’s [40]
In the citing paper
“three attention check questions were included to measure the quality of responses”
From Fear of missing out, social media influencers, and the social, psychological and financial wellbeing of young consumers (2025) · cites this paper as [96]
In the citing paper
“Increased error in some population (whether systematic or random) makes measures less capable at capturing individual differences in the trait of interest in that population (see e.g., refs. 20–22; see also ref. 23).”
From Simpson’s gender-equality paradox (2025) · cites this paper as [22]
In the citing paper
“It is difficult to completely account for error (see refs. 21 and 22), but scale reliability may be used as an indication of the prevalence of quality responses.”
From Simpson’s gender-equality paradox (2025) · cites this paper as [22]
In the citing paper
“It also uses more extensive data quality controls, as recommended in ref. 22.”
From Simpson’s gender-equality paradox (2025) · cites this paper as [22]
In the citing paper
“Invalid responses were excluded if there are the same responses for all items and if items receive consistent extreme values (Meade & Craig, 2012). The valid rate was 100% as a result of applying these criteria.”
From University students' Motivated Attention and use of regulation strategies on social media (2015) · cites this paper as Meade & Craig, 2012
In the citing paper
“These responses can compromise the quality and accuracy of the data, weaken the relationships among survey variables, and impact factor loading and structure”
From Measuring undergraduate students’ reliance on generative AI during problem-solving: Scale development and validation. (2025) · cites this paper as Meade & Craig, 2012
In the citing paper
“Additionally, we checked our results for responses where the first option is selected for each question [24].We did not find such cases.”
From Using AI-based coding assistants in practice: State of affairs, perceptions, and ways forward (2025) · cites this paper as [24]
Citations
Cites
Tabachnick, B. G., & Fidell, L. S. (1983) · Cited by 79,882
Grice, H. P. (1975) · Speech Acts · doi:10.1163/9789004368811_003 · Cited by 23,221
Nylund, K. L., Asparouhov, T., & Muthén, B. O. (2007) · Structural Equation Modeling: A Multidisciplinary Journal · doi:10.1080/10705510701575396 · Cited by 11,415
(2007) · Cited by 10,092
A new scale of social desirability independent of psychopathology.
Crowne, D. P., & Marlowe, D. (1960) · Journal of Consulting Psychology · doi:10.1037/h0047358 · Cited by 8,686
Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method
Dillman, D. A., Smyth, J. D., & Christian, L. M. (2008) · Cited by 5,345
The Psychology of Survey Response
Tourangeau, R., Rips, L. J., & Rasinski, K. (2000) · doi:10.1017/cbo9780511819322 · Cited by 4,734
A Brief Tutorial on the Development of Measures for Use in Survey Questionnaires
Hinkin, T. R. (1998) · Organizational Research Methods · doi:10.1177/109442819800100106 · Cited by 4,095
(1999) · Medical Entomology and Zoology · Cited by 3,151
Two-component models of socially desirable responding.
Paulhus, D. L. (1984) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.46.3.598 · Cited by 3,013
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846
ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES1
Samejima, F. (1968) · ETS Research Bulletin Series · doi:10.1002/j.2333-8504.1968.tb00153.x · Cited by 2,670
Personality Psychology in Europe
Vera-Bermudez, J., Raad, D. B., de Vries, J., García, A. M. P., Sanchez-Elvita, A., & Heck, G. V. (1997) · University of Groningen research database (University of Groningen / Centre for Information Technology) · Cited by 2,462
MMPI-2 : Minnesota Multiphasic Personality Inventory-2 : manual for administration and scoring
Hathaway, S. R., & McKinley, J. (1989) · University of Minnesota Press eBooks · Cited by 2,450
Raskin, R., & Terry, H. (1988) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.54.5.890 · Cited by 2,273
Raskin, R., & Terry, H. (1988) · Journal of Personality and Social Psychology · doi:10.1037//0022-3514.54.5.890 · Cited by 2,267
Assessing psychopathic attributes in a noninstitutionalized population.
Levenson, M. R., Kiehl, K. A., & Fitzpatrick, C. M. (1995) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.68.1.151 · Cited by 1,775
The Revised NEO Personality Inventory (NEO-PI-R)
Costa, P. T., & McCrae, R. R. (2008) · SAGE Publications Ltd eBooks · doi:10.4135/9781849200479.n9 · Cited by 1,569
Detecting and Deterring Insufficient Effort Responding to Surveys
Huang, J. L. W., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2011) · Journal of Business and Psychology · doi:10.1007/s10869-011-9231-8 · Cited by 1,315
Investigating population heterogeneity with factor mixture models.
Lubke, G. H., & Muthén, B. (2005) · Psychological Methods · doi:10.1037/1082-989x.10.1.21 · Cited by 1,189
The Role of Constructs in Psychological and Educational Measurement
(2001) · Routledge eBooks · doi:10.4324/9781410607454 · Cited by 876
(2002) · Cited by 771
Ascertaining the validity of individual protocols from Web-based personality inventories
Johnson, J. A. (2004) · Journal of Research in Personality · doi:10.1016/j.jrp.2004.09.009 · Cited by 618
The "Science of the Sophomore" Revisited: From Conjecture to Empiricism
Gordon, M. E., Slade, L. A., & Schmitt, N. (1986) · Academy of Management Review · doi:10.5465/amr.1986.4282666 · Cited by 576
The SAGE Handbook of Personality Theory and Assessment
Austin, E. J., Parker, J. D. A., Petrides, K. V., & Saklofske, D. H. (2008) · Cited by 554
Psychological Experiments on the Internet
(2000) · Elsevier eBooks · doi:10.1016/b978-0-12-099980-4.x5000-x · Cited by 466
Spelke, E. S., Hirst, W. C., & Neisser, U. (1976) · Cognition · doi:10.1016/0010-0277(76)90018-4 · Cited by 434
ESTIMATION OF LATENT ABILITY USING A RESPONSE PATTERN OF GRADED SCORES
SAMEJlMA, F. (2010) · Cited by 425
Careless Responding to Reverse-Worded Items: Implications for Confirmatory Factor Analysis
Woods, C. M. (2006) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/s10862-005-9004-7 · Cited by 402
Carrier, L. M., Cheever, N. A., Rosen, L. D., Benitez, S., & Chang, J. (2009) · Computers in Human Behavior · doi:10.1016/j.chb.2008.10.012 · Cited by 370
The Impact of Anonymity on Responses to Sensitive Questions1
Ong, A. D., & Weiss, D. (2000) · Journal of Applied Social Psychology · doi:10.1111/j.1559-1816.2000.tb02462.x · Cited by 332
Detection of Feigned Mental Disorders
Rogers, R., Sewell, K. W., Martin, M. A., & Vitacco, M. J. (2003) · Assessment · doi:10.1177/1073191103010002007 · Cited by 319
Stark, S. E., Chernyshenko, O. S., Drasgow, F., & Williams, B. A. (2006) · Journal of Applied Psychology · doi:10.1037/0021-9010.91.1.25 · Cited by 230
The human subject in psychological research
Schultz, D. P. (1968) · PsycEXTRA Dataset · doi:10.1037/e515702009-001 · Cited by 212
Detection of malingering on the MMPI: A meta-analysis
Berry, D. T. R., Baer, R. A., & Harris, M. J. (1991) · Clinical Psychology Review · doi:10.1016/0272-7358(91)90005-f · Cited by 210
Identifiability and self‐presentation: Computer‐mediated communication and intergroup interaction
Douglas, K. M., & McGarty, C. (2001) · British Journal of Social Psychology · doi:10.1348/014466601164894 · Cited by 199
Clark, S. L., Muthén, B., Kaprio, J. A., D'Onofrio, B. M., Viken, R. J., & Rose, R. J. (2013) · Structural Equation Modeling A Multidisciplinary Journal · doi:10.1080/10705511.2013.824786 · Cited by 189
MMPI-2 random responding indices: Validation using a self-report methodology.
Berry, D. T. R., Wetter, M. W., Baer, R. A., Larsen, L. H., Clark, C. D., & Monroe, K. (1992) · Psychological Assessment · doi:10.1037/1040-3590.4.3.340 · Cited by 177
Potential of the Internet for Personality Research
Buchanan, T. (2000) · Elsevier eBooks · doi:10.1016/b978-012099980-4/50006-x · Cited by 175
Generation Digital: Politics, Commerce, and Childhood in the Age of the Internet
Montgomery, K. C. (2007) · Cited by 146
Research Validity Scales for the NEO--PI--R: Development and Initial Validation
Schinka, J. A., Kinder, B. N., & Kremer, T. (1997) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6801_10 · Cited by 107
Identifying the Random Responder
Beach, D. A. (1989) · The Journal of Psychology · doi:10.1080/00223980.1989.10542966 · Cited by 104
Kurtz, J. E., & Parrish, C. L. (2001) · Journal of Personality Assessment · doi:10.1207/s15327752jpa7602_12 · Cited by 83
Clark, M. E., Gironda, R. J., & Young, R. W. (2003) · Psychological Assessment · doi:10.1037/1040-3590.15.2.223 · Cited by 79
Detection of Random Responding on the MMPI--A
Baer, R. A., Ballenger, J. F., Berry, D. T. R., & Wetter, M. W. (1997) · Journal of Personality Assessment · doi:10.1207/s15327752jpa6801_11 · Cited by 75
Detection of random responding on the MMPI-2: Utility of F, back F, and VRIN scales.
Berry, D. T. R., Wetter, M. W., Baer, R. A., Widiger, T. A., Sumpter, J. C., Reynolds, S. K., & Hallam, R. A. (1991) · Psychological Assessment · doi:10.1037/1040-3590.3.3.418 · Cited by 67
Campbell, M. J., & Waters, W. E. (1990) · Journal of Epidemiology & Community Health · doi:10.1136/jech.44.1.75 · Cited by 55
Personality measurement and testing
Boyle, G. J., Matthews, G., & Saklofske, D. H. (2008) · Sage eBooks · Cited by 49
A comparison of cognitive-behavioral and hypnotic treatments of chronic pain
Edelson, J., & Fitzpatrick, J. L. (1989) · Journal of Clinical Psychology · doi:10.1002/1097-4679(198903)45:2 · Cited by 40
Privacy, Publicity, and Accountability of Self-Presentation in an On-Line Discussion Group*
Lee, H. (2005) · Sociological Inquiry · doi:10.1111/j.1475-682x.2006.00142.x · Cited by 32
Survey confidentiality vs anonymity: Young men's self-reported substance use.
Moore, R. S., & Ames, G. M. (2002) · Journal of alcohol and drug education · Cited by 18
Flagged first, then by citations
Cited by
Sorted by citations
The Importance of Climate Risks for Institutional Investors
Krueger, P., Sautner, Z., & Starks, L. T. (2020) · The Review of Financial Studies · doi:10.1093/rfs/hhz137 · Cited by 3,034
Methods for the detection of carelessly invalid responses in survey data
Curran, P. G. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.006 · Cited by 1,356
Detecting and Deterring Insufficient Effort Responding to Surveys
Huang, J. L. W., Curran, P. G., Keeney, J., Poposki, E. M., & DeShon, R. P. (2011) · Journal of Business and Psychology · doi:10.1007/s10869-011-9231-8 · Cited by 1,315
Hulland, J., Baumgartner, H., & Smith, K. M. (2018) · Journal of the Academy of Marketing Science · doi:10.1007/s11747-017-0532-y · Cited by 1,067
Many Labs 2: Investigating variation in replicability across samples and settings
Klein, R. A., Vianello, M., Hasselman, F., Adams, B. G., Adams, R. B., Alper, S., Aveyard, M., Axt, J. R., Babalola, M. T., Bahník, Š., et al. (2018) · Advances in Methods and Practices in Psychological Science · doi:10.1177/2515245918810225 · Cited by 944
Metadata
- Authors
- Meade, A. W., & Craig, S. B.
- Affiliations
- North Carolina State University (US)
- Venue
- Psychological Methods
- Published
- 2012-09
- Volume
- 17
- Issue
- 3
- Pages
- 437-455
- ISSN
- 1082-989X (ISSN-L)
- 1939-1463
- OpenAlex
- W2116323796
- Cited by
- 3,865
- References
- 50
- Citations by year
- 2026: 604 · 2025: 481 · 2024: 362 · 2023: 355 · 2022: 405 · 2021: 383 · 2020: 300 · 2019: 247 · 2018: 206 · 2017: 195 · 2016: 122 · 2015: 114 · 2014: 65 · 2013: 14 · 2012: 5
- FWCI
- 209.22
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Psychological Methods (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Keywords
- survey data quality, reaction time (psychology), data simulation, online surveys
- MeSH terms
- Data Collection, Humans, Surveys and Questionnaires, Reaction Time, Research Design, Logistic Models, Internet, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
