CoChat
Sign Up
Cited by 3,865Survey Methodology and NonresponsePsychometric Methodologies and TestingSurvey Sampling and Estimation Techniques

Identifying careless responses in survey data.

Meade, Adam W., Craig, S. Bartholomew·2012·Psychological Methods
DOI 10.1037/a0028085
Open this paper in CoChat — free account

Abstract

When data are collected via anonymous Internet surveys, particularly under conditions of obligatory participation (such as with student samples), data quality can be a concern. However, little guidance exists in the published literature regarding techniques for detecting careless responses. Previously several potential approaches have been suggested for identifying careless respondents via indices computed from the data, yet almost no prior work has examined the relationships among these indicators or the types of data patterns identified by each. In 2 studies, we examined several methods for identifying careless responses, including (a) special items designed to detect careless response, (b) response consistency indices formed from responses to typical survey items, (c) multivariate outlier analysis, (d) response time, and (e) self-reported diligence. Results indicated that there are two distinct patterns of careless response (random and nonrandom) and that different indices are needed to identify these different response patterns. We also found that approximately 10%-12% of undergraduates completing a lengthy survey for course credit were identified as careless responders. In Study 2, we simulated data with known random response patterns to determine the efficacy of several indicators of careless response. We found that the nature of the data strongly influenced the efficacy of the indices to identify careless responses. Recommendations include using identified rather than anonymous responses, incorporating instructed response items before data collection, as well as computing consistency indices and multivariate outlier analysis to ensure high-quality data.

Integrity signals

Registry

Crossref · 10.1037/a0028085

Cites

51 of 51 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Questionnaires with the same option throughout were excluded.

    “According to Meade and Craig’s [40] recommendations, questionnaires with the same option (e.g., choice 1) throughout the whole questionnaire were excluded.”

    From Effects of attractions and social attributes on peoples' usage intention and media dependence towards chatbot: The mediating role of parasocial interaction and emotional support (2025) · cites this paper as Meade and Craig’s [40]

  • Three attention check questions were included to measure the quality of responses and minimize careless responses.

    “three attention check questions were included to measure the quality of responses”

    From Fear of missing out, social media influencers, and the social, psychological and financial wellbeing of young consumers (2025) · cites this paper as [96]

  • Increased error in a population, whether systematic or random, makes measures less capable at capturing individual differences in the trait of interest in that population.

    “Increased error in some population (whether systematic or random) makes measures less capable at capturing individual differences in the trait of interest in that population (see e.g., refs. 20–22; see also ref. 23).”

    From Simpson’s gender-equality paradox (2025) · cites this paper as [22]

  • Scale reliability may be used as an indication of the prevalence of quality responses.

    “It is difficult to completely account for error (see refs. 21 and 22), but scale reliability may be used as an indication of the prevalence of quality responses.”

    From Simpson’s gender-equality paradox (2025) · cites this paper as [22]

  • The Gosling-Potter data use more extensive data quality controls, as recommended in ref. 22.

    “It also uses more extensive data quality controls, as recommended in ref. 22.”

    From Simpson’s gender-equality paradox (2025) · cites this paper as [22]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Related

Authors
Meade, A. W., & Craig, S. B.
Affiliations
North Carolina State University (US)
ORCID
Venue
Psychological Methods
Publisher
American Psychological Association (APA)
Published
2012-09
Volume
17
Issue
3
Pages
437-455
ISSN
  • 1082-989X (ISSN-L)
  • 1939-1463
DOI
10.1037/a0028085
OpenAlex
W2116323796
Cited by
3,865
References
50
Citations by year
2026: 604 · 2025: 481 · 2024: 362 · 2023: 355 · 2022: 405 · 2021: 383 · 2020: 300 · 2019: 247 · 2018: 206 · 2017: 195 · 2016: 122 · 2015: 114 · 2014: 65 · 2013: 14 · 2012: 5
FWCI
209.22
Type
journal article
Language
en
Open access
closed
Locations
  • Psychological Methods (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
  • Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
Keywords
survey data quality, reaction time (psychology), data simulation, online surveys
MeSH terms
Data Collection, Humans, Surveys and Questionnaires, Reaction Time, Research Design, Logistic Models, Internet, Self Report

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review