Dealing with Careless Responding in Survey Data: Prevention, Identification, and Recommended Best Practices
Open accessAbstract
Surveys administered online have several benefits, but they are particularly prone to careless responding, which occurs when respondents fail to read item content or give sufficient attention, resulting in raw data that may not accurately reflect respondents’ true levels of the constructs being measured. Careless responding can lead to various psychometric issues, potentially impacting any area of psychology that uses self-reported surveys and assessments. This review synthesizes the careless responding literature to provide a comprehensive understanding of careless responding and ways to prevent, identify, report, and clean careless responding from data sets. Further, we include recommendations for different levels of screening for careless responses. Finally, we highlight some of the most promising areas for future work on careless responding.
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Inattentive responses were improbable given the precision required for entering valid chess moves and the targeted recruitment of chess-interested individuals.
“Although no dedicated attention checks were included, inattentive responses were improbable given the precision required for entering valid chess moves and the targeted recruitment of chess-interested individuals (Ward & Meade, 2023).”
From Investigating appropriate reliance on AI-Based decision support systems: the role of expertise, trust, and self-confidence (2025) · cites this paper as Ward & Meade, 2023
The sections “When Careless Responding Is Likely to Be a Problem” and “Context” identify lack of interest as a risk factor for careless responding, which lends some support to the rationale for recruiting chess-interested participants. They do not address chess moves or establish that inattentive responses were improbable in this study. “Minimal Screening” also recommends screening for careless responses; page numbers are unavailable in our copy. (LLM, conf=0.85)
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Cites
Identifying careless responses in survey data.
Meade, A. W., & Craig, S. B. (2012) · Psychological Methods · doi:10.1037/a0028085 · Cited by 3,865 · see citing works
Self-citationTwo-component models of socially desirable responding.
Paulhus, D. L. (1984) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.46.3.598 · Cited by 3,013 · see citing works
Self-reports: How the questions shape the answers.
Schwarz, N. (1999) · American Psychologist · doi:10.1037/0003-066x.54.2.93 · Cited by 2,846 · see citing works
Methods for the detection of carelessly invalid responses in survey data
Curran, P. G. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.006 · Cited by 1,356 · see citing works
Careless responding in questionnaire measures: Detection, impact, and remedies
Goldammer, P., Annen, H., Stöckli, P. L., & Jonas, K. (2020) · The Leadership Quarterly · doi:10.1016/j.leaqua.2020.101384 · Cited by 200 · see citing works
Brühlmann, F., Petralito, S., Aeschbach, L. F., & Opwis, K. (2020) · Methods in Psychology · doi:10.1016/j.metip.2020.100022 · Cited by 168 · see citing works
Curran, P. G., & Hauser, K. A. (2019) · Journal of Research in Personality · doi:10.1016/j.jrp.2019.103849 · Cited by 79 · see citing works
Flagged first, then by citations
Research works printed but not linked: 50
- Arias V, Garrido L, Jenaro C, Martínez-Molina Arias B. 2020. A little garbage in, lots of garbage out: assessing the impact of careless responding in personality survey data. Behav. Res. Methods 52(6):2489-505
- Baer R, Ballenger J, Berry D, Wetter M. 1997. Detection of random responding on the MMPI-A. J. Pers. Assess. 68(1):139-51
- Barber LK, Barnes CM, Carlson KD. 2013. Random and systematic error effects of insomnia on survey be- havior. Organ. Res. Methods 16(4):616-49
- Barge S, Gehlbach H. 2012. Using the theory of satisficing to evaluate the quality of survey data. Res. High. Educ. 53(2):182-200
- Bassett J, Cleveland A, Acorn D, Nix M, Snyder T. 2017. Are they paying attention? Students' lack of motiva- tion and attention potentially threaten the utility of course evaluations. Assess. Eval. High. Educ. 42(3):431- 42
- Beach D. 1989. Identifying the random responder. J. Psychol. 123(1):101-3
- Beck MF, Albano AD, Smith WM. 2019. Person-fit as an index of inattentive responding: a comparison of methods using polytomous survey data. Appl. Psychol. Meas. 43(5):374-87
- Berry D, Wetter M, Baer R, Larsen L, Clark C, Monroe K. 1992. MMPI-2 random responding indices: vali- dation using a self-report methodology. Psychol. Assess. 4(3):340-45
- Bowling NA, Gibson AM, Houpt JW, Brower CK. 2021. Will the questions ever end? Person-level increases in careless responding during questionnaire completion. Organ. Res. Methods 24(4):718-38
- Bowling NA, Huang J, Bragg C, Khazon S, Liu M, Blackmore C. 2016. Who cares and who is careless? Insufficient effort responding as a reflection of respondent personality. J. Pers. Soc. Psychol. 111(2):218-29
- Brower C. 2018. Too long and too boring: the effects of survey length and interest on careless responding. PhD Thesis, Wright State Univ., Dayton, OH
- Brower C. 2020. What are you looking at? Using eye-tracking to provide insight into careless responding. PhD Thesis, Wright State Univ., Dayton, OH
- PhD Thesis, Wright State Univ., Dayton, OH Carrier L, Cheever N, Rosen L, Benitez S, Chang J. 2009. Multitasking across generations: multitasking choices and difficulty ratings in three generations of Americans. Comput. Hum. Behav. 25(2):483-89
- Conijn J, Franz G, Emons W, de Beurs E, Carlier I. 2019. The assessment and impact of careless responding in routine outcome monitoring within mental health care. Multivar. Behav. Res. 54(4):593-611
- Curran PG, Kotrba L, Denison D. 2010. Careless responding in surveys: applying traditional techniques to organi- zational settings. Paper presented at the 25th Annual Conference of the Society for Industrial and Orga- nizational Psychology, Atlanta, GA
- Dunn A, Heggestad E, Shanock L, Theilgard N. 2018. Intra-individual response variability as an indicator of insufficient effort responding: comparison to other indicators and relationships with individual differ- ences. J. Bus. Psychol. 33(1):105-21
- Ehlers C, Greene-Shortridge TM, Weekley JA, Zajack MD. 2009. The exploration of statistical methods in detecting random responding. Paper presented at the 24th Annual Conference of the Society for Industrial and Organizational Psychology, New Orleans, LA
- Francavilla N, Meade A, Young A. 2019. Social interaction and internet-based surveys: examining the effects of virtual and in-person proctors on careless response. Appl. Psychol. 68(2):223-49
- Galesic M, Bosnjak M. 2009. Effects of questionnaire length on participation and indicators of response quality in a web survey. Public Opin. Q. 73(2):349-60
- Gibson A. 2016. Stop with the questions already! The effects of questionnaire length and monetary incentives on insuf- ficient effort responding. Master's Thesis, Wright State Univ., Dayton, OH
- Gibson A. 2019. Stop what you're doing, right now! Effects of interactive messages on careless responding. PhD Thesis, Wright State Univ., Dayton, OH
- Gibson A, Bowling N. 2020. The effects of questionnaire length and behavioral consequences on careless responding. Eur. J. Psychol. Assess. 36(2):410-20
- Godinho A, Kushnir V, Cunningham J. 2016. Unfaithful findings: identifying careless responding in addictions research. Addiction 111(6):955-56
- Gough H, Bradley P. 1996. California Psychological Inventory. Palo Alto, CA: Consult. Psychol. Press
- Haertzen C, Hill H, Belleville R. 1963. Development of the addiction research center inventory (ARCI): selection of items that are sensitive to the effects of various drugs. Psychopharmacologia 4(3):155-66
- Huang JL, Bowling N, Liu M, Li Y. 2014. Detecting insufficient effort responding with an infrequency scale: evaluating validity and participant reactions. J. Bus. Psychol. 30(2):299-311
- Huang JL, Curran PG, Keeney J, Poposki EM, DeShon RP. 2012. Detecting and deterring insufficient effort responding to surveys. J. Bus. Psychol. 27(1):99-114
- Huang JL, Liu M, Bowling N. 2015. Insufficient effort responding: examining an insidious confound in survey data. J. Appl. Psychol. 100(3):828-45
- Johnson J. 2005. Ascertaining the validity of individual protocols from Web-based personality inventories. J. Res. Pers. 39(1):103-29
- Jones A, Earnest J, Adam M, Clarke R, Yates J, Pennington C. 2022. Careless responding in crowdsourced alco- hol research: a systematic review and meta-analysis of practices and prevalence. Exp. Clin. Psychopharmacol. 30(4):381-99
- Kam CCS, Meyer J. 2015. How careless responding and acquiescence response bias can influence construct dimensionality. Organ. Res. Methods 18(3):512-41
- Kurtz J, Parrish C. 2001. Semantic response consistency and protocol validity in structured personality assess- ment: the case of the NEO-PI-R. J. Pers. Assess. 76(2):315-32
- McGonagle A, Fisher G, Barnes-Farrell J, Grosch J. 2015. Individual and work factors related to perceived work ability and labor force outcomes. J. Appl. Psychol. 100(2):376-98
- Nichols D, Greene R, Schmolck P. 1989. Criteria for assessing inconsistent patterns of item endorsement on the MMPI: rationale, development, and empirical trials. J. Clin. Psychol. 45(2):239-50
- Patton J, Cheng Y, Hong M, Diao Q. 2019. Detection and treatment of careless responses to improve item parameter estimation. J. Educ. Behav. Stat. 44(3):309-41
- Rauti C. 2017. Assessing the effects of survey instructions and physical attractiveness on careless responding in online surveys. Master's Thesis, Univ. Windsor, Windsor, Can.
- Rios J, Guo H, Mao Liu O. 2017. Evaluating the impact of careless responding on aggregated-scores: to filter unmotivated examinees or not? Int. J. Test. 17(1):74-104
- Schneider S, May M, Stone A. 2018. Careless responding in internet-based quality of life assessments. Qual. Life. Res. 27(4):1077-88
- Thompson A. 1975. Random responding and the questionnaire measurement of psychoticism. Soc. Behav. Pers. 3(2):111-15
- Ward MK, Meade A. 2018. Applying social psychology to prevent careless responding during online surveys. Appl. Psychol. 67(2):231-63
- Woods C. 2006. Careless responding to reverse-worded items: implications for confirmatory factor analysis. J. Psychopathol. Behav. Assess. 28(3):186-91
- Yentes RD. 2020. In search of best practices for the identification and removal of careless responders. PhD Thesis, N.C. State Univ., Raleigh
- Errata An online log of corrections to Annual of Psychology ://www.annualreviews.org/errata/psych Related Articles From the Annual Review of Clinical Psychology, Volume 18 (2022) Temperamental and Theoretical Contributions to Clinical Psychology Jerome Kagan What Do We Know About the Genetic Architecture of Psychopathology?
- Richard E. Zinbarg, Alexander L. Williams, and Susan Mineka Dissociation and Dissociative Disorders Reconsidered: Beyond Sociocognitive and Trauma Models Toward a Transtheoretical Framework Steven Jay Lynn, Craig Polizzi, Harald Merckelbach, Chui-De Chiu, Reed Maxwell, Dalena van Heugten, and Scott O. Lilienfeld Psychosocial Treatments for Bipolar Disorder in Children and Adolescents Haley M. Brickman and Mary A. Fristad Major Depression and Its Recurrences: Life Course Matters Scott M. Monroe and Kate L. Harkness Suicide in African American Adolescents: Understanding Risk by Studying Resilience W. LaVome Robinson, Christopher R. Whipple, Kate Keenan, Caleb E. Flack, and LaRicka Wingate Psychopathy: Current Knowledge and Future Directions Christopher J. Patrick Cognitive Aging and the Promise of Physical Activity Kirk I. Erickson, Shannon D. Donofry, Kelsey R. Sewell, Belinda M. Brown, and Chelsea M. Stillman Neuroplasticity, the Prefrontal Cortex, and Psychopathology-Related Deviations in Cognitive Control Monica Luciana and Paul F. Collins The Biopsychosocial Puzzle of Painful Sex Marta Meana and Yitzchak M. Binik Mechanisms of Behavior Change in Substance Use Disorder With and Without Formal Treatment Katie Witkiewitz, Rory A. Pfund, and Jalie A. Tucker Police Violence and Public Health Jordan E. DeVylder, Deidre M. Anglin, Lisa Bowleg, Lisa Fedina, and Bruce G. Link Allostasis, Action, and Affect in Depression: Insights from the Theory of Constructed Emotion Clare Shaffer, Christiana Westlin, Karen S. Quigley, Susan Whitfield-Gabrieli, and Lisa Feldman Barrett The Psychology of Pandemics Steven Taylor From the Annual Review of Developmental Psychology, Volume 4 (2022) Becoming a Cognitive Scientist Susan E. Carey Drivers of Lexical Processing and Implications for Early Learning Arielle Borovsky Human Morality Is Based on an Early-Emerging Moral Core Brandon M. Woo, Enda Tan, and J. Kiley Hamlin On the Origins of Mind: A Comparative Perspective Kresimir Durdevic and Josep Call Sleep and Memory in Infancy and Childhood Gina M. Mason and Rebecca M.C. Spencer Effects of Racism on Child Development: Advancing Antiracist Developmental Science Iheoma U. Iruka, Nicole Gardner-Neblett, Nicole A. Telfer, Nneka Ibekwe-Okafor, Stephanie M. Curenton, Jacqueline Sims, Amber B. Sansbury, and Enrique W. Neblett Inequitable Experiences and Outcomes in Young Children: Addressing Racial and Social-Economic Disparities in Physical and Mental Health Brenda Jones Harden and Natalie Slopen Ownership and Value in Childhood Madison L. Pesowski, Shaylene E. Nancekivell, Arber Tasimi, and Ori Friedman Development of Religious Cognition Rebekah A. Richert and Kathleen H. Corriveau Gender Development in Gender Diverse Children Benjamin E. deMayo, Ashley E. Jordan, and Kristina R. Olson Development of Reward Circuitry During Adolescence: Depression, Social Context, and Considerations for Future Research on Disparities in Sexual and Gender Minority Youth
- Barry J. Milne, Stephanie D'Souza, Signe Hald Andersen, and Leah S. Richmond-Rakerd From the Annual Review of Neuroscience, Volume 45 (2022) Multiple-Timescale Representations of Space: Linking Memory to Navigation
- Xiao-Jing Wang Beyond Wrapping: Canonical and Noncanonical Functions of Schwann Cells Carla Taveggia and M. Laura Feltri Synaptic Mechanisms Regulating Mood State Transitions in Depression Puja K. Parekh, Shane B. Johnson, and Conor Liston From the Annual Review of Organizational Psychology and Organizational Behavior, Volume 9 (2022) From Traditional Research to Responsible Research: The Necessity of Scientific Freedom and Scientific Responsibility for Better Societies
- Anne S. Tsui Recovery from Work: Advancing the Field Toward the Future Sabine Sonnentag, Bonnie Hayden Cheng, and Stacey L. Parker The Science of Leadership: A Theoretical Model and Research Agenda Andrew M. Carton Stigmatized Work and Stigmatized Workers Glen Kreiner, Christine A. Mihelcic, and Sven Mikolon The Power of Listening at Work Avraham N. Kluger and Guy Itzchakov Compensation, Benefits, and Total Rewards: A Bird's-Eye (Re)View Ingrid Smithey Fulmer and Junting Li Smart Heuristics for Individuals, Teams, and Organizations Gerd Gigerenzer, Jochen Reb, and Shenghua Luan When Gender Matters in Organizational Negotiations Hannah Riley Bowles, Bobbi Thomason, and Inmaculada Macias-Alonso New Developments in Social Network Analysis Daniel J. Brass Trust Within the Workplace: A Review of Two Waves of Research and a Glimpse of the Third Kurt T. Dirks and Bart de Jong Cross-Cultural Innovation and Entrepreneurship Ute Stephan Relational Dynamics of Leadership: Problems and Prospects Terri A. Scandura and Jeremy D. Meuser The Structure of Intrinsic Motivation Ayelet Fishbach and Kaitlin Woolley Revisiting Behavioral Integrity: Progress and New Directions After 20 Years
- Tony Simons, Hannes Leroy, and Lisa Nishii Informal (Field-Based) Learning Scott I. Tannenbaum and Mikhail A. Wolfson Assessing Interests in the Twenty-First-Century Workforce: Building on a Century of Interest Measurement Christopher D. Nye Accumulating Knowledge in the Organizational Sciences Frank A. Bosco From the Annual Review of Public Health, Volume 43 (2022) Advances in Gender-Transformative Approaches to Health Promotion Jane Fisher and Shelly Makleff Methods to Address Confounding and Other Biases in Meta-Analyses: Review and Recommendations Maya B. Mathur and Tyler J. VanderWeele Qualitative Research Methods in Chronic Disease: Introduction and Opportunities to Promote Health Equity Rachel C. Shelton, Morgan M. Philbin, and Shoba Ramanadhan Risks and Opportunities to Ensure Equity in the Application of Big Data Research in Public Health Paul Wesson, Yulin Hswen, Gilmer Valdes, Kristefer Stojanovski, and Margaret A. Handley Social Epidemiology: Past, Present, and Future Ana V. Diez Roux The Recent Rise of Suicide Mortality in the United States
- Gonzalo Martínez-Alés, Tammy Jiang, Katherine M. Keyes, and Jaimie L. Gradus A Review of the Quality and Impact of Mobile Health Apps Quinn Grundy Reimagining Rural: Shifting Paradigms About Health and Well-Being in the Rural United States R.A. Afifi, E.A. Parker, G. Dino, D.M. Hall, and B. Ulin Scaling Up Public Health Interventions: Engaging Partners Across Multiple Levels Jennifer Leeman, Alix Boisson, and Vivian Go Social Capital, Black Social Mobility, and Health Disparities
- Ayden I. Scheim, Kellan E. Baker, Arjee J. Restar, and Randall L. Sell Mobile Health (mHealth) in Low-and Middle-Income Countries Judith McCool, Rosie Dobson, Robyn Whittaker, and Chris Paton Shifting the Demand for Vaccines: A Review of Strategies Neeraj Sood, Tahmina Nasserie, Sushant Joshi, and Eran Bendavid The Indian Health Service and American Indian/Alaska Native Health Outcomes Gina Kruse, Victor A. Lopez-Carmen, Anpotowin Jensen, Lakotah Hardie, and Thomas D. Sequist From the Annual Review of Vision Science, Volume 8 (2022) The Boston Keratoprosthesis-The First 50 Years: Some
Cited by
Sorted by citations
Dogru, E. Ö., & Krämer, N. C. (2025) · Journal of Decision Systems · doi:10.1080/12460125.2025.2593251 · Cited by 15 · see citing works
“Although no dedicated attention checks were included, inattentive responses were improbable given the precision required for entering valid chess moves and the targeted recruitment of chess-interested individuals (Ward & Meade, 2023).”
Are Attributions of Consciousness to AI Chatbots Epistemically Innocent?
Peters, U. (2026) · Minds and Machines · doi:10.1007/s11023-026-09795-8
- Authors
- Ward, M. K., & Meade, A. W.
- Affiliations
- Tacoma, Washington, USA;
- North Carolina State University (US)
- Corresponding author
- Meade, Adam W.
- Venue
- Annual Review of Psychology
- Publisher
- Annual Reviews
- Published
- 2023-01-18
- Volume
- 74
- Issue
- 1
- Pages
- 577-596
- ISSN
- 0066-4308 (ISSN-L)
- 1545-2085
- OpenAlex
- W4293870301
- Cited by
- 512
- References
- 7
- Citations by year
- 2026: 200 · 2025: 179 · 2024: 101 · 2023: 32
- FWCI
- 13.10
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Locations
- Annual Review of Psychology (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Sociology and Political Science › Survey Methodology and Nonresponse
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Physical Sciences › Mathematics › Statistics and Probability › Survey Sampling and Estimation Techniques
- Keywords
- careless responding, online surveys, survey data quality, data cleaning, self-report measures, screening thresholds
- MeSH terms
- Humans, Psychometrics, Surveys and Questionnaires, Self Report
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
