I Just Want My Research Credit: Frequency of Suboptimal Effort in a Non-Clinical Healthy Undergraduate Sample
Abstract
Although performance validity testing is becoming fairly routine in clinical settings, research protocols involving neuropsychological tests infrequently include assessments of performance validity. The current study utilized an embedded measure of effort over two administrations of CNS Vital Signs to determine the frequency of poor effort in non-clinical healthy undergraduate students participating in a research study for course credit. Results indicate that more than 1 in 10 college students participating in a cognitive test battery for research showed test scores consistent with inadequate effort, which was associated with poor performance on testing across many domains. This conclusion was supported by poor performance on many other subtests. Healthy college students with suboptimal effort (n = 11) had an overall score in the 15th percentile on average compared to the 48th percentile in the rest of the students (n = 66). Those who failed validity indicators on the baseline administration were more likely to fail validity indicators on the repeat administration. Those who were tested in the morning were also more likely to fail validity indicators. The current study provides evidence for the potential limitations of conducting research using neuropsychological tests with healthy college student volunteers in the absence of performance validity testing. Revised college-level cutoffs are proposed.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Participation from the comfort of one's own home could have contributed to careless responding and suboptimal effort.
“Participation from the comfort of one's own home could have contributed to careless responding (Goldammer et al., 2020) and suboptimal effort (DeRight & Jorgensen, 2015).”
From Mindfulness Effects on Anxiety: Disentangling the Role of Decentering and Treatment Expectations (2024) · cites this paper as DeRight & Jorgensen, 2015
A lack of participant motivation has a negative impact on the quality of data collected.
“There is evidence that a lack of participant motivation has a negative impact on the quality of data collected [4]”
From Gamification of Cognitive Assessment and Cognitive Training: A Systematic Review of Applications and Efficacy (2016) · cites this paper as [4]
Cites
The Halstead-Reitan neuropsychological test battery: Theory and clinical interpretation
Reitan, R. M., & Wolfson, D. E. (1993) · Medical Entomology and Zoology · Cited by 3,413 · see citing works
Paced Auditory Serial-Addition Task: A Measure of Recovery from Concussion
Gronwall, D. M. A. (1977) · Perceptual and Motor Skills · doi:10.2466/pms.1977.44.2.367 · Cited by 2,395 · see citing works
The relation of the Trail Making Test to organic brain damage.
Reitan, R. M. (1955) · Journal of Consulting Psychology · doi:10.1037/h0044509 · Cited by 1,817 · see citing works
California Verbal Learning Test
Delis, D. C., Freeland, J., Kramer, J. H., & Kaplan, E. (1987) · PsycTESTS Dataset · doi:10.1037/t48844-000 · Cited by 1,450 · see citing works
Archives of Clinical Neuropsychology
Reynolds, C. R. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0182 · Cited by 1,054 · see citing works
Reliability and validity of a computerized neurocognitive test battery, CNS vital signs
GUALTIERI, C., & JOHNSON, L. (2006) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2006.05.007 · Cited by 873 · see citing works
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants1, C. (2009) · The Clinical Neuropsychologist · doi:10.1080/13854040903155063 · Cited by 869 · see citing works
Symptom validity assessment: Practice issues and medical necessityNAN Policy & Planning Committee
BUSH, S., Ruff, R. M., Tröster, A. I., Barth, J. T., Koffler, S. P., Pliskin, N. H., REYNOLDS, C., & Silver, C. (2005) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2005.02.002 · Cited by 836 · see citing works
Thomas, R. K. (2016) · The SAGE Encyclopedia of Theory in Psychology · doi:10.4135/9781483346274.n39 · Cited by 752 · see citing works
Tombaugh, T. N. (1997) · Psychological Assessment · doi:10.1037/1040-3590.9.3.260 · Cited by 582 · see citing works
Sensitivity and specificity of the ImPACT Test Battery for concussion in athletes
Philip Schatz, A., Pardini, J., LOVELL, M., Collins, M., & Podell, K. (2005) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2005.08.001 · Cited by 576 · see citing works
Passingham, R. E. (2008) · Brain Behavior and Evolution · doi:10.1159/000123620 · Cited by 285 · see citing works
Wintre, M. G., North, C., & Sugar, L. A. (2001) · Canadian Psychology/Psychologie canadienne · doi:10.1037/h0086893 · Cited by 164 · see citing works
Mild Traumatic Brain Injury: Symptom Validity Assessment and Malingering
Schroeder, R. W. (2012) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acs110 · Cited by 143 · see citing works
Neuropsychological testing of high school athletes Preliminary norms and test-retest indices
Barr, W. (2003) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/18.1.91 · Cited by 139 · see citing works
PERFORMANCE ON THE TEST OF MEMORY MALINGERING IN A MIXED PEDIATRIC SAMPLE
Donders, J. (2005) · Child Neuropsychology · doi:10.1080/09297040490917298 · Cited by 116 · see citing works
An, K. Y., Zakzanis, K. K., & Joordens, S. (2012) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acs085 · Cited by 108 · see citing works
Carone, D. A. (2008) · Brain Injury · doi:10.1080/02699050802491297 · Cited by 106 · see citing works
Schatz, P., & Glatts, C. (2013) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/act009 · Cited by 99 · see citing works
Assessment of effort in children: A systematic review
DeRight, J., & Carone, D. A. (2013) · Child Neuropsychology · doi:10.1080/09297049.2013.864383 · Cited by 75 · see citing works
Self-citationO’Bryant, S. E., & Lucas, J. A. (2006) · The Clinical Neuropsychologist · doi:10.1080/13854040590967568 · Cited by 70 · see citing works
Noncredible Effort during Pediatric Neuropsychological Exam: A Case Series and Literature Review
Kirkwood, M. W., Kirk, J. W., Blaha, R. Z., et al. (2010) · Child Neuropsychology · doi:10.1080/09297049.2010.495059 · Cited by 69 · see citing works
Detecting simulation of attention deficits using reaction time tests☆
Willison, J. R., & Tombaugh, T. N. (2005) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2005.07.005 · Cited by 66 · see citing works
Armistead‐Jehle, P., & Hansen, C. L. (2011) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acr049 · Cited by 59 · see citing works
Bauer, L., Yantz, C. L., Ryan, L. M., et al. (2005) · Applied Neuropsychology · doi:10.1207/s15324826an1204_3 · Cited by 52 · see citing works
Forbey, J. D., Lee, T. T. C., Ben‐Porath, Y. S., Arbisi, P. A., & Gartland, D. M. (2013) · Assessment · doi:10.1177/1073191113478154 · Cited by 47 · see citing works
Base Rate of Performance Invalidity among Non-Clinical Undergraduate Research Participants
Eglit, G. M. L., Stenclik, J. H., Gavett, B. E., Adams, J. W., Lynch, J. K., & Mccaffrey, R. J. (2014) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acu028 · Cited by 36 · see citing works
Victoria Symptom Validity Test Performance in Children and Adolescents with Neurological Disorders
Brooks, B. L. (2012) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acs087 · Cited by 29 · see citing works
Effort in College Undergraduates Is Sufficient on the Word Memory Test
Santos, O. A., Kazakov, D., Reamer, M. K., et al. (2014) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acu039 · Cited by 28 · see citing works
Continuing Challenges for Computer-based Neuropsychological Tests
Letz, R. (2003) · NeuroToxicology · doi:10.1016/s0161-813x(03)00047-0 · Cited by 26 · see citing works
An Exploration of the Impact of Invalid MMPI–2 Protocols on Collateral Self-Report Measure Scores
Forbey, J. D., & Lee, T. T. C. (2011) · Journal of Personality Assessment · doi:10.1080/00223891.2011.608757 · Cited by 24 · see citing works
Embedded Validity Indicators on CNS Vital Signs in Youth with Neurological Diagnoses
Brooks, B. L., Sherman, E. M. S., & Iverson, G. L. (2014) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acu029 · Cited by 16 · see citing works
Subject-Selection Procedures in Neuropsychological Research: A Meta-Analysis and Prospective Study
Stanczak, E. M., Stanczak, D. E., & Templer, D. I. (2000) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/15.7.587 · Cited by 12 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Methods for the detection of carelessly invalid responses in survey data
Curran, P. G. (2016) · Journal of Experimental Social Psychology · doi:10.1016/j.jesp.2015.07.006 · Cited by 1,356 · see citing works
Lumsden, J., Edwards, E. A., Lawrence, N. S., Coyle, D., & Munafò, M. R. (2016) · JMIR Serious Games · doi:10.2196/games.5888 · Cited by 555 · see citing works
“There is evidence that a lack of participant motivation has a negative impact on the quality of data collected [4], with tasks commonly being viewed as too boring and repetitive for participant attention to be sustained.”
BackgroundMindfulness Effects on Anxiety: Disentangling the Role of Decentering and Treatment Expectations
Bos, D. P. A., Keesman, M., Roggeveen, A., Vase, L., Evers, A. W. M., & Peerdeman, K. J. (2024) · Behavior Therapy · doi:10.1016/j.beth.2024.03.004 · Cited by 5 · see citing works
Stone, B. M. (2025) · International Journal of Mental Health and Addiction · doi:10.1007/s11469-023-01151-6 · Cited by 3 · see citing works
- Authors
- DeRight, J., & Jorgensen, R. S.
- Affiliations
- Syracuse University (US)
- Corresponding author
- DeRight, Jonathan
- Venue
- The Clinical Neuropsychologist
- Publisher
- Informa UK Limited
- Published
- 2014-12-10
- Volume
- 29
- Issue
- 1
- Pages
- 101-117
- ISSN
- 0920-1637 (ISSN-L)
- 1385-4046
- 1744-4144
- OpenAlex
- W1982809779
- Cited by
- 93
- References
- 33
- Citations by year
- 2026: 6 · 2025: 7 · 2024: 9 · 2023: 12 · 2022: 12 · 2021: 11 · 2020: 7 · 2019: 4 · 2018: 5 · 2017: 5 · 2016: 11 · 2015: 4
- FWCI
- 10.32
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- The Clinical Neuropsychologist (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Abilities and Testing
- Keywords
- performance validity, CNS Vital Signs, neuropsychological test validity, research volunteers, validity indicators, cut-off values, cognitive test performance
- MeSH terms
- Adult, Female, Humans, Male, Neuropsychological Tests, Research, Students, Universities, Intellectual Property
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
