CoChat
Sign Up
Cited by 452Psychological Testing and AssessmentHistorical and Modern Theater StudiesPsychometric Methodologies and Testing

Evidence-Based Indicators of Neuropsychological Change in the Individual Patient: Relevant Concepts and Methods

Open access
Duff, K.·2012·Archives of Clinical Neuropsychology
DOI 10.1093/arclin/acr120PMC PMC3499091
Open this paper in CoChat — free accountOpen PDF

Abstract

Repeated assessments are a relatively common occurrence in clinical neuropsychology. The current paper will review some of the relevant concepts (e.g., reliability, practice effects, alternate forms) and methods (e.g., reliable change index, standardized based regression) that are used in repeated neuropsychological evaluations. The focus will be on the understanding and application of these concepts and methods in the evaluation of the individual patient through examples. Finally, some future directions for assessing change will be described.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1093/arclin/acr120

Cites

4 of 4 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 63

  1. Attix, D. K., Story, T. J., Chelune, G. J., Ball, J. D., Stutts, M. L., Hart, R. P., et al. (2009). The prediction of change: Normative neuropsychological trajectories. The Clinical Neuropsychologist, 23 (1), 21-38.
  2. Barr, W. B. (2003). Neuropsychological testing of high school athletes. Preliminary norms and test-retest indices. Archives of Clinical Neuropsychology, 18 (1), 91-101.
  3. Barr, W. B., & McCrea, M. (2001). Sensitivity and specificity of standardized neurocognitive testing immediately following sports concussion. Journal of the International Neuropsychological Society, 7 (6), 693-702.
  4. Beglinger, L. J., Gaydos, B., Tangphao-Daniels, O., Duff, K., Kareken, D. A., Crawford, J., et al. (2005). Practice effects and the use of alternate forms in serial neuropsychological testing. Archives of Clinical Neuropsychology, 20 (4), 517-529.
  5. Benedict, R. H. (2005). Effects of using same-versus alternate-form memory tests during short-interval repeated assessments in multiple sclerosis. Journal of the International Neuropsychological Society, 11 (6), 727-736.
  6. Benedict, R. H., Schretlen, D., Groninger, L., & Brandt, J. (1998). Hopkins verbal learning test-revised: Normative data and analysis of inter-form and test-retest reliability. The Clinical Neuropsychologist, 12 (1), 43-55.
  7. Calero, M. D., & Navarro, E. (2007). Cognitive plasticity as a modulating variable on the effects of memory training in elderly persons. Archives of Clinical Neuropsychology, 22 (1), 63-72.
  8. Chelune, G. J., Naugle, R. I., Luders, H., Sedlak, J., & Awad, I. A. (1993). Individual change after epilepsy surgery: Practice effects and base-rate information. Neuropsychology, 7 (1), 41-52.
  9. Crawford, J. R., & Garthwaite, P. H. (2006). Comparing patients' predicted test scores from a regression equation with their obtained scores: A significance test and point estimate of abnormality with accompanying confidence limits. Neuropsychology, 20 (3), 259-271.
  10. Crawford, J. R., & Garthwaite, P. H. (2007). Using regression equations built from summary data in the neuropsychological assessment of the individual case. Neuropsychology, 21 (5), 611-620.
  11. Cysique, L. A., Franklin, D., Jr., Abramson, I., Ellis, R. J., Letendre, S., Collier, A., et al. (2011) Normative data and validation of a regression based summary score for assessing meaningful neuropsychological change. Journal of Clinical and Experimental Neuropsychology, 33 (5), 505-522.
  12. Darby, D., Maruff, P., Collie, A., & McStephen, M. (2002). Mild cognitive impairment can be detected by multiple assessments in a single day. Neurology, 59 (7), 1042-1046.
  13. Delis, D. C., Kramer, J. H., Kaplan, E., & Ober, B. A. (2000). California Verbal Learning Test-Second Edition. San Antonio: Psychological Corporation.
  14. Duff, K., Beglinger, L. J., Moser, D. J., & Paulsen, J. S. (2010). Predicting cognitive change within domains. Clinical Neuropsychology, 24 (5), 779-792.
  15. Duff, K., Beglinger, L. J., Moser, D. J., Paulsen, J. S., Schultz, S. K., & Arndt, S. (2010). Predicting cognitive change in older adults: The relative contribution of practice effects. Archives of Clinical Neuropsychology, 25 (2), 81-88.
  16. Duff, K., Beglinger, L. J., Moser, D. J., Schultz, S. K., & Paulsen, J. S. (2010). Practice effects and outcome of cognitive training: Preliminary evidence from a memory training course. American Journal of Geriatric Psychiatry, 18 (1), 91.
  17. Duff, K., Beglinger, L., Schultz, S., Moser, D., McCaffrey, R., Haase, R., et al. (2007). Practice effects in the prediction of long-term cognitive outcome in three patient samples: A novel prognostic index. Archives of Clinical Neuropsychology, 22 (1), 15-24.
  18. Duff, K., Beglinger, L., Van Der Heiden, S., Moser, D., Arndt, S., Schultz, S., et al. (2008). Short-term practice effects in amnestic mild cognitive impairment: Implications for diagnosis and treatment. International Psychogeriatrics, 20 (5), 986-999.
  19. Duff, K., Schoenberg, M. R., Patton, D. E., Mold, J., Scott, J. G., & Adams, R. A. (2004). Predicting change with the RBANS in a community dwelling elderly sample. Journal of the International Neuropsychological Society, 10, 828-834.
  20. Duff, K., Schoenberg, M. R., Patton, D. E., Mold, J. W., Scott, J. G., & Adams, R. L. (2008). Predicting cognitive change across 3 years in community-dwelling elders. Clinical Neuropsychology, 22 (4), 651-661.
  21. Frerichs, R. J., & Tuokko, H. A. (2005). A comparison of methods for measuring cognitive change in older adults. Archives of Clinical Neuropsychology, 20 (3), 321-333.
  22. Friedman, M. A., Fernandez, M., Wefel, J. S., Myszka, K. A., Champlin, R. E., & Meyers, C. A. (2009). Course of cognitive decline in hematopoietic stem cell transplantation: A within-subjects design. Archives of Clinical Neuropsychology, 24 (7), 689-698.
  23. Heaton, R. K., Temkin, N., Dikmen, S., Avitable, N., Taylor, M. J., Marcotte, T. D., et al. (2001). Detecting change: A comparison of three neuropsychological methods, using normal and clinical samples. Archives of Clinical Neuropsychology, 16 (1), 75-91.
  24. Hensel, A., Luck, T., Luppa, M., Glaesmer, H., Angermeyer, M. C., & Riedel-Heller, S. G. (2009). Does a reliable decline in Mini Mental State Examination total score predict dementia? Diagnostic accuracy of two reliable change indices. Dementia and Geriatric Cognitive Disorders, 27 (1), 50-58.
  25. Hermann, B. P., Seidenberg, M., Schoenfeld, J., Peterson, J., Leveroni, C., & Wyler, A. R. (1996). Empirical techniques for determining the reliability, mag- nitude, and pattern of neuropsychological change after epilepsy surgery. Epilepsia, 37 (10), 942-950.
  26. Higginson, C. I., Wheelock, V. L., Levine, D., King, D. S., Pappas, C. T., & Sigvardt, K. A. (2009). The clinical significance of neuropsychological changes following bilateral subthalamic nucleus deep brain stimulation for Parkinson's disease. Journal of Clinical and Experimental Neuropsychology, 31 (1), 65- 72.
  27. Hinton-Bayre, A. D. (2005). Methodology is more important than statistics when determining reliable change. Journal of the International Neuropsychological Society, 11 (6), 788-789.
  28. Hinton-Bayre, A. D. (2010). Deriving reliable change statistics from test-retest normative data: Comparison of models and mathematical expressions. Archives of Clinical Neuropsychology, 25 (3), 244-256.
  29. Iverson, G. L. (2001). Interpreting change on the WAIS-III/WMS-III in clinical samples. Archives of Clinical Neuropsychology, 16 (2), 183-191.
  30. Ivnik, R. J., Smith, G. E., Lucas, J. A., Petersen, R. C., Boeve, B. F., Kokmen, E., et al. (1999). Testing normal older people three or four times at 1-to 2-year intervals: Defining normal variance. Neuropsychology, 13 (1), 121-127.
  31. Koh, C. L., Lu, W. S., Chen, H. C., Hsueh, I. P., Hsieh, J. J., & Hsieh, C. L. (2011) Test-retest reliability and practice effect of the oral-format Symbol Digit Modalities Test in patients with stroke. Archives of Clinical Neuropsychology, 26 (4), 356-363.
  32. Kormi-Nouri, R., Nilsson, L. G., & Ohta, N. (2005). The novelty effect: Support for the novelty-encoding hypothesis. Scandinavian Journal of Psychology, 46 (2), 133-143.
  33. Levine, A. J., Hinkin, C. H., Miller, E. N., Becker, J. T., Selnes, O. A., & Cohen, B. A. (2007). The generalizability of neurocognitive test/retest data derived from a nonclinical sample for detecting change among two HIV+ cohorts. Journal of Clinical and Experimental Neuropsychology, 29 (6), 669-678.
  34. Loring, D. W., Williamson, D. J., Meador, K. J., Wiegand, F., & Hulihan, J. (2011) Topiramate dose effects on cognition: A randomized double-blind study. Neurology, 76 (2), 131-137.
  35. Maassen, G. H., Bossema, E. R., & Brand, N. (2006). Reliable change assessment with practice effects in sport concussion research: A comment on Hinton-Bayre. British Journal of Sports Medicine, 40 (10), 829-833.
  36. Maassen, G. H., Bossema, E., & Brand, N. (2009). Reliable change and practice effects: Outcomes of various indices compared. Journal of Clinical and Experimental Neuropsychology, 31 (3), 339-352.
  37. Martin, R., Griffith, H. R., Sawrie, S., Knowlton, R., & Faught, E. (2006). Determining empirically based self-reported cognitive change: Development of reliable change indices and standardized regression-based change norms for the multiple abilities self-report questionnaire in an epilepsy sample. Epilepsy Behavior, 8 (1), 239-245.
  38. Martin, R., Sawrie, S., Gilliam, F., Mackey, M., Faught, E., Knowlton, R., et al. (2002). Determining reliable cognitive change after epilepsy surgery: Development of reliable change indices and standardized regression-based change norms for the WMS-III and WAIS-III. Epilepsia, 43 (12), 1551-1558.
  39. Mikos, A., Zahodne, L., Okun, M. S., Foote, K., & Bowers, D. (2010). Cognitive declines after unilateral deep brain stimulation surgery in Parkinson's disease: A controlled study using Reliable Change, part II. Clinical Neuropsychology, 24 (2), 235-245.
  40. Millis, S. R., Rosenthal, M., Novack, T. A., Sherer, M., Nick, T. G., Kreutzer, J. S., et al. (2001). Long-term neuropsychological outcome after traumatic brain injury. Journal of Head Trauma Rehabilitation, 16 (4), 343-355.
  41. Mohile, S. G., Lacy, M., Rodin, M., Bylow, K., Dale, W., Meager, M. R., et al. (2010) Cognitive effects of androgen deprivation therapy in an older cohort of men with prostate cancer. Critical Reviews in Oncology/Hematology, 75 (2), 152-159.
  42. Muslimovic, D., Post, B., Speelman, J. D., De Haan, R. J., & Schmand, B. (2009). Cognitive decline in Parkinson's disease: A prospective longitudinal study. Journal of the International Neuropsychological Society, 15 (3), 426 -437.
  43. Nakhutina, L., Pramataris, P., Morrison, C., Devinsky, O., & Barr, W. B. (2010). Reliable change indices and regression-based measures for the Rey-Osterreith Complex Figure test in partial epilepsy patients. Clinical Neuropsychology, 24 (1), 38-44.
  44. O'Mahar, K. M., Duff, K., Scott, J. G., Linck, J. F., Adams, R. L., & Mold, J. W. (in press). The temporal stability of the Repeatable Battery for the Assessment of Neuropsychological Status Effort Index in geriatric samples. Archives of Clinical Neuropsychology.
  45. Ouimet, L. A., Stewart, A., Collins, B., Schindler, D., & Bielajew, C. (2009). Measuring neuropsychological change following breast cancer treatment: An analysis of statistical models. Journal of Clinical and Experimental Neuropsychology, 31 (1), 73-89.
  46. Patton, D. E., Duff, K., Schoenberg, M. R., Mold, J., Scott, J. G., & Adams, R. L. (2005). Base rates of longitudinal RBANS discrepancies at one-and two-year intervals in community-dwelling older adults. Clinical Neuropsychology, 19 (1), 27-44.
  47. Poppenk, J., Kohler, S., & Moscovitch, M. (2010). Revisiting the novelty effect: When familiarity, not novelty, enhances memory. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36 (5), 1321-1330.
  48. Portaccio, E., Goretti, B., Zipoli, V., Iudice, A., Pina, D. D., Malentacchi, G. M., et al. (2010). Reliability, practice effects, and change indices for Rao's Brief Repeatable Battery. Multiple Sclerosis, 16 (5), 611-617.
  49. Rapport, L. J., Brines, D. B., Axelrod, B. N., & Theisen, M. E. (1997). Full scale IQ as a mediator of practice effects: The rich get richer. The Clinical Neuropsychologist, 11 (4), 375-380.
  50. Reid-Arndt, S. A., Hsieh, C., & Perry, M. C. (2010). Neuropsychological functioning and quality of life during the first year after completing chemotherapy for breast cancer. Psychooncology, 19 (5), 535-544.
  51. Rinehardt, E., Duff, K., Schoenberg, M., Mattingly, M., Bharucha, K., & Scott, J. (2010) Cognitive change on the repeatable battery of neuropsychological status (RBANS) in Parkinson's disease with and without bilateral subthalamic nucleus deep brain stimulation surgery. Clinical Neuropsychology, 24 (8), 1339-1354.
  52. Salthouse, T. A. (2010). Influence of age on practice effects in longitudinal neurocognitive change. Neuropsychology, 24 (5), 563-572.
  53. Sawrie, S. M., Chelune, G. J., Naugle, R. I., & Luders, H. O. (1996). Empirical methods for assessing meaningful neuropsychological change following epi- lepsy surgery. Journal of the International Neuropsychological Society, 2 (6), 556-564.
  54. Schoenberg, M. R., Rinehardt, E., Duff, K., Mattingly, M., Bharucha, K. G., & Scott, J. G. (in press) Assessing reliable change using the Repeatable Battery for the Assessment of Neuropsychological Status for Patients with Parkinson's disease undergoing Deep Brain Stimulation (DBS) surgery. The Clinical Neuropsychologist.
  55. Sherman, E. M., Wiebe, S., Fay-McClymont, T. B., Tellez-Zenteno, J., Metcalfe, A., Hernandez-Ronquillo, L., et al. (2011). Neuropsychological outcomes after epilepsy surgery: Systematic review and pooled estimates. Epilepsia, 52 (5), 857 -869.
  56. Suchy, Y., Kraybill, M. L., & Franchow, E. (2011). Practice effect and beyond: Reaction to novelty as an independent predictor of cognitive decline among older adults. Journal of the International Neuropsychological Society, 17 (1), 101-111.
  57. Temkin, N. R., Heaton, R. K., Grant, I., & Dikmen, S. S. (1999). Detecting significant change in neuropsychological test performance: A comparison of four models. Journal of the International Neuropsychological Society, 5 (4), 357 -369.
  58. Troster, A. I., Woods, S. P., & Morgan, E. E. (2007). Assessing cognitive change in Parkinson's disease: Development of practice effect-corrected reliable change indices. Archives of Clinical Neuropsychology, 22 (6), 711-718.
  59. Vearncombe, K. J., Rolfe, M., Wright, M., Pachana, N. A., Andrew, B., & Beadle, G. (2009). Predictors of cognitive decline after chemotherapy in breast cancer patients. Journal of the International Neuropsychological Society, 15 (6), 951-962.
  60. Wechsler, D. (2008). WAIS-IV technical and interpretive manual. San Antonio: Pearson.
  61. Woods, S. P., Childers, M., Ellis, R. J., Guaman, S., Grant, I., & Heaton, R. K. (2006). A battery approach for measuring neuropsychological change. Archives of Clinical Neuropsychology, 21 (1), 83-89.
  62. Zabel, T. A., von Thomsen, C., Cole, C., Martin, R., & Mahone, E. M. (2009). Reliability concerns in the repeated computerized assessment of attention in children. Clinical Neuropsychology, 23 (7), 1213-1231.
  63. Zahodne, L. B., Okun, M. S., Foote, K. D., Fernandez, H. H., Rodriguez, R. L., Kirsch-Darrow, L., et al. (2009). Cognitive declines one year after unilateral deep brain stimulation surgery in Parkinson's disease: A controlled study using reliable change. Clinical Neuropsychology, 23 (3), 385-405.

Cited by

Sorted by citations

Authors
Duff, K.
Affiliations
University of Utah (US)
Corresponding author
Duff, K.
ORCID
Duff, K.
Venue
Archives of Clinical Neuropsychology
Publisher
Oxford University Press (OUP)
Published
2012-02-29
Volume
27
Issue
3
Pages
248-261
ISSN
  • 0887-6177 (ISSN-L)
  • 1873-5843
DOI
10.1093/arclin/acr120
PMC
PMC3499091
OpenAlex
W2110090942
Cited by
452
References
4
Citations by year
2026: 45 · 2025: 51 · 2024: 42 · 2023: 41 · 2022: 39 · 2021: 38 · 2020: 37 · 2019: 32 · 2018: 20 · 2017: 30 · 2016: 28 · 2015: 19 · 2014: 11 · 2013: 12 · 2012: 7
FWCI
3.64
Type
journal article
Language
en
Open access
Locations
  • Archives of Clinical Neuropsychology (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • National Academy of Neuropsychology
  • National Institutes of Health
  • National Institute on Aging (K23AG028417)
Topics
  • Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
  • Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
  • Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Keywords
neuropsychological changes, reliable change index, practice effects, alternate forms, test-retest reliability
MeSH terms
Age Factors, Mental Disorders, Humans, Neuropsychological Tests, Neuropsychology, Sex Factors, Time Factors, Reproducibility of Results, Outcome Assessment, Health Care, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review