Ecological validity reconsidered: the Night Out Task versus the D-KEFS
Abstract
Introduction Although executive functioning (EF) correlates with execution of instrumental activities of daily living (IADLs), tests of EF have been criticized for having poor ecological validity. Attempts have been made to develop new tests that approximate naturalistic daily tasks. However, the incremental utility of such tests has not been convincingly demonstrated. The Night Out Task (NOT) is a novel measure designed to increase ecological validity. This study examined whether the NOT correlates with traditional lab- and home-based measures of EF and IADLs, and whether it outperforms traditional measures of EF in predicting IADLs. Method Participants (50 adults aged 60 to 95) completed (1) the Delis Kaplan Executive Function System (D-KEFS) and IADLs in the laboratory, and (2) ecological momentary assessment of EF and daily IADL tasks at home across three weeks (using the Daily Assessment of Independent Living and Executive Skills protocol; DAILIES). Results The NOT correlated with a lab-based measure of EF beyond covariates, and lab-based IADLs beyond covariates and beyond the D-KEFS. However, it was unrelated to at-home variables beyond covariates. In contrast, the D-KEFS was a significant predictor of at-home IADLs, and this association was mediated by at-home EF performance. Conclusion This study provides a preliminary validation of the NOT as a correlate of office-based performances in a primarily college educated white sample. Despite its high face validity, the NOT…
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The authors have repeatedly shown that naturalistic elements of a test do not necessarily improve upon prediction of objective real-world outcomes beyond measures with low face validity.
“Notably, we have repeatedly shown that naturalistic elements of a test do not necessarily improve upon prediction of objective realworld outcomes beyond measures with low face validity (Suchy et al., 2022; Suchy et al., in press; Ziemnik & Suchy, 2019), demonstrating that predictive validity cannot be assumed based on test appearance alone.”
From Conceptualization of the term 'ecological validity' in neuropsychological research on executive function assessment: a systematic review and call to action (2024) · cites this paper as Suchy et al., 2022
Cites
Neuropsychological studies of the frontal lobes.
Stuss, D. T., & Benson, D. F. (1984) · Psychological Bulletin · doi:10.1037/0033-2909.95.1.3 · Cited by 1,073 · see citing works
BURGESS, P. W., ALDERMAN, N., FORBES, C., COSTELLO, A., M-A.COATES, L., DAWSON, D. R., ANDERSON, N. D., GILBERT, S. J., DUMONTHEIL, I., & CHANNON, S. (2006) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617706060310 · Cited by 663 · see citing works
Moore, R. C., Depp, C. A., Wetherell, J. L., & Lenze, E. J. (2016) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2016.01.011 · Cited by 274 · see citing works
Zartman, A. L., Hilsabeck, R. C., Guarnaccia, C. A., & Houtz, A. (2013) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/act014 · Cited by 59 · see citing works
Rocke, K., Hays, P., Edwards, D., & Berg, C. (2008) · American Journal of Occupational Therapy · doi:10.5014/ajot.62.5.528 · Cited by 49 · see citing works
Ziemnik, R. E., & Suchy, Y. (2019) · Psychological Assessment · doi:10.1037/pas0000751 · Cited by 31 · see citing works
Self-citation
Flagged first, then by citations
Cited by
Sorted by citations
Suchy, Y., DesRuisseaux, L. A., Gereau Mora, M., Brothers, S. L., & Niermeyer, M. A. (2024) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617723000735 · Cited by 32 · see citing works
Self-citation
- Authors
- Suchy, Y., Lipio Brothers, S., DesRuisseaux, L. A., Gereau, M. M., Davis, J. R., Chilton, R. L. C., & Schmitter-Edgecombe, M.
- Affiliations
- University of Utah (US)
- Department of Psychology, University of Utah, Salt Lake City, UT, USA
- Washington State University (US)
- Department of Psychology, Washington State University, Pullman, WA, USA
- Corresponding author
- Suchy, Yana
- Venue
- Journal of Clinical and Experimental Neuropsychology
- Publisher
- Informa UK Limited
- Published
- 2022-09-14
- Volume
- 44
- Issue
- 8
- Pages
- 562-579
- ISSN
- 0165-0475 (ISSN-L)
- 1380-3395
- 1744-411X
- OpenAlex
- W4309656325
- Cited by
- 13
- References
- 6
- Citations by year
- 2026: 5 · 2025: 1 · 2024: 5 · 2023: 2
- FWCI
- 1.69
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical and Experimental Neuropsychology (publishedVersion)
- PubMed (publishedVersion)
- Funders
- funding associated with the work featured in this article
- Topics
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Cognitive Functions and Memory
- Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
- Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
- Keywords
- ecological validity, executive function, instrumental activities of daily living, Delis-Kaplan Executive Function System, dailies, ecological momentary assessment, older adults, home-based activities, test validation
- MeSH terms
- Activities of Daily Living, Adult, Humans, Executive Function, Independent Living
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
