The night out task and scoring application: an ill-structured, open-ended clinic-based test representing cognitive capacities used in everyday situations
The DOI matches its Crossref record, with a small difference in title or year; 11 of the 11 references it cites were found in a registry or an index.
Abstract
Objective: The night out task (NOT) was developed as a naturalistic, open-ended, multitasking measure that requires individuals to complete eight subtasks comparable to those encountered during real-world functioning (e.g., pack travel bag, prepare tea). We examined psychometric properties and administration feasibility of this direct observation measure within a clinic-like setting using a tablet-based coding application. Method: A sample of 148 community-dwelling older adults (82% cognitively healthy; 18% mild cognitive impairment) and 57 younger adults completed the NOT along with other neurocognitive tests and questionnaires. Results: Inter-rater reliability across NOT primary (i.e., time, accuracy, efficiency) and process-related (e.g., error-types, self-corrections) variables was mostly excellent. NOT primary measures showed expected patterns of convergent and discriminant validity with measures of cognition, demographics, and well-being. External validity was established by the NOT ability to distinguish between age and diagnostic (cognitively healthy vs. mild cognitive impairment) groups. Demonstrating incremental validity, the NOT primary variables (execution time in particular) were predictive of self-reported functional abilities and completion quality of in-home everyday tasks over and earlier variables such as demographics, cognition, and mobility. Conclusions: These findings suggest that the NOT and its app interface, which allows for continuous logging of observations, are a feasible in-clinic measure to assess cognitive capacities important for real-world functioning. With further validation, the NOT may allow for earlier detection of functional difficulties. Understanding errors and strategies used during NOT performance could also have implications for individualized interventions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Objective: ' Ecological validity ' (EV) is classically defined as test ' s ability to predict real-world functioning, either alone or together with test ' s similarity to real-world tasks.”
From Conceptualization of the term 'ecological validity' in neuropsychological research on executive function assessment: a systematic review and call to action (2024) · cites this paper as Sbordone, 1996
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Pilkonis, P. A., Choi, S. W., Reise, S. P., Stover, A. M., Riley, W. T., Cella, D., & PROMIS Cooperative Group (2011) · Assessment · doi:10.1177/1073191111411667 · Cited by 2,132
DEFICITS IN STRATEGY APPLICATION FOLLOWING FRONTAL LOBE DAMAGE IN MAN
SHALLICE, T., & BURGESS, P. W. (1991) · Brain · doi:10.1093/brain/114.2.727 · Cited by 2,017
Assessment of executive functions: Review of instruments and identification of critical issues.
CHAN, R., SHUM, D., TOULOPOULOU, T., & CHEN, E. (2008) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2007.08.010 · Cited by 1,401
Yu, L., Buysse, D. J., Germain, A., Moul, D. E., Stover, A., Dodds, N. E., Johnston, K. L., & Pilkonis, P. A. (2012) · Behavioral Sleep Medicine · doi:10.1080/15402002.2012.636266 · Cited by 1,241
Behavioural Assessment of the Dysexecutive Syndrome
(2004) · Methodology Of Frontal And Executive Function · doi:10.4324/9780203344187-15 · Cited by 1,149
Quantification of Five Neuropsychological Approaches to Defining Mild Cognitive Impairment
Jak, A. J., Bondi, M. W., Delano-Wood, L., Wierenga, C., Corey-Bloom, J., Salmon, D. P., & Delis, D. C. (2009) · The American Journal of Geriatric Psychiatry · doi:10.1097/jgp.0b013e31819431d5 · Cited by 823
BURGESS, P. W., ALDERMAN, N., FORBES, C., COSTELLO, A., M-A.COATES, L., DAWSON, D. R., ANDERSON, N. D., GILBERT, S. J., DUMONTHEIL, I., & CHANNON, S. (2006) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617706060310 · Cited by 663
Strategy application disorder: the role of the frontal lobes in human multitasking
Burgess, P. W. (2000) · Psychological Research · doi:10.1007/s004269900006 · Cited by 368
Ecological validity of a simplified version of the multiple errands shopping test
ALDERMAN, N., BURGESS, P. W., KNIGHT, C., & HENMAN, C. (2003) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617703910046 · Cited by 337 · cited 1×
“Prior studies also reinforce the importance of capturing data other than accuracy and time when predicting real-world functioning, including process-related variables such as the types of errors being made, strategies used, and task efficiency (Alderman et al., 2003; Sanders, Low, & Schmitter-Edgecombe, 2014).”
MethodDevelopment of a simplified version of the multiple errands test for use in hospital settings
Knight, C., Alderman, N., & Burgess, P. W. (2002) · Neuropsychological Rehabilitation · doi:10.1080/09602010244000039 · Cited by 197
Research works printed but not linked: 54
- Albert, 2011, The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendation from the National Institute of Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease, Alzheimer’s & Dementia, 7, 270, 10.1016/j.jalz.2011.03.008 · DOI · Paper page
- Bailey, 2013, Action perception predicts action performance, Neuropsychologia, 51, 2294, 10.1016/j.neuropsychologia.2013.06.022 · DOI
- Baum, 2003, Test manual: Executive function performance test
- Brandt, 2003, Telephone interview for cognitive status
- Burgess, 1997, Methodology of frontal and executive function, 81
- Bulzacka, 2016, Clinical utility of the multiple errands test in schizophrenia: A preliminary assessment, Psychiatry research, 240, 390, 10.1016/j.psychres.2016.04.056 · DOI
- Burns, 2019, Associations between self-generated strategy use and MET-home performance in adults with stroke, Neuropsychological Rehabilitation, 10.1080/09602011.2019.1601112 · DOI
- Burton, 2009, Functional abilities in older adults with mild cognitive impairment, Gerontology, 55, 570, 10.1159/000228918 · DOI · Paper page
- Castiel, 2012, Use of the multiple errands test – simplified version in the assessment of suboptimal effort, Neuropsychological Rehabilitation, 22, 734, 10.1080/09602011.2012.686884 · DOI
- Cipresso, 2014, Virtual Multiple Errands Test (VMET): A virtual reality-based tool to detect early executive functions deficit in Parkinson's disease, Frontiers in Behavioral Neuroscience, 8, 405, 10.3389/fnbeh.2014.00405 · DOI · Paper page
- Clark, 2017, Reliability and construct validity of a revised Baycrest multiple errands test, Neuropsychological Rehabilitation, 27, 667, 10.1080/09602011.2015.1117981 · DOI
- Craik, 2006, Planning and task management in older adults: Cooking breakfast, Memory & Cognition, 34, 1236, 10.3758/bf03193268 · DOI · Paper page
- Delis, 2001, Delis-Kaplan executive function system: Examiner’s manual
- Dassel, 2008, The impact of caregiver executive skills on reports of patient functioning, The Gerontologist, 48, 781, 10.1093/geront/48.6.781 · DOI
- Dawadi, 2013, Automated cognitive health assessment using smart home monitoring of complex tasks, IEEE Transactions on Systems, Man and Cybernetics, 43, 1302, 10.1109/TSMC.2013.2252338 · DOI · Paper page
- Dawson, 2009, Further development of the multiple errands test: Standardized scoring, reliability, and ecological validity for the Baycrest version, Archives of Physical Medicine and Rehabilitation, 90, S41, 10.1016/j.apmr.2009.07.012 · DOI
- Tomaszewski Farias, 2020, The everyday compensation (EComp) questionnaire: Construct validity and associations with diagnosis and longitudinal change in cognition and everyday function in older adults, Journal of the International Neuropsychological Society, 26, 303, 10.1017/S135561771900119X · DOI
- Feinkohl, 2016, Reheating breakfast: Age and multitasking on a computer-based and a non-computer-based task, Computers in Human Behavior, 55, 432, 10.1016/j.chb.2015.09.040 · DOI
- Finlayson, 2003, Integrating a performance-based observation measure of functional status into a population-based longitudinal study of aging, Canadian Journal on Aging/La Revue Canadienne du Vieillissement, 22, 185, 10.1017/S0714980800004505 · doi:10.1017/s0714980800004505
- Giovannetti, 2008, Coffee with jelly or unbuttered toast: Commissions and omissions are dissociable aspects of everyday action impairment in Alzheimer’s disease, Neuropsychology, 22, 235, 10.1037/0894-4105.22.2.235 · DOI
- Gold, 2012, An examination of instrumental activities of daily living assessment in older adults and mild cognitive impairment, Journal of Clinical and Experimental Neuropsychology, 34, 11, 10.1080/13803395.2011.614598 · DOI · Paper page
- Goverover, 2004, Everyday problem solving among four groups of individuals with cognitive impairments: Examination of the discriminant validity of the observed tasks of daily living—Revised, OTJR: Occupation, Participation and Health, 24, 103, 10.1177/153944920402400304 · DOI
- Kline, 2000, Handbook of psychological testing
- Kosowicz, 2017, Improving multitasking assessment in healthy older adults using a prop-based version of the breakfast task, Applied Neuropsychology Adult, 24, 252, 10.1080/23279095.2015.1136310 · DOI
- Lamberts, 2010, A real-life, ecologically valid test of executive functioning: The executive secretarial task, Journal of Clinical and Experimental Neuropsychology, 32, 56, 10.1080/13803390902806550 · DOI · Paper page
- Levine, 2000, Assessment of strategic self-regulation in traumatic brain injury: Its relationship to injury severity and psychosocial outcome, Neuropsychology, 14, 491, 10.1037/0894-4105.14.4.491 · doi:10.1037/0894-4105.14.4.491 · Paper page
- Loeb, 1996, Independent Living Scales (ILS) stimulus booklet
- 2002, Rehabilitation of executive function: facilitation of effective goal management on complex tasks using periodic auditory alerts, Neuropsychologia, 40, 271, 10.1016/S0028-3932(01)00094-X · DOI · Paper page
- Marson, 2006, Geriatric neuropsychology: Assessment and intervention, 158
- Martyr, 2012, Executive function and activities of daily living in Alzheimer's disease: A correlational meta-analysis, Dementia and Geriatric Cognitive Disorders, 33, 189, 10.1159/000338233 · DOI · Paper page
- McAlister, 2013, Naturalistic assessment of executive function and everyday multitasking in healthy older adults, Aging, Neuropsychology and Cognition, 20, 735, 10.1080/13825585.2013.781990 · DOI
- McAlister, 2016, Examination of variables that may affect the relationship between cognition and functional status in individuals with mild cognitive impairment: A meta-analysis, Archives of Clinical Neuropsychology, 31, 123
- Meijer, 2018, Movement timing and cognitive control: Adult-age differences in multi-tasking, Psychological Research, 82, 203, 10.1007/s00426-017-0876-4 · DOI
- Parsons, 2017, Validity of a newly developed measure of memory: Feasibility study of the virtual environment grocery store, Journal of Alzheimer's Disease, 59, 1227, 10.3233/JAD-170295 · DOI
- Podsiadlo, 1991, The timed "up & go": A test of basic functional mobility for frail elderly persons, Journal of the American Geriatrics Society, 39, 142, 10.1111/j.1532-5415.1991.tb01616.x · DOI · Paper page
- Rand, 2009, Validation of the virtual MET as an assessment tool for executive functions, Neuropsychological Rehabilitation, 19, 583, 10.1080/09602010802469074 · DOI · Paper page
- Reitan, 1985, The Halstead-Reitan Neuropsychological Test Battery: Therapy and clinical interpretation
- Rey, 1964, Psychological appraisal of children with cerebral deficits
- Robertson, 2017, Naturalistic tasks performed in realistic environments: A review with implications for neuropsychological assessment, The Clinical Neuropsychologist, 31, 16, 10.1080/13854046.2016.1208847 · DOI
- Royall, 2007, The cognitive correlates of functional status: A review from the committee on research of the American neuropsychiatric association, The Journal of Neuropsychiatry and Clinical Neurosciences, 19, 249, 10.1176/jnp.2007.19.3.249 · DOI · Paper page
- Sanders, 2014, Assessment of planning abilities in individuals with mild cognitive impairment using and open-ended problem-solving task, Journal of Clinical and Experimental Neuropsychology, 36, 1084, 10.1080/13803395.2014.983462 · DOI
- Sanders, 2012, Identifying the nature of impairment in planning ability with normal aging, Journal of Clinical and Experimental Neuropsychology, 34, 724, 10.1080/13803395.2012.670210 · DOI
- Sbordone, 1996, Ecological Validity of Neuropsychological Testing, 15
- Seidel, 2013, Neuroimaging correlates of everyday action in dementia, Journal of Clinical and Experimental Neuropsychology, 35, 993, 10.1080/13803395.2013.844773 · DOI
- 2019, Night Out Task and tablet application
- Schmitter-Edgecombe, 2012, Naturalistic assessment of everyday functioning in individuals with mild cognitive impairment: The day out task, Neuropsychology, 26, 631, 10.1037/a0029352 · DOI · Paper page
- Schmitter-Edgecombe, 2011, Cognitive correlates of functional performance in older adults: Comparison of self-report, direct observation, and performance-based measures, Journal of the International Neuropsychological Society, 17, 853, 10.1017/S1355617711000865 · DOI · Paper page
- 2014, Assessment of functional change and cognitive correlates in the progression from healthy cognitive aging to dementia, Neuropsychology, 28, 881, 10.1037/neu0000109 · DOI
- Schmitter-Edgecombe, 2014, Development and psychometric properties of the independent activities of daily living: Compensation scale, Archives of Clinical Neuropsychology, 29, 776, 10.1093/arclin/acu053 · DOI
- Schmitter-Edgecombe, 2009, Characterizing multiple memory deficits and their relation to everyday functioning in individuals with mild cognitive impairment, Neuropsychology, 23, 168, 10.1037/a0014186 · DOI · Paper page
- Schwartz, 2003, Naturalistic action test
- Turner, 2020, Goal-oriented attention self-regulation (GOALS) training in older adults, Aging & Mental Health, 24, 464, 10.1080/13607863.2018.1534080 · DOI
- Weakley, 2019, Compensatory strategy use improves real-world functional performance in community dwelling older adults, Neuropsychology, 33, 1121, 10.1037/neu0000591 · DOI
- Wechsler, 2008, WAIS-IV technical and interpretive manual
Cited by
Suchy, Y., DesRuisseaux, L. A., Gereau Mora, M., Brothers, S. L., & Niermeyer, M. A. (2024) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617723000735 · Cited by 32
Romero-Ayuso, D., del Pino-González, A., Torres-Jiménez, A., Juan-González, J., Celdrán, F. J., Franchella, M. C., Ortega-López, N., Triviño-Juárez, J. M., Garach-Gómez, A., Arrabal-Fernández, L., et al. (2024) · Children · doi:10.3390/children11080986 · Cited by 20
Ecological validity reconsidered: the Night Out Task versus the D-KEFS
Suchy, Y., Lipio Brothers, S., DesRuisseaux, L. A., Gereau, M. M., Davis, J. R., Chilton, R. L. C., & Schmitter-Edgecombe, M. (2022) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2022.2142527 · Cited by 13
Self-citation
Metadata
- Authors
- Schmitter-Edgecombe, M., Cunningham, R., McAlister, C., Arrotta, K., & Weakley, A.
- Affiliations
- Department of Psychology, Washington State University, Pullman, WA, USA
- Mayo Clinic School of Medicine, Mayo Clinic Health System, La Crosse, WI, USA
- Neurological Institute, Cleveland Clinic, Cleveland, OH, USA
- Neurology Department, School of Medicine, University of California, Davis, Sacramento, CA, USA
- Venue
- Archives of Clinical Neuropsychology
- Publisher
- Oxford University Press (OUP)
- Published
- 2020-10-22
- Volume
- 36
- Issue
- 4
- Pages
- 537-553
- ISSN
- 1873-5843
- Cited by
- 24
- References
- 11
- Language
- en
- Licence
- academic.oup.com/journals/pages/open_access/funder_policies/chorus/standard_publication_model (vor)
- Funders
- National Institute of Biomedical Imaging and Bioengineering (R01 EB009675)
- U.S. Department of Education: Graduate Assistance in Areas of National Need (P200A150115)
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The night out task and scoring application: an ill-structured, open-ended clinic-based test representing cognitive capacities used in everyday situations. https://app.cochat.ai/papers/10.1093/arclin/acaa080
- The paper
- Schmitter-Edgecombe, M., Cunningham, R., McAlister, C., Arrotta, K., & Weakley, A. (2021). The night out task and scoring application: an ill-structured, open-ended clinic-based test representing cognitive capacities used in everyday situations. Archives of Clinical Neuropsychology. https://doi.org/10.1093/arclin/acaa080
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.