Ecological validity in neurocognitive assessment: Systematized review, content analysis, and proposal of an instrument
Open accessAbstract
Objectives The main objectives of this study are to identify the dimensions of Ecological Validity (EV) within the definitions of this concept, understand how they are operationalized in neurocognitive tests, and propose a checklist for EV attributes in neurocognitive tests. Method A systematized review was combined with content analysis of the selected papers, using the inductive method. We analyzed 82 studies on the EV of neurocognitive tests, 19 literature reviews and 63 empirical studies. Based on this review, we identified the relevant criteria for evaluating EV. Results EV is a multidimensional concept with two main dimensions: representativeness and generalization. Representativeness involves the subdimensions simplicity-complexity and artificial-natural and several criteria organized on a continuum from low EV to high EV. Generalization is dependent on representativeness and is influenced by different cognitive and non-cognitive factors. We propose six stages for operationalizing EV, from defining the objectives of the neurocognitive assessment to the methodology for scoring and interpreting the results. Conclusion This systematized review helps to operationalize the concept of EV by providing a tool for evaluating and improving EV while developing new tests. Further studies with a longitudinal design can compare the predictive value of tests with higher versus lower EV-checklist scores.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Executive functioning tests with ecological validity are often associated with instrumental activities of daily living (I-ADL).
“Executive functioning tests with ecological validity are often associated with instrumental activities of daily living (I-ADL) [7,8] and reflect the functional and predictive relationship between neurocognitive assessment performance and real-world behavior [9].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
Few studies have systematically explored the ecological validity of different assessment tools.
“Despite the importance of neuropsychological assessments for predicting functional outcomes across personal, educational/occupational, and community contexts, few studies have systematically explored the ecological validity of different assessment tools or investigated the relationship between cognition and activities of daily living (ADL) [7].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
VR provides more precise information than self-reported measures that lack contextual specificity.
“The use of these enhances ecological validity by situating responses within specific contexts, involving meaningful tasks that replicate real-life activities and providing more precise information than self-reported measures that lack contextual specificity [6,7].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
VR enhances ecological validity by situating responses within specific contexts.
“The use of these enhances ecological validity by situating responses within specific contexts, involving meaningful tasks that replicate real-life activities and providing more precise information than self-reported measures that lack contextual specificity [6,7].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
Few studies have investigated the relationship between cognition and activities of daily living (ADL).
“Despite the importance of neuropsychological assessments for predicting functional outcomes across personal, educational/occupational, and community contexts, few studies have systematically explored the ecological validity of different assessment tools or investigated the relationship between cognition and activities of daily living (ADL) [7].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
VR involves meaningful tasks that replicate real-life activities.
“The use of these enhances ecological validity by situating responses within specific contexts, involving meaningful tasks that replicate real-life activities and providing more precise information than self-reported measures that lack contextual specificity [6,7].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
VR tools have the potential to create immersive environments where individuals perform tasks that reflect everyday activities, providing more accurate data on their functional cognition.
“VR tools have the potential to create immersive environments where individuals perform tasks that reflect everyday activities, providing more accurate data on their functional cognition [7,36,37].”
From Enhancing Ecological Validity: Virtual Reality Assessment of Executive Functioning in Children and Adolescents with ADHD (2024) · cites this paper as [7]
Cites
Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group (2009) · PLoS Medicine · doi:10.1371/journal.pmed.1000097 · Cited by 64,042 · see citing works
A typology of reviews: an analysis of 14 review types and associated methodologies
Grant, M. J., & Booth, A. (2009) · Health Information & Libraries Journal · doi:10.1111/j.1471-1842.2009.00848.x · Cited by 10,817 · see citing works
The standards for educational and psychological testing.
Eignor, D. R. (2013) · American Psychological Association eBooks · doi:10.1037/14047-013 · Cited by 6,975 · see citing works
Updating guidance for reporting systematic reviews: development of the PRISMA 2020 statement
Page, M. J., McKenzie, J. E., Bossuyt, P. M., Boutron, I., Hoffmann, T. C., Mulrow, C. D., Shamseer, L., Tetzlaff, J. M., & Moher, D. (2021) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2021.02.003 · Cited by 3,490 · see citing works
Adapting Educational and Psychological Tests for Cross-Cultural Assessment
(2004) · Psychology Press eBooks · doi:10.4324/9781410611758 · Cited by 1,633 · see citing works
Representative design and probabilistic theory in a functional psychology.
Brunswik, E. (1955) · Psychological Review · doi:10.1037/h0047470 · Cited by 1,550 · see citing works
Behavioural Assessment of the Dysexecutive Syndrome
(2004) · Methodology Of Frontal And Executive Function · doi:10.4324/9780203344187-15 · Cited by 1,149 · see citing works
Chaytor, N., & Schmitter-Edgecombe, M. (2003) · Neuropsychology Review · doi:10.1023/b:nerv.0000009483.91468.fb · Cited by 1,055 · see citing works
Archives of Clinical Neuropsychology
Reynolds, C. R. (2014) · Encyclopedia of Special Education · doi:10.1002/9781118660584.ese0182 · Cited by 1,054 · see citing works
The ecological validity of tests of executive function
BURGESS, P. W., ALDERMAN, N., EVANS, J., EMSLIE, H., & WILSON, B. A. (1998) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617798466037 · Cited by 932 · see citing works
Heilbronner, R. L., Sweet, J. J., Morgan, J. E., Larrabee, G. J., Millis, S. R., & Participants1, C. (2009) · The Clinical Neuropsychologist · doi:10.1080/13854040903155063 · Cited by 869 · see citing works
Parsons, T. D. (2015) · Frontiers in Human Neuroscience · doi:10.3389/fnhum.2015.00660 · Cited by 837 · see citing works
Issues, Designs, and Technical Guidelines for Adapting Tests Into Multiple Languages and Cultures
Hambleton, R., Merenda, P. F., & Spielberger, C. (2004) · Psychology Press eBooks · doi:10.4324/9781410611758-6 · Cited by 837 · see citing works
BURGESS, P. W., ALDERMAN, N., FORBES, C., COSTELLO, A., M-A.COATES, L., DAWSON, D. R., ANDERSON, N. D., GILBERT, S. J., DUMONTHEIL, I., & CHANNON, S. (2006) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617706060310 · Cited by 663 · see citing works
Solanto, M. V., Abikoff, H., Sonuga-Barke, E., Schachar, R., Logan, G. D., Wigal, T., Hechtman, L., Hinshaw, S., & Turkel, E. (2001) · Journal of Abnormal Child Psychology · doi:10.1023/a:1010329714819 · Cited by 639 · see citing works
What Is Ecological Validity? A Dimensional Analysis
Schmuckler, M. A. (2001) · Infancy · doi:10.1207/s15327078in0204_02 · Cited by 561 · see citing works
Improving the ecological validity of executive functioning assessment
CHAYTOR, N., SCHMITTEREDGECOMBE, M., & BURR, R. (2006) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2005.12.002 · Cited by 462 · see citing works
The ecological validity of laboratory and analogue assessment methods of ADHD symptoms
Barkley, R. A. (1991) · Journal of Abnormal Child Psychology · doi:10.1007/bf00909976 · Cited by 419 · cited 8× · see citing works
“…and its dimensions (Schmuckler, 2001); (b) criticisms regarding the concept of EV (Holleman et al., 2020); (c) EV of neuropsychological measures (Barkley, 1991; Chaytor & Schmitter-Edgecombe, 2003; Olson et al., 2013; Poletti, 2010; Romero-Ayuso et al., 2019; Silver, 2000; Spooner & Pachana,…”
InfluentialThe Role of Representative Design in an Ecological Approach to Cognition.
Dhami, M. K., Hertwig, R., & Hoffrage, U. (2004) · Psychological Bulletin · doi:10.1037/0033-2909.130.6.959 · Cited by 409 · see citing works
Ecological Validity of Neuropsychological Testing
Long, C. J., & Sbordone, R. J. (1996) · Cited by 404 · see citing works
Multisensory integration, perception and ecological validity
de Gelder, B., & Bertelson, P. (2003) · Trends in Cognitive Sciences · doi:10.1016/j.tics.2003.08.014 · Cited by 366 · see citing works
The ‘Real-World Approach’ and Its Problems: A Critique of the Term Ecological Validity
Holleman, G. A., Hooge, I. T. C., Kemner, C., & Hessels, R. S. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00721 · Cited by 345 · see citing works
Goldberg, T. E., Harvey, P. D., Wesnes, K. A., Snyder, P. J., & Schneider, L. S. (2015) · Alzheimer's & Dementia: Diagnosis, Assessment & Disease Monitoring · doi:10.1016/j.dadm.2014.11.003 · Cited by 343 · see citing works
Ecological validity of a simplified version of the multiple errands shopping test
ALDERMAN, N., BURGESS, P. W., KNIGHT, C., & HENMAN, C. (2003) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617703910046 · Cited by 337 · see citing works
SPOONER, D., & PACHANA, N. (2006) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2006.04.004 · Cited by 323 · see citing works
Norris, G., & Tate, R. L. (2000) · Neuropsychological Rehabilitation · doi:10.1080/096020100389282 · Cited by 249 · see citing works
International Guidelines on Computer-Based and Internet-Delivered Testing
The International Test Commission (2006) · International Journal of Testing · doi:10.1207/s15327574ijt0602_4 · Cited by 200 · see citing works
Farias, S. T., Harrell, E. R., Neumann, C. S., & Houtz, A. W. (2003) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/18.6.655 · Cited by 195 · see citing works
Zhang, Y., Xiong, Y., Yu, Q., Shen, S., Chen, L., & Lei, X. (2021) · BMC Geriatrics · doi:10.1186/s12877-020-01986-x · Cited by 190 · see citing works
Behavioural assessment of the dysexecutive syndrome
Wilson, B., Evans, J., Alderman, N., Burgess, P. W., & Emslie, H. C. (1997) · UCL Discovery (University College London) · Cited by 190 · see citing works
Ecological validity: Some critical issues for the neuropsychologist.
Sbordone, R. J. (1996) · Cited by 181 · see citing works
Failures of sustained attention in life, lab, and brain: Ecological validity of the SART
Smilek, D., Carriere, J. S. A., & Cheyne, J. A. (2010) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2010.05.002 · Cited by 179 · see citing works
Wilson, B. A. (1993) · Applied and Preventive Psychology · doi:10.1016/s0962-1849(05)80091-5 · Cited by 148 · see citing works
Ecological Validity of Neuropsychological Assessment in Childhood Traumatic Brain Injury
Silver, C. H. (2000) · Journal of Head Trauma Rehabilitation · doi:10.1097/00001199-200008000-00002 · Cited by 135 · see citing works
Parsons, T. D., Carlew, A. R., Magtoto, J., & Stonecipher, K. (2015) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2015.1109524 · Cited by 134 · see citing works
Suitability of the Miller Function and Participation Scales (M–FUN) for Use With Israeli Children
Rihtman, T., & Parush, S. (2013) · American Journal of Occupational Therapy · doi:10.5014/ajot.2014.008573 · Cited by 122 · see citing works
DAWSON, D. R., Anderson, N. D., Burgess, P. W., Cooper, E., Krpan, K. M., & Stuss, D. T. (2009) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2009.07.012 · Cited by 119 · see citing works
The ecological validity of executive tests in a severely brain injured sample
WOOD, R., & LIOSSI, C. (2006) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2005.06.014 · Cited by 118 · see citing works
The Ecological Validity of Clinical Tests of Memory and Attention in Multiple Sclerosis
Higginson, C. I., Arnett, P. A., & Voss, W. D. (2000) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/15.3.185 · Cited by 116 · see citing works
Kourtesis, P., Collina, S., Doumas, L. A. A., & MacPherson, S. E. (2020) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617720000764 · Cited by 115 · see citing works
Chevignard, M., Soo, C. A., Galvin, J., Catroppa, C., & Eren, S. (2012) · Brain Injury · doi:10.3109/02699052.2012.666366 · Cited by 105 · see citing works
Ecological validity of measures of executive functioning
Odhuba, R., van den Broek, M. D., & Johns, L. (2005) · British Journal of Clinical Psychology · doi:10.1348/014466505x29431 · Cited by 104 · see citing works
Assessment of cognitive instrumental activities of daily living: A systematic review
Romero-Ayuso, D., Castillero-Perea, Á., González, P., Navarro, E., Molina-Massó, J. P., Funes, M. J., Ariza-Vega, P., Toledano-González, A., & Triviño-Juárez, J. M. (2021) · Disability and Rehabilitation · doi:10.1080/09638288.2019.1665720 · Cited by 95 · see citing works
Helmstaedter, C., Hauff, M., & Elger, C. E. (1998) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.20.3.365.824 · Cited by 94 · see citing works
Renison, B., Ponsford, J. L., Testa, R., Richardson, B. L., & Brownfield, K. (2012) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617711001883 · Cited by 93 · see citing works
Kibby, M. Y., Schmitter‐Edgecombe, M., & Long, C. J. (1998) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/13.6.523 · Cited by 90 · see citing works
Evers, A., Sijtsma, K., Lucassen, W., & Meijer, R. R. (2010) · International Journal of Testing · doi:10.1080/15305058.2010.518325 · Cited by 88 · see citing works
der Elst, W. V., van Boxtel, M. P. J., van Breukelen, G., & Jolles, J. (2008) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2008.09.002 · Cited by 88 · see citing works
EFPA REVIEW MODEL FOR THE DESCRIPTION AND EVALUATION OF PSYCHOLOGICAL AND EDUCATIONAL TESTS
Evers, A., & Sjöberg, A. (2013) · Cited by 85 · see citing works
Mitchell, M. B., & Miller, L. S. (2008) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803390701679893 · Cited by 85 · see citing works
Parsons, T. D. (2011) · Studies in computational intelligence · doi:10.1007/978-3-642-17824-5_13 · Cited by 83 · see citing works
The Working Brain: An Introduction to Neuropsychology
Gonnella, C. (1976) · Physical Therapy · doi:10.1093/ptj/56.1.118 · Cited by 82 · see citing works
Ecological validity of virtual environments to assess human navigation ability
van der Ham, I. J. M., Faber, A., Venselaar, M., van Kreveld, M. J., & Löffler, M. (2015) · Frontiers in Psychology · doi:10.3389/fpsyg.2015.00637 · Cited by 81 · see citing works
Ecological Validity of Neuropsychological and Personality Measures of Executive Functions
Ready, R. E., Stierman, L., & Paulsen, J. S. (2001) · The Clinical Neuropsychologist · doi:10.1076/clin.15.3.314.10269 · Cited by 80 · see citing works
Cognitive assessment for clinicians
Kipps, C. (2005) · Journal of Neurology Neurosurgery & Psychiatry · doi:10.1136/jnnp.2004.059758 · Cited by 78 · see citing works
Campbell, Z., Zakzanis, K. K., Jovanovski, D., Joordens, S., Mraz, R., & Graham, S. J. (2009) · Applied Neuropsychology · doi:10.1080/09084280903297891 · Cited by 78 · see citing works
Krohn, S., Tromp, J., Quinque, E. M., Belger, J., Klotzsche, F., Rekers, S., Chojecki, P., de Mooij, J., Akbal, M., McCall, C., et al. (2020) · Journal of Medical Internet Research · doi:10.2196/16724 · Cited by 72 · see citing works
The Concept of Ecological Validity: What Are Its Limitations and Is It Bad to Be Invalid?
Lewkowicz, D. J. (2001) · Infancy · doi:10.1207/s15327078in0204_03 · Cited by 68 · see citing works
Verdejo‐García, A., Bechara, A., Recknor, E. C., & Pérez-Garcı́a, M. (2006) · Psychologica Belgica · doi:10.5334/pb-46-1-2-55 · Cited by 66 · see citing works
Executive function measures for children: A scoping review of ecological validity.
Wallisch, A., Little, L. M., Dean, E., & Dunn, W. (2018) · OTJR: Occupational Therapy Journal of Research · doi:10.1177/1539449217727118 · Cited by 65 · see citing works
Towards ecological validity of research in cognitive ergonomics
Hoc, J. (2001) · Theoretical Issues in Ergonomics Science · doi:10.1080/14639220110104970 · Cited by 65 · see citing works
Hogerzeil, H. (2010) · doi:10.5040/9798400688379.ch-007 · Cited by 65 · see citing works
Chaytor, N. S., Temkin, N. R., Machamer, J., & DIKMEN, S. S. (2007) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617707070592 · Cited by 65 · see citing works
Block Design as a Measure of Everyday Spatial Ability: A Study of Ecological Validity
Groth-Marnat, G., & Teal, M. (2000) · Perceptual and Motor Skills · doi:10.2466/pms.2000.90.2.522 · Cited by 64 · cited 1× · see citing works
“…et al., 2013; Davies et al., 2011; Drozdick & Cullum, 2011; Farias et al., 2003; Farley et al., 2011; Gioia, 2009; Gioia & Brekke, 2009; Groth-Marnat & Teal, 2000; Groth-Marnat & Baker, 2003; Higginson et al., 2000; Holleman et al., 2020; Kieffaber et al., 2007; Maeir et al., 2011;…”
Díaz-Orueta, U., Blanco‐Campal, A., Lamar, M., Libon, D. J., & Burke, T. (2020) · Frontiers in Psychology · doi:10.3389/fpsyg.2020.00361 · Cited by 64 · see citing works
Digit Span as a Measure of Everyday Attention: A Study of Ecological Validity
Groth-Marnat, G., & Baker, S. (2003) · Perceptual and Motor Skills · doi:10.2466/pms.2003.97.3f.1209 · Cited by 62 · see citing works
Routinization preferences, anxiety, and depression in an elderly French sample
Bouisson, J. (2002) · Journal of Aging Studies · doi:10.1016/s0890-4065(02)00051-8 · Cited by 61 · see citing works
The Working Brain: An Introduction to Neuropsychology, by
Luria, A. R. (1976) · Cited by 61 · see citing works
Impairmentversusdeficiency in neuropsychological assessment: Implications for ecological validity
Silverberg, N. D., & Millis, S. R. (2009) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617708090139 · Cited by 58 · see citing works
Owen, W. J., Borowsky, R., & Sarty, G. E. (2004) · Brain and Language · doi:10.1016/s0093-934x(03)00418-8 · Cited by 57 · see citing works
Chaytor, N. S., Barbosa‐Leiker, C., Germine, L. T., Fonseca, L. M., McPherson, S., & Tuttle, K. R. (2020) · The Clinical Neuropsychologist · doi:10.1080/13854046.2020.1811893 · Cited by 57 · see citing works
The Revised Dutch Rating System for Test Quality
Evers, A. (2001) · International Journal of Testing · doi:10.1207/s15327574ijt0102_4 · Cited by 56 · see citing works
Aubin, G., Béliveau, M.-F., & Klinger, É. (2015) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2015.1074083 · Cited by 53 · see citing works
Age and Planning Tasks: The Influence of Ecological Validity
Phillips, L. H., Kliegel, M., & Martin, M. (2005) · The International Journal of Aging and Human Development · doi:10.2190/em1w-hayc-tmlm-ww8x · Cited by 51 · see citing works
Introduction to the Neuropsychology of Everyday Life
Tupper, D. E., & Cicerone, K. D. (1990) · Foundations of neuropsychology · doi:10.1007/978-1-4613-1503-2_1 · Cited by 49 · see citing works
Urbano, G. C., Caracuel, A., Vilar‐López, R., Valls-Serrano, C., Bateman, A., & Verdejo‐García, A. (2013) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2013.776011 · Cited by 46 · see citing works
Ecological validity of the Dysexecutive Questionnaire: Results from the PariS-TBI study
Azouvi, P., Vallat‐Azouvi, C., Millox, V., Darnoux, E., Ghout, I., Azerad, S., Ruet, A., Bayen, É., Pradat-Diehl, P., Aegerter, P., et al. (2014) · Neuropsychological Rehabilitation · doi:10.1080/09602011.2014.990907 · Cited by 42 · see citing works
Vingerhoets, G., Vermeule, E., & Santens, P. (2004) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2004.09.003 · Cited by 40 · see citing works
Temple, R., Zgaljardic, D. J., Abreu, B. C., Seale, G. S., Ostir, G. V., & Ottenbacher, K. J. (2008) · Brain Injury · doi:10.1080/02699050802590361 · Cited by 39 · see citing works
Ecological validity of neuropsychological and other predictors following head injury
Bowman, M. L. (1996) · The Clinical Neuropsychologist · doi:10.1080/13854049608406699 · Cited by 37 · see citing works
Ecological validity of executive function tests in moderate traumatic brain injury in Ghana
Adjorlolo, S. (2016) · The Clinical Neuropsychologist · doi:10.1080/13854046.2016.1172667 · Cited by 37 · see citing works
Davies, S. R., Field, A. R. J., Andersen, T., & Pestell, C. (2011) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2011.574608 · Cited by 35 · see citing works
Ecological Validity and Neuroanatomical Correlates of the NIH EXAMINER Executive Composite Score
Possin, K. L., LaMarre, A. K., Wood, K., Mungas, D., & Kramer, J. H. (2013) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617713000611 · Cited by 34 · see citing works
Ecological Validity of the Conners' Continuous Performance Test II in a School-Based Sample
Weis, R., & Totten, S. J. (2004) · Journal of Psychoeducational Assessment · doi:10.1177/073428290402200104 · Cited by 32 · see citing works
Yantz, C. L., Johnson-Greene, D. E., Higginson, C. I., & Emmerson, L. C. (2010) · Neuropsychological Rehabilitation · doi:10.1080/09602011003765690 · Cited by 32 · see citing works
Ecological Validity of Performance Validity Testing
Lippa, S. M., Pastorek, N. J., Romesser, J. M., Linck, J. F., Sim, A. H., Wisdom, N. M., & Miller, B. (2014) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acu002 · Cited by 31 · see citing works
Ziemnik, R. E., & Suchy, Y. (2019) · Psychological Assessment · doi:10.1037/pas0000751 · Cited by 31 · see citing works
Maeir, A., Krauss, S., & Katz, N. (2011) · OTJR Occupational Therapy Journal of Research · doi:10.3928/15394492-20101108-07 · Cited by 29 · see citing works
History and Philosophy of Psychology
Valentine, E. R. (2012) · International Studies in the Philosophy of Science · doi:10.1080/02698595.2012.748501 · Cited by 29 · see citing works
Expanding the Ecological Validity of WAIS-IV and WMS-IV With the Texas Functional Living Scale
Drozdick, L. W., & Cullum, C. M. (2010) · Assessment · doi:10.1177/1073191110382843 · Cited by 28 · see citing works
Kvavilashvili, L., & Ellis, J. R. (2004) · University of Hertfordshire Research Archive (University of Hertfordshire) · Cited by 28 · see citing works
Weber, E., Goverover, Y., & DeLuca, J. (2019) · Multiple Sclerosis Journal · doi:10.1177/1352458519860318 · Cited by 27 · see citing works
Zgaljardic, D. J., Yancy, S., Temple, R., Watford, M., & Miller, R. (2011) · Rehabilitation Psychology · doi:10.1037/a0025466 · Cited by 26 · see citing works
Bromley, E., Mikesell, L., Mates, A., Smith, M. E., & Brekke, J. S. (2011) · Schizophrenia Bulletin · doi:10.1093/schbul/sbr002 · Cited by 26 · see citing works
Horan, B., Heckenberg, R. A., Maruff, P. T., & Wright, B. J. (2020) · BMC Psychology · doi:10.1186/s40359-020-00429-x · Cited by 26 · see citing works
Ecological Validity of Pediatric Neuropsychological Measures: Current State and Future Directions
Olson, K. J., Jacobson, K. K., & Oot, P. V. (2012) · Applied Neuropsychology Child · doi:10.1080/21622965.2012.686330 · Cited by 25 · see citing works
de Paula, J. J., Costa, M. V., Bocardi, M. B., Cortezzi, M., de Moraes, E. N., & Malloy‐Diniz, L. F. (2013) · International Psychogeriatrics · doi:10.1017/s1041610213001282 · Cited by 24 · see citing works
The Ecological Validity of Pediatric Neuropsychological Tests of Attention
Price, K. J., Joschko, M., & Kerns, K. (2003) · The Clinical Neuropsychologist · doi:10.1076/clin.17.2.170.16506 · Cited by 23 · cited 5× · see citing works
“…Farias et al., 2003; Farley et al., 2011; Kibby et al., 1998; Maeir et al., 2011; Olson et al., 2013; Pezzuti et al., 2013; Phillips et al., 2006; Price et al., 2003; Ready et al., 2001; Renison et al., 2012; van der Elst et al., 2008; Wilson, 1993; Wood and Liossi, 2006; Zgaljardic et al.,…”
InfluentialDubreuil, P., Adam, S., Bier, N., & Gagnon, L. (2007) · Archives of Clinical Neuropsychology · doi:10.1016/j.acn.2007.08.002 · Cited by 20 · see citing works
Ecological validity of the five digit test and the oral trails test
Paiva, G. C. D. C., Fialho, M. B., Costa, D. D. S., & de Paula, J. J. (2015) · Arquivos de Neuro-Psiquiatria · doi:10.1590/0004-282x20150184 · Cited by 18 · see citing works
Contributions of the EFPA Standing Committee on Tests and Testing to Standards and Good Practice
Bartram, D. (2011) · European Psychologist · doi:10.1027/1016-9040/a000093 · Cited by 17 · see citing works
Gioia, D., & Brekke, J. S. (2009) · Psychiatry · doi:10.1521/psyc.2009.72.1.94 · Cited by 16 · see citing works
Zhou, S., Jiang, W., Wang, H., Wei Min, N. I., & Yu, Q. (2020) · Cancer Medicine · doi:10.1002/cam4.3244 · Cited by 16 · see citing works
Vordenberg, J. A., Barrett, J. J., Doninger, N. A., Contardo, C. P., & Ozoude, K. A. (2014) · The Clinical Neuropsychologist · doi:10.1080/13854046.2014.881555 · Cited by 15 · see citing works
Benge, J. F., Artz, J. D., & Kiselica, A. M. (2020) · The Clinical Neuropsychologist · doi:10.1080/13854046.2020.1837246 · Cited by 15 · see citing works
Farley, K. L., Higginson, C. I., Sherman, M. F., & MacDougall, E. E. (2011) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acr069 · Cited by 14 · see citing works
Faith, L. A., & Rempfer, M. V. (2018) · Psychiatry Research · doi:10.1016/j.psychres.2018.04.060 · Cited by 14 · see citing works
Chen, D., Chen, J., Yang, H., Liang, X., Xie, Y., Li, S., Ding, L., & Li, Q. (2019) · Minerva Anestesiologica · doi:10.23736/s0375-9393.19.13462-1 · Cited by 13 · see citing works
The ecological validity of everyday cognition in hospitalized patients with serious mental illness
Thornton, A. E., Kristinsson, H. B., DeFreitas, V. G., & Thornton, W. L. (2009) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803390903002209 · Cited by 13 · see citing works
Tang, S.-F., Chen, I., Chiang, H.-Y. A., Wu, C., Hsueh, I., Yu, W.-H., & Hsieh, C. (2017) · Psychiatry Research · doi:10.1016/j.psychres.2017.11.066 · Cited by 11 · see citing works
Rumpf, U., Menze, I., Müller, N. G., & Schmicker, M. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.01046 · Cited by 11 · see citing works
Ecological Measures of Cognitive Functioning: A Validation Study
Goodman, C. R., & Zarit, S. H. (1995) · International Psychogeriatrics · doi:10.1017/s1041610295001839 · Cited by 11 · see citing works
Ecological Validity of Neuropsychological Assessment and Perceived Employability
Wen, J. H., Boone, K. B., & Kim, K. S. (2006) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803390500409609 · Cited by 9 · see citing works
Sandvoll, A. M., Grov, E. K., & Simonsen, M. (2020) · PeerJ · doi:10.7717/peerj.10202 · Cited by 9 · see citing works
Understanding the Ecological Validity of Neuropsychological Testing Using an Ethnographic Approach
Gioia, D. (2009) · Qualitative Health Research · doi:10.1177/1049732309348500 · Cited by 8 · see citing works
Krishna, M., Beulah, E., Jones, S., Sundarachari, R., Saroja, A. O., Kumaran, K., Karat, S. C., Copeland, J. R. M., Prince, M. J., & Fall, C. H. D. (2015) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.4404 · Cited by 8 · see citing works
Kenney, L. E., Margolis, S. A., Davis, J. D., & Tremont, G. N. (2019) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acz033 · Cited by 7 · see citing works
Pezzuti, L., Mastrantonio, E., & Orsini, A. (2013) · Aging Neuropsychology and Cognition · doi:10.1080/13825585.2012.761668 · Cited by 6 · see citing works
Modeling the Ecological Validity of Neurocognitive Assessment in Adults with Acquired Brain Injury
Kieffaber, P. D., Marcoulides, G. A., White, M. H., & Harrington, D. E. (2007) · Journal of Clinical Psychology in Medical Settings · doi:10.1007/s10880-007-9075-6 · Cited by 6 · see citing works
Poletti, M. (2010) · Neuropsychological Trends · doi:10.7358/neur-2010-007-pole · Cited by 5 · see citing works
Introduction to the special issue on the neuropsychology of daily life.
Woods, S. P. (2021) · Neuropsychology · doi:10.1037/neu0000716 · Cited by 5 · see citing works
Smilek, D., Carriere, J. S. A., & Cheyne, J. A. (2011) · Neuropsychologia · doi:10.1016/j.neuropsychologia.2011.01.037 · Cited by 4 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Suchy, Y., DesRuisseaux, L. A., Gereau Mora, M., Brothers, S. L., & Niermeyer, M. A. (2024) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617723000735 · Cited by 32 · see citing works
Romero-Ayuso, D., del Pino-González, A., Torres-Jiménez, A., Juan-González, J., Celdrán, F. J., Franchella, M. C., Ortega-López, N., Triviño-Juárez, J. M., Garach-Gómez, A., Arrabal-Fernández, L., et al. (2024) · Children · doi:10.3390/children11080986 · Cited by 15 · see citing works
“The study hypothesizes that children and adolescents with ADHD exhibit lower performance and higher error rates in commission, omission, and rule-breaking behaviors than their neurotypical peers [7].”
BackgroundInfluentialFinley, J. A., Robinson, A. D., VanLandingham, H. B., Ulrich, D. M., Phillips, M. S., & Soble, J. R. (2024) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617724000365 · Cited by 12 · see citing works
High-dimensional Metaverse Platforms and the Virtually Extended Self
Parsons, T. D. (2024) · Journal of Cognition · doi:10.5334/joc.327 · Cited by 7 · see citing works
Activities of Daily Living Inventory (ADLI): Proposal of a new instrument and preliminary data
Pinto, J. O., Pontes, D., Dores, A. R., Peixoto, B., & Barbosa, F. (2024) · Applied Neuropsychology Adult · doi:10.1080/23279095.2024.2367741 · Cited by 6 · see citing works
Self-citationChatzidimitriou, E., Ntritsos, G., Lagoudaki, R., Poptsi, E., Tsardoulias, E., Symeonidis, A. L., Tsolaki, M., Konstantinopoulou, E., Papadopoulou, K., Charalambous, P., et al. (2025) · Journal of Personalized Medicine · doi:10.3390/jpm15100466 · Cited by 7 · see citing works
Sensory Processing of Time and Space in Autistic Children
Coelho, F., Rando, B., Salgado, M., & Abreu, A. M. (2025) · Children · doi:10.3390/children12101366 · Cited by 4 · see citing works
Vinci, L. A., Passaro, A., & Stasolla, F. (2026) · Information · doi:10.3390/info17020186 · Cited by 3 · see citing works
Ricchetti, G., Navarro-Egido, A., Rodríguez-Bailón, M., Salazar‐Frías, D., Narganes-Pineda, C., & Funes, M. J. (2024) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/13803395.2024.2391363 · Cited by 3 · see citing works
Ecological validity of neuropsychological interventions: A systematic review
Pinto, J. O., Pontes, D., Peixoto, B., Dores, A. R., & Barbosa, F. (2024) · Applied Neuropsychology Adult · doi:10.1080/23279095.2024.2328694 · Cited by 3 · see citing works
Self-citationHenneghan, A. M., Franco‐Rocha, O. Y., Dyk, K. V., Paolillo, E. W., Haywood, D., & Moore, R. C. (2025) · Psycho-Oncology · doi:10.1002/pon.70324 · Cited by 2 · see citing works
Martin, E., Trouilloud, D. O., & Tallet, J. (2024) · Journal of Motor Learning and Development · doi:10.1123/jmld.2023-0019 · Cited by 2 · see citing works
The Virtual Supermarket Test: A New Tool for Assessing Planning in Adults From Buenos Aires
Seivane, M. S., & Brenlla, M. E. (2025) · Assessment · doi:10.1177/10731911251390381 · Cited by 1 · see citing works
Baandrup, L., Gholy, S. M., & Garcı́a-Borreguero, D. (2026) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2026.102280 · Cited by 1 · see citing works
Imparato, A., Reich, N., Riviere, G., Eliez, S., Graser, C. J., Schneider, M., & Sandini, C. (2026) · medRxiv · doi:10.64898/2026.04.27.26351677 · Cited by 1 · see citing works
Identifying conflict monitoring as a key executive component related to impaired self-awareness.Preprint
Ricchetti, G., Navarro-Egido, A., Rodríguez‐Bailón, M., Frias, D., Narganes-Pineda, C., & Funes, M. J. (2024) · doi:10.31219/osf.io/zpdwx
Cabral, J. D. D., de Oliveira, B. L. C. A., Simon, S. S., Passinho, J. S., Cappi, C., Bertola, L., Alves, C. H. L., Simões, V. M. F., & Alves, G. S. (2026) · Frontiers in Dementia · doi:10.3389/frdem.2025.1673362
Exploring scientific reasoning in medical students via Lawson's test
Estrada-Rojo, F., Hernández-Chávez, A., Muñoz-Comonfort, A., López-Toledo, G., Gomez‐Virgilio, L., Estrada-Bernal, F., & Sampieri-Cabrera, R. (2026) · Educación Médica · doi:10.1016/j.edumed.2026.101163
Neveu, E., Salles, J., Roy, I., Vera-Estay, E., Dooley, J. J., & Beauchamp, M. H. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1648238
Babb, E. C., Lee, C. D., Jasim, M., & Jung, H.-T. (2026) · CHI Conference on Human Factors in Computing Systems · doi:10.1145/3772318.3791489
Loizidou, M. A., Christoforou, C., & Constantinidou, F. (2026) · Brain Communications · doi:10.1093/braincomms/fcag149
Designing Inclusive Counselling Practices for Culturally Diverse Student Communities
Sarkar, R., & Madhu, R. (2026) · IGI Global eBooks · doi:10.4018/979-8-3373-5987-8.ch010
Weisse, M., Dardier, V., di Vettimo, S. S., Tsimba, V., Trolès, N., Douguet, A.-S., Jouan, V., & Lacroix, A. (2026) · Journal of Integrative Neuroscience · doi:10.31083/jin47025
Fundamentos de neuropsicologia
Flóres-Lázaro, J. C. (2026) · Ediciones Comunicación Científica eBooks · doi:10.52501/cc.355
The evolution of phonemic verbal fluency test: bridging tradition with contemporary insights
Morkovina, O. I., Ziberova, A., Sapunova, O., Manukyan, P. A., & Sharapkova, A. A. (2026) · Frontiers in Psychology · doi:10.3389/fpsyg.2026.1774256
Ramos‐Henderson, M., Aliaga-Gallardo, P., Aguilar-Riveros, C., Fuentes, L., Calderón, C., & Domic‐Siede, M. (2026) · Journal of Alzheimer s Disease · doi:10.1177/13872877261470712
Abdullah, N. H. I., Phon, D. N. E., Zulkifli, N. S. A., Pudjoatmodjo, B., Roedavan, R., & Prasetyanto, F. (2026) · IEEE International Conference on Innovation, Ethics & Emerging Tech in Engineering and Computing Education (IE2C) · doi:10.1109/ie2c69620.2026.11663046
Östling, A.-K., Dahl, B., Andersson, G., & Björck, C. (2026) · Applied Neuropsychology Adult · doi:10.1080/23279095.2026.2734319
- Authors
- Pinto, J. O., Dores, A. R., Peixoto, B., & Barbosa, F.
- Affiliations
- Universidade do Porto (PT)
- Cooperativa de Ensino Superior Politécnico e Universitário (PT)
- Corresponding author
- Pinto, Joana O.
- Venue
- Applied Neuropsychology: Adult
- Publisher
- Informa UK Limited
- Published
- 2023-02-08
- Volume
- 32
- Issue
- 2
- Pages
- 577-594
- ISSN
- 2327-9095 (ISSN-L)
- 2327-9109
- OpenAlex
- W4319655983
- Cited by
- 28
- References
- 122
- Citations by year
- 2026: 15 · 2025: 4 · 2024: 9
- FWCI
- 1.41
- Type
- journal article
- Language
- en
- Open access
- hybrid
- doi.org
- Licence
- Locations
- Applied Neuropsychology Adult (publishedVersion, cc-by-nc-nd)
- PubMed (publishedVersion)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion, cc-by-nc-nd)
- Portuguese National Funding Agency for Science, Research and Technology (RCAAP Project by FCT) (submittedVersion)
- Funders
- FCT—Fundação para a Ciência e (UIDB/05210/2020)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Historical and Modern Theater Studies
- Social Sciences › Psychology › Applied Psychology › Psychological Testing and Assessment
- Social Sciences › Education › Health Education and Validation
- Keywords
- ecological validity, neuropsychological assessment, neurocognitive tests, representativeness, generalization, test development
- MeSH terms
- Humans, Neuropsychological Tests, Reproducibility of Results
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
