The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Statistics to tell the truth, the whole truth, and nothing but the truth Formulae, illustrative numerical examples, and heuristic interpretation of effect size analyses for neuropsychological researchers
Open accessAbstract
If, as neuropsychologists, we think of the relationship between brain and behavior as the same as that between truth and reality, we must be equipped with statistical procedures that are coherent in terms of what we measure and what it represents. I believe that this necessary statistical procedure is effect size analysis, and without it, I believe that we fail to tell the truth, the whole truth, and nothing but the truth when describing our neuropsychological research. Accordingly, I review here the standard calculations of commonly employed effect sizes in two group designs and show how to adjust some familiar (and perhaps not so familiar) formulae using illustrative numerical examples. I also put forth an argument to adopt Cohen's measure as an expression of effect size based on its apropos to neuropsychological research. It is also argued that the interpretation of the magnitude of an effect size should depend on context, and not on pre-established heuristic benchmarks. It is noted, however, that effect sizes greater than 3.0 (OL%<5) might seem particularly appropriate when evaluating the sensitivity of neuropsychological tasks and in establishing test markers in neuropsychological disorders.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“meta-analytic effect sizes of d=0.46 to 0.55 (Willcutt et al., 2005) predict that 30%–36% of children with ADHD may demonstrate set shifting impairments based on converting effect size differences into expected population overlap proportions (Zakzanis et al., 2001).”
From Executive Functioning Heterogeneity in Pediatric ADHD (2019) · cites this paper as Zakzanis et al., 2001
In the citing paper
“which suggest that only 35%−40% of children with ADHD are likely to have working memory impairments based on converting effect sizes into proportion non-overlap (Zakzanis, 2001)”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“which correspond to impairment rates of 32% to 50% (Zakzanis, 2001)”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“The ‘proportion of non-overlap’ statistic estimates the proportion of ADHD cases that fall outside of the Non-ADHD range and thus provides an estimate of the percentage of ADHD children with impairments in each short-term/working memory component (Zakzanis, 2001); these estimates tend to align closely with values obtained from within-subject/heterogeneity methods for defining neurocognitive”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“This effect size suggests that approximately 38% of children with ADHD exhibit impaired visuospatial short-term memory (Zakzanis, 2001).”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“meta-analytic effect sizes of d=0.46 to 0.55 (Willcutt et al., 2005) predict that 30%–36% of children with ADHD may demonstrate set shifting impairments based on converting effect size differences into expected population overlap proportions (Zakzanis et al., 2001).”
From Executive Functioning Heterogeneity in Pediatric ADHD (2019) · cites this paper as Zakzanis et al., 2001
In the citing paper
“Thus, whereas the uncorrected estimates suggest that 40%–44% of children with ADHD have working memory deficits (Zakzanis, 2001), these estimates increase to 81%–84% when using criterion, recall-based tasks with prominent executive demands (Coghill et al., 2014) and a sufficient number of administered trials (Wells et al., 2018).”
From Executive Functioning Heterogeneity in Pediatric ADHD (2019) · cites this paper as Zakzanis, 2001
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS)
Kurtzke, J. F. (1983) · Neurology · doi:10.1212/wnl.33.11.1444 · Cited by 15,011
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Kaplan, E., Goodglass, H., Weıntraub, S., & Segal, O. (2001) · Medical Entomology and Zoology · Cited by 4,378
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Hall, R., Wittgenstein, L., & Anscombe, G. E. M. (1967) · The Philosophical Quarterly · doi:10.2307/2217461 · Cited by 4,043
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
Cited by
Sorted by citations
Correlational effect size benchmarks
Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015) · Journal of Applied Psychology · doi:10.1037/a0038047 · Cited by 768
Reaction time variability in ADHD: A meta-analytic review of 319 studies
Kofler, M. J., Rapport, M. D., Sarver, D. E., Raiker, J. S., Orban, S. A., Friedman, L. M., & Kolomeyer, E. G. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.06.001 · Cited by 648
Inhibitory control and psychopathology: A meta-analysis of studies using the stop signal task
LIPSZYC, J., & SCHACHAR, R. (2010) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617710000895 · Cited by 627
Maruff, P., Thomas, E., Cysique, L., Brew, B., Collie, A., Snyder, P., & Pietrzak, R. H. (2009) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acp010 · Cited by 635
Innovative technology-based interventions for autism spectrum disorders: a meta-analysis
Grynszpan, O., Weiss, P. L. (., Perez-Diaz, F., & Gal, E. (2014) · Autism · doi:10.1177/1362361313476767 · Cited by 621
A Model of the Development of Frontal Lobe Functioning: Findings From a Meta-Analysis
Romine, C. B., & Reynolds, C. R. (2005) · Applied Neuropsychology · doi:10.1207/s15324826an1204_2 · Cited by 526
Rapport, M. D., Orban, S. A., Kofler, M. J., & Friedman, L. M. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.08.005 · Cited by 517
Kasper, L. J., Alderson, R. M., & Hudec, K. L. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.07.001 · Cited by 414
Response inhibition and psychopathology: A meta-analysis of go/no-go task performance
Wright, L., Lipszyc, J., Dupuis, A., Thayapararajah, S. W., & Schachar, R. (2014) · Journal of Abnormal Psychology · doi:10.1037/a0036295 · Cited by 360
Palta, P., Schneider, A. L. C., Biessels, G. J., Touradji, P., & Hill-Briggs, F. (2014) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617713001483 · Cited by 364
Kazantzis, N., Whittington, C., Zelencich, L., Kyrios, M., Norton, P. J., & Hofmann, S. G. (2016) · Behavior Therapy · doi:10.1016/j.beth.2016.05.002 · Cited by 280
Executive Functioning Heterogeneity in Pediatric ADHD
Kofler, M. J., Irwin, L. N., Soto, E. F., Groves, N. B., Harmon, S. L., & Sarver, D. E. (2019) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-018-0438-2 · Cited by 288
The C factor: cognitive dysfunction as a transdiagnostic dimension in psychopathology
Abramovitch, A., Short, T., & Schweiger, A. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102007 · Cited by 269
Neuropsychological consequences of chronic opioid use: a quantitative review and meta-analysis.
Baldacchino, A., Balfour, D. J. K., Passetti, F., Humphris, G., & Matthews, K. (2012) · Neuroscience & Biobehavioral Reviews · doi:10.1016/j.neubiorev.2012.06.006 · Cited by 263
Psychological functioning of people living with chronic pain: a meta-analytic review
Burke, A. L. J., Mathias, J. L., & Denson, L. A. (2015) · British Journal of Clinical Psychology · doi:10.1111/bjc.12078 · Cited by 239
Grynszpan, O., Perbal, S., Pelissolo, A., Fossati, P., Jouvent, R., Dubal, S., & Perez-Diaz, F. (2011) · Psychological Medicine · doi:10.1017/s0033291710000607 · Cited by 225
Brief report: Cognitive flexibility in autism spectrum disorders: A quantitative review
Leung, R. C., & Zakzanis, K. K. (2014) · Journal of Autism and Developmental Disorders · doi:10.1007/s10803-014-2136-4 · Cited by 192
Self-citationHutchinson, A. D., & Mathias, J. L. (2007) · Journal of Neurology, Neurosurgery & Psychiatry · doi:10.1136/jnnp.2006.100669 · Cited by 187
Relapse rates after psychotherapy for depression – stable long-term effects? A meta-analysis
Steinert, C., Hofmann, M., Kruse, J., & Leichsenring, F. (2014) · Journal of Affective Disorders · doi:10.1016/j.jad.2014.06.043 · Cited by 172
Falleti, M. G., Maruff, P., Burman, P., & Harris, A. (2006) · Psychoneuroendocrinology · doi:10.1016/j.psyneuen.2006.01.005 · Cited by 165
Publication Bias in Meta-Analysis: Confidence Intervals for Rosenthal’s Fail-Safe Number
Fragkos, K. C., Tsagris, M., & Frangos, C. C. (2014) · International Scholarly Research Notices · doi:10.1155/2014/825383 · Cited by 159
The cognitive neuropsychology of obsessive-compulsive disorder: a critical review
Abramovitch, A., & Cooperman, A. (2015) · Journal of Obsessive-Compulsive and Related Disorders · doi:10.1016/j.jocrd.2015.01.002 · Cited by 126
Executive Functioning and Emotion Regulation in Children with and without ADHD
Groves, N. B., Wells, E. L., Soto, E. F., Marsh, C. L., Jaisle, E. M., Harvey, T. K., & Kofler, M. J. (2022) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-021-00883-0 · Cited by 110
The impact of treatment on HPA axis activity in unipolar major depression
McKay, M. S., & Zakzanis, K. K. (2010) · Journal of Psychiatric Research · doi:10.1016/j.jpsychires.2009.07.012 · Cited by 103
Self-citationWorking memory and short-term memory deficits in ADHD: A bifactor modeling approach
Kofler, M. J., Singh, L. J., Soto, E. F., Chan, E. S. M., Miller, C. E., Harmon, S. L., & Spiegel, J. A. (2020) · Neuropsychology · doi:10.1037/neu0000641 · Cited by 101
“For example, early meta-analytic estimates indicated medium magnitude impairments (d=0.55–0.63; Willcutt et al., 2005), which suggest that only 35%−40% of children with ADHD are likely to have working memory impairments based on converting effect sizes into proportion non-overlap (Zakzanis, 2001).”
BackgroundMethodResultInfluentialHeterogeneity in ADHD: Neurocognitive predictors of peer, family, and academic functioning
Kofler, M. J., Sarver, D. E., Spiegel, J. A., Day, T. N., Harmon, S. L., & Wells, E. L. (2017) · Child Neuropsychology · doi:10.1080/09297049.2016.1205010 · Cited by 94
“…slowed processing speed (Frazier, Demaree, & Youngstrom, 2004; Kofler et al., 2013) based on comparisons with typically developing groups (Zakzanis, 2001).1 In comparison, approximately 27% to 47% underperform on IQ tests (Frazier et al., 2004) that depend on these executive and other…”
BackgroundMethodInfluentialMcLennan, S. N., & Mathias, J. L. (2010) · International Journal of Geriatric Psychiatry · doi:10.1002/gps.2431 · Cited by 95
Kofler, M. J., Raiker, J. S., Sarver, D. E., Wells, E. L., & Soto, E. F. (2016) · Clinical Psychology Review · doi:10.1016/j.cpr.2016.04.004 · Cited by 88
Tucha, O., Prell, S., Mecklinger, L., Bormann-Kischkel, C., Kübber, S., Linder, M., Walitza, S., & Lange, K. W. (2006) · Psychopharmacology · doi:10.1007/s00213-006-0318-2 · Cited by 89
Steinert, C., Stadter, K., Stark, R., & Leichsenring, F. (2017) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2032 · Cited by 83
Telepsychology for Posttraumatic Stress Disorder: A systematic review
Bolton, A., & Dorstyn, D. (2015) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x15571996 · Cited by 74
An Examination of Relations Among Working Memory, ADHD Symptoms, and Emotion Regulation
Groves, N. B., Kofler, M. J., Wells, E. L., Day, T. N., & Chan, E. S. M. (2020) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-019-00612-8 · Cited by 75
Batastini, A. B., King, C. M., Morgan, R. D., & McDaniel, B. (2016) · Psychological Services · doi:10.1037/ser0000042 · Cited by 72
Meta-analytic review of memory impairment in behavioral variant frontotemporal dementia
Poos, J. M., Jiskoot, L. C., Papma, J. M., van Swieten, J. C., & van den Berg, E. (2018) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617718000115 · Cited by 57
Hutchinson, A. D., Mathias, J. L., & Banich, M. T. (2008) · Neuropsychology · doi:10.1037/0894-4105.22.3.341 · Cited by 56
Irwin, L. N., Soto, E. F., Chan, E. S. M., Miller, C. E., Carrington-Forde, S., Groves, N. B., & Kofler, M. J. (2021) · Child Neuropsychology · doi:10.1080/09297049.2020.1866521 · Cited by 39
“005), suggesting that approximately 38% of children with ADHD are perceived by their caregivers as demonstrating ADL impairments based on converting this effect size into the proportion of population non-overlap (Zakzanis, 2001).”
BackgroundInfluentialDorstyn, D. S., Saniotis, A., & Sobhanian, F. (2013) · Journal of Telemedicine and Telecare · doi:10.1177/1357633x13501767 · Cited by 31
Cognitive profiles discriminate between genetic variants of behavioral frontotemporal dementia
Poos, J. M., Jiskoot, L. C., Leijdesdorff, S. M. J., Seelaar, H., Panman, J. L., van der Ende, E. L., Mol, M. O., Meeter, L. H. H., Pijnenburg, Y. A. L., Donker Kaat, L., et al. (2020) · Journal of Neurology · doi:10.1007/s00415-020-09738-y · Cited by 31
Singh, L. J., Gaye, F., Cole, A. M., Chan, E. S. M., & Kofler, M. J. (2022) · Neuropsychology · doi:10.1037/neu0000798 · Cited by 32
Little, A., Byrne, C., & Coetzer, R. (2021) · NeuroRehabilitation · doi:10.3233/nre-201544 · Cited by 29
Schachar, R. J., Dupuis, A., Arnold, P. D., Anagnostou, E., Kelley, E., Georgiades, S., Nicolson, R., Townes, P., Burton, C. L., & Crosbie, J. (2023) · Research on Child and Adolescent Psychopathology · doi:10.1007/s10802-022-00958-6 · Cited by 30
de Oliveira, T. M. D., Felício, D. C., Filho, J. E., Fonseca, D. S., Durigan, J. L. Q., & Malaguti, C. (2022) · Journal of Bodywork and Movement Therapies · doi:10.1016/j.jbmt.2022.03.010 · Cited by 19
Standardized Effect Sizes and Image-Based Meta-Analytical Approaches for fMRI Data
Bossier, H., Nichols, T. E., & Moerkerke, B. (2019) · doi:10.1101/865881 · Cited by 14
Comparing neurocognition in severe chronic schizophrenia and frontotemporal dementia
Chan, H.-M., Stolwyk, R., Kelso, W., Neath, J., Walterfang, M., Mocellin, R., Pavlis, A., & Velakoulis, D. (2014) · Australian & New Zealand Journal of Psychiatry · doi:10.1177/0004867414529477 · Cited by 11
Chan, E. S. M., Gaye, F., Cole, A. M., Singh, L. J., & Kofler, M. J. (2023) · Neuropsychology · doi:10.1037/neu0000918 · Cited by 6
Metadata
- Authors
- Zakzanis, K. K.
- Affiliations
- University of Toronto (CA)
- Corresponding author
- Zakzanis, K. K.
- ORCID
- Zakzanis, K. K.
- Venue
- Archives of Clinical Neuropsychology
- Publisher
- Oxford University Press (OUP)
- Published
- 2001-10-01
- Volume
- 16
- Issue
- 7
- Pages
- 653-667
- ISSN
- 0887-6177 (ISSN-L)
- 1873-5843
- OpenAlex
- W1967986230
- Cited by
- 346
- References
- 47
- Citations by year
- 2025: 8 · 2024: 6 · 2023: 10 · 2022: 9 · 2021: 8 · 2020: 15 · 2019: 9 · 2018: 12 · 2017: 11 · 2016: 22 · 2015: 16 · 2014: 25 · 2013: 21 · 2012: 19
- FWCI
- 15.64
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Archives of Clinical Neuropsychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- effect size, Cohen's d, neuropsychological research, statistical interpretation, test sensitivity, brain-behavior relationship
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
