The DOI matches its Crossref record; 47 of the 47 references it cites were found in a registry or an index.
Statistics to tell the truth, the whole truth, and nothing but the truth Formulae, illustrative numerical examples, and heuristic interpretation of effect size analyses for neuropsychological researchers
Open accessAbstract
If, as neuropsychologists, we think of the relationship between brain and behavior as the same as that between truth and reality, we must be equipped with statistical procedures that are coherent in terms of what we measure and what it represents. I believe that this necessary statistical procedure is effect size analysis, and without it, I believe that we fail to tell the truth, the whole truth, and nothing but the truth when describing our neuropsychological research. Accordingly, I review here the standard calculations of commonly employed effect sizes in two group designs and show how to adjust some familiar (and perhaps not so familiar) formulae using illustrative numerical examples. I also put forth an argument to adopt Cohen's measure as an expression of effect size based on its apropos to neuropsychological research. It is also argued that the interpretation of the magnitude of an effect size should depend on context, and not on pre-established heuristic benchmarks. It is noted, however, that effect sizes greater than 3.0 (OL%<5) might seem particularly appropriate when evaluating the sensitivity of neuropsychological tasks and in establishing test markers in neuropsychological disorders.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“meta-analytic effect sizes of d=0.46 to 0.55 (Willcutt et al., 2005) predict that 30%–36% of children with ADHD may demonstrate set shifting impairments based on converting effect size differences into expected population overlap proportions (Zakzanis et al., 2001).”
From Executive Functioning Heterogeneity in Pediatric ADHD (2019) · cites this paper as Zakzanis et al., 2001
In the citing paper
“which suggest that only 35%−40% of children with ADHD are likely to have working memory impairments based on converting effect sizes into proportion non-overlap (Zakzanis, 2001)”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“which correspond to impairment rates of 32% to 50% (Zakzanis, 2001)”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“The ‘proportion of non-overlap’ statistic estimates the proportion of ADHD cases that fall outside of the Non-ADHD range and thus provides an estimate of the percentage of ADHD children with impairments in each short-term/working memory component (Zakzanis, 2001); these estimates tend to align closely with values obtained from within-subject/heterogeneity methods for defining neurocognitive”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“This effect size suggests that approximately 38% of children with ADHD exhibit impaired visuospatial short-term memory (Zakzanis, 2001).”
From Working memory and short-term memory deficits in ADHD: A bifactor modeling approach (2020) · cites this paper as Zakzanis, 2001
In the citing paper
“meta-analytic effect sizes of d=0.46 to 0.55 (Willcutt et al., 2005) predict that 30%–36% of children with ADHD may demonstrate set shifting impairments based on converting effect size differences into expected population overlap proportions (Zakzanis et al., 2001).”
From Executive Functioning Heterogeneity in Pediatric ADHD (2019) · cites this paper as Zakzanis et al., 2001
In the citing paper
“Thus, whereas the uncorrected estimates suggest that 40%–44% of children with ADHD have working memory deficits (Zakzanis, 2001), these estimates increase to 81%–84% when using criterion, recall-based tasks with prominent executive demands (Coghill et al., 2014) and a sufficient number of administered trials (Wells et al., 2018).”
From Executive Functioning Heterogeneity in Pediatric ADHD (2019) · cites this paper as Zakzanis, 2001
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
Rating neurologic impairment in multiple sclerosis: An expanded disability status scale (EDSS)
Kurtzke, J. F. (1983) · Neurology · doi:10.1212/wnl.33.11.1444 · Cited by 15,011
Wasserman, S. S., Hedges, L. V., & Olkin, I. (1988) · Journal of Educational Statistics · doi:10.2307/1164953 · Cited by 10,377
The file drawer problem and tolerance for null results.
Rosenthal, R. (1979) · Psychological Bulletin · doi:10.1037/0033-2909.86.3.638 · Cited by 9,068
Meta-Analytic Procedures for Social Research
Rosenthal, R. (1991) · doi:10.4135/9781412984997 · Cited by 4,635
Kaplan, E., Goodglass, H., Weıntraub, S., & Segal, O. (2001) · Medical Entomology and Zoology · Cited by 4,378
Meta-Analytic Procedures for Social Research.
Metcalfe, A., & Rosenthal, R. (1994) · Journal of the Royal Statistical Society Series D (The Statistician) · doi:10.2307/2348600 · Cited by 4,249
Hall, R., Wittgenstein, L., & Anscombe, G. E. M. (1967) · The Philosophical Quarterly · doi:10.2307/2217461 · Cited by 4,043
Cohen, J. (1994) · American Psychologist · doi:10.1037/0003-066x.49.12.997 · Cited by 4,003
The Handbook of Research Synthesis.
Howel, D., Cooper, H., & Hedges, L. V. (1995) · Biometrics · doi:10.2307/2533295 · Cited by 3,657
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
DSM-IV: Diagnostic and Statistical Manual of Mental Disorders
Bell, C. C. (1994) · JAMA · doi:10.1001/jama.1994.03520100096046 · Cited by 2,608
Statistical Methods for Meta-Analysis
(1985) · Elsevier eBooks · doi:10.1016/c2009-0-03396-0 · Cited by 2,570
A Fail-Safe N for Effect Size in Meta-Analysis
Orwin, R. G. (1983) · Journal of Educational Statistics · doi:10.2307/1164923 · Cited by 2,064
The Handbook of Research Synthesis.
Light, R. J., Cooper, H., & Hedges, L. V. (1994) · Journal of the American Statistical Association · doi:10.2307/2291021 · Cited by 1,880
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778
Meta-Analysis: Quantitative Methods for Research Synthesis
Rosenthal, R., & Wolf, F. M. (1987) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2071646 · Cited by 1,371
Writing meta-analytic reviews.
Rosenthal, R. (1995) · Psychological Bulletin · doi:10.1037/0033-2909.118.2.183 · Cited by 1,108
Nass, R. D. (1984) · Annals of Neurology · doi:10.1002/ana.410160524 · Cited by 1,081
Schmidt, F. L. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.2.115 · Cited by 1,044
Theory-Testing in Psychology and Physics: A Methodological Paradox
Meehl, P. E. (1967) · Philosophy of Science · doi:10.1086/288135 · Cited by 962
The test of significance in psychological research.
Bakan, D. (1966) · Psychological Bulletin · doi:10.1037/h0020412 · Cited by 957
The case against statistical significance testing
Carver, R. (1978) · Harvard Educational Review · doi:10.17763/haer.48.3.t490261645281841 · Cited by 931
Do studies of statistical power have an effect on the power of studies?
Sedlmeier, P., & Gigerenzer, G. (1989) · Psychological Bulletin · doi:10.1037/0033-2909.105.2.309 · Cited by 748
The fallacy of the null-hypothesis significance test.
Rozeboom, W. W. (1960) · Psychological Bulletin · doi:10.1037/h0042040 · Cited by 740
Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations
Olejnik, S. F., & Algina, J. J. (2000) · Contemporary Educational Psychology · doi:10.1006/ceps.2000.1040 · Cited by 703
What If There Were No Significance Tests?
(2013) · Psychology Press eBooks · doi:10.4324/9781315827353 · Cited by 675
What If There Were No Significance Tests?
(2016) · Routledge eBooks · doi:10.4324/9781315629049 · Cited by 543
Cooper, H. M. (1979) · Journal of Personality and Social Psychology · doi:10.1037/0022-3514.37.1.131 · Cited by 418
Evaluating Results Using Corrected and Uncorrected Effect Size Estimates
Snyder, P., & Lawson, S. (1993) · The Journal of Experimental Education · doi:10.1080/00220973.1993.10806594 · Cited by 265
Meta-Analysis: Quantitative Methods for Research Synthesis
Beltramini, R. F., & Wolf, F. M. (1987) · Journal of Marketing Research · doi:10.2307/3151396 · Cited by 244
The appropriate use of null hypothesis testing.
Frick, R. W. (1996) · Psychological Methods · doi:10.1037/1082-989x.1.4.379 · Cited by 216
A nonparametric technique for meta-analysis effect size calculation.
KRAEMER, H. C., & Andrews, G. (1982) · Psychological Bulletin · doi:10.1037/0033-2909.91.2.404 · Cited by 140
Thompson, B. (1999) · Theory & Psychology · doi:10.1177/095935439992006 · Cited by 138
Reporting Practices and APA Editorial Policies Regarding Statistical Significance and Effect Size
Vacha-haase, T., Nilsson, J. E., Reetz, D. R., Lance, T. S., & Thompson, B. (2000) · Theory & Psychology · doi:10.1177/0959354300103006 · Cited by 138
Neuropsychological Differential Diagnosis
Kaplan, E., Zakzanis, K. K., & Leach, L. R. (1999) · Medical Entomology and Zoology · Cited by 133
Self-citationThe illogic of statistical inference for cumulative science
Guttman, L. (1985) · Applied Stochastic Models and Data Analysis · doi:10.1002/asm.3150010103 · Cited by 109
Null hypothesis disrespect in neuropsychology: dangers of alpha and beta errors.
Soper, H., Cicchetti, D. V., Satz, P., Light, R., & Orsini, D. L. (1988) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/01688638808408240 · Cited by 81
Thompsons, B., & Snyder, P. (1998) · Journal of Counseling & Development · doi:10.1002/j.1556-6676.1998.tb02702.x · Cited by 76
Thompson, B. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058002016 · Cited by 72
Thompson, B. (1999) · Educational Psychology Review · doi:10.1023/a:1022028509820 · Cited by 67
Heterogeneity in the cognitive profile of normal elderly
Valdois, S., Joanette, Y., Poissant, A., et al. (1990) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/01688639008401003 · Cited by 60
The Test of Significance in Psychological Research
Bakan, D. (2017) · Routledge eBooks · doi:10.4324/9781315134918-32 · Cited by 58
Brain is Related to Behavior (p < .05)
Zakzanis, K. K. (1998) · Journal of Clinical and Experimental Neuropsychology · doi:10.1076/jcen.20.3.419.821 · Cited by 47
Self-citationBieliauskas, L. A., Fastenau, P. S., Lacy, M. A., & Roper, B. L. (1997) · Journal of Clinical and Experimental Neuropsychology · doi:10.1080/01688639708403769 · Cited by 28
Rating Neurological Impairment in Multiple Sclerosis
Tawfiq, M. S. (2016) · Journals & Books Hosting (International Knowledge Sharing Platform) · Cited by 6
Cited by
Sorted by citations
Correlational effect size benchmarks
Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., & Pierce, C. A. (2015) · Journal of Applied Psychology · doi:10.1037/a0038047 · Cited by 768
Reaction time variability in ADHD: A meta-analytic review of 319 studies
Kofler, M. J., Rapport, M. D., Sarver, D. E., Raiker, J. S., Orban, S. A., Friedman, L. M., & Kolomeyer, E. G. (2013) · Clinical Psychology Review · doi:10.1016/j.cpr.2013.06.001 · Cited by 648
Inhibitory control and psychopathology: A meta-analysis of studies using the stop signal task
LIPSZYC, J., & SCHACHAR, R. (2010) · Journal of the International Neuropsychological Society · doi:10.1017/s1355617710000895 · Cited by 627
Maruff, P., Thomas, E., Cysique, L., Brew, B., Collie, A., Snyder, P., & Pietrzak, R. H. (2009) · Archives of Clinical Neuropsychology · doi:10.1093/arclin/acp010 · Cited by 635
Innovative technology-based interventions for autism spectrum disorders: a meta-analysis
Grynszpan, O., Weiss, P. L. (., Perez-Diaz, F., & Gal, E. (2014) · Autism · doi:10.1177/1362361313476767 · Cited by 621
Metadata
- Authors
- Zakzanis, K. K.
- Affiliations
- University of Toronto (CA)
- Corresponding author
- Zakzanis, K. K.
- ORCID
- Zakzanis, K. K.
- Venue
- Archives of Clinical Neuropsychology
- Publisher
- Oxford University Press (OUP)
- Published
- 2001-10-01
- Volume
- 16
- Issue
- 7
- Pages
- 653-667
- ISSN
- 0887-6177 (ISSN-L)
- 1873-5843
- OpenAlex
- W1967986230
- Cited by
- 346
- References
- 47
- Citations by year
- 2025: 8 · 2024: 6 · 2023: 10 · 2022: 9 · 2021: 8 · 2020: 15 · 2019: 9 · 2018: 12 · 2017: 11 · 2016: 22 · 2015: 16 · 2014: 25 · 2013: 21 · 2012: 19
- FWCI
- 15.64
- Type
- journal article
- Language
- en
- Open access
- bronze
- doi.org
- Locations
- Archives of Clinical Neuropsychology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
- Keywords
- effect size, Cohen's d, neuropsychological research, statistical interpretation, test sensitivity, brain-behavior relationship
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
