The DOI matches its Crossref record; 63 of the 65 references it cites were found in a registry or an index.
An improved effect size for single-case research: nonoverlap of all pairs.
Abstract
Nonoverlap of All Pairs (NAP), an index of data overlap between phases in single-case research, is demonstrated and field tested with 200 published AB contrasts. NAP is a novel application of an established effect size known in various forms as Area Under the Curve (AUC), the Common Language Effect Size (CL), the Probability of Superiority (PS), the Dominance Statistic (DS), Mann-Whitney's U, and Sommers D, among others. NAP was compared with 3 other non-overlap-based indices: PND (percent of nonoverlapping data), PEM (percent of data points exceeding the median), and PAND (percent of all nonoverlapping data), as well as Pearson's R(2). Five questions were addressed about NAP: (a) typical NAP values, (b) its ability to discriminate among typical single-case research results, (c) its power and precision (confidence interval width), (d) its correlation with the established effect size index, R(2), and (e) its relationship with visual judgments. Results were positive, the new index equaling or outperforming the other overlap indices on most criteria.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“the effect sizes found in SCED studies are usually higher than those found in group designs such as randomized controlled trials (Parker and Vannest, 2009; Shadish et al., 2016).”
From A Multiple-Baseline Evaluation of Acceptance and Commitment Therapy Focused on Repetitive Negative Thinking for Comorbid Generalized Anxiety Disorder and Depression (2020) · cites this paper as Parker and Vannest, 2009
In the citing paper
“Parker and Vannest (2009) proposed the following ranges in order to interpret NAP; scores ranging from 0 to 0.65 are weak effects, from 0.66 to 0.92 are medium effects, and from 0.93 to 1.0 are large or strong effects.”
From Acceptance and commitment therapy for individuals with depressive and anxiety symptoms following acquired brain injury: A non-concurrent multiple baseline design across four cases. (2023) · cites this paper as Parker and Vannest (2009)
Citations
Cites
Statistical Power Analysis for the Behavioral Sciences
Muller, K. E., & Cohen, J. (1989) · Technometrics · doi:10.2307/1270020 · Cited by 99,603
Statistical power analysis for the behavioral sciences
Cohen, J. (1990) · Computers Environment and Urban Systems · doi:10.1016/0198-9715(90)90050-4 · Cited by 65,417
Statistical Power Analysis for the Behavioral Sciences (2nd ed.).
Lachenbruch, P. A., & Cohen, J. (1989) · Journal of the American Statistical Association · doi:10.2307/2290095 · Cited by 37,421
An analysis of variance test for normality (complete samples)
SHAPIRO, S. S., & WILK, M. B. (1965) · Biometrika · doi:10.1093/biomet/52.3-4.591 · Cited by 23,480
The meaning and use of the area under a receiver operating characteristic (ROC) curve.
Hanley, J. A., & McNeil, B. J. (1982) · Radiology · doi:10.1148/radiology.143.1.7063747 · Cited by 21,856
Practical Nonparametric Statistics
Groggel, D. J., & Conover, W. J. (2000) · Technometrics · doi:10.2307/1271101 · Cited by 11,076
The Use of Single-Subject Research to Identify Evidence-Based Practice in Special Education
Horner, R. H., Carr, E. G., Halle, J., McGee, G., Odom, S., & Wolery, M. (2005) · Exceptional Children · doi:10.1177/001440290507100203 · Cited by 3,368
Robust Tests for the Equality of Variances
Brown, M. B., & Forsythe, A. B. (1974) · Journal of the American Statistical Association · doi:10.1080/01621459.1974.10482955 · Cited by 3,216
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Graphical Methods for Data Analysis.
Healy, M. J. R., Chambers, J. M., Cleveland, W. S., Kleiner, B., & Tukey, P. A. (1984) · Journal of the Royal Statistical Society Series A (General) · doi:10.2307/2981587 · Cited by 2,315
Empirically supported psychological interventions: Controversies and evidence
Chambless, D. L., & Ollendick, T. H. (2001) · Annual Review of Psychology · doi:10.1146/annurev.psych.52.1.685 · Cited by 2,097
Graphical Methods for Data Analysis.
Fisher, N. I., Chambers, J. M., Cleveland, W. S., Kleiner, B., & Tukey, P. A. (1984) · Biometrics · doi:10.2307/2531418 · Cited by 1,982
Practical Nonparametric Statistics (2nd ed).
Groeneveld, R. A., & Conover, W. J. (1981) · Journal of the American Statistical Association · doi:10.2307/2287607 · Cited by 1,869
Practical significance: a concept whose time has come
Kirk, R. E. (1996) · Educational and Psychological Measurement · doi:10.1177/0013164496056005002 · Cited by 1,778 · cited 4×
“The most commonly published effect sizes in the social sciences are members of the R(2) family (including Eta(2)) (Kirk, 1996), though they entail data assumptions that single case studies often”
BackgroundInfluentialTactics of Scientific Research
Sidman, M. (1960) · Cited by 1,638
Dominance statistics: Ordinal analyses to answer ordinal questions.
Cliff, N. (1993) · Psychological Bulletin · doi:10.1037/0033-2909.114.3.494 · Cited by 1,542
Meta-Analysis: Quantitative Methods for Research Synthesis
Rosenthal, R., & Wolf, F. M. (1987) · Contemporary Sociology A Journal of Reviews · doi:10.2307/2071646 · Cited by 1,371
A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong
Vargha, A., & Delaney, H. D. (2000) · Journal of Educational and Behavioral Statistics · doi:10.3102/10769986025002101 · Cited by 1,164
The Quantitative Synthesis of Single-Subject Research
Scruggs, T. E., Mastropieri, M. A., & Casto, G. C. (1987) · Remedial and Special Education · doi:10.1177/074193258700800206 · Cited by 971
A common language effect size statistic
McGraw, K. O., & Wong, S. P. (1992) · Psychological Bulletin · doi:10.1037/0033-2909.111.2.361 · Cited by 934 · cited 1×
“Applied to continuous data it is the Common Language Effect Size (CL) (McGraw & Wong, 1992).”
MethodStatistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1989) · American Psychologist · doi:10.1037/0003-066x.44.10.1276 · Cited by 863
Effect sizes for research: A broad practical approach.
Grissom, R. J., & Kim, J. J. (2005) · Cited by 732
Statistical procedures and the justification of knowledge in psychological science.
Rosnow, R. L., & Rosenthal, R. (1992) · American Psychological Association eBooks · doi:10.1037/10109-027 · Cited by 653
Practical Nonparametric Statistics, 3rd Ed.
Borror, C. M. (2001) · Journal of Quality Technology · doi:10.1080/00224065.2001.11980074 · Cited by 590
Social Science Research and Decision-Making.
Conklin, G. H., Weiss, C. H., & Bucuvalas, M. J. (1982) · Social Forces · doi:10.2307/2578408 · Cited by 540
M., D. W., & Cleveland, W. S. (1986) · The American Journal of Psychology · doi:10.2307/1422498 · Cited by 438
The Improvement Rate Difference for Single-Case Research
Parker, R. I., Vannest, K. J., & Brown, L. (2009) · Exceptional Children · doi:10.1177/001440290907500201 · Cited by 418
Self-citationPercentage of All Non-Overlapping Data (PAND)
Parker, R. I., Hagan–Burke, S., & Vannest, K. J. (2007) · The Journal of Special Education · doi:10.1177/00224669070400040101 · Cited by 396
Self-citationParameters behind “Nonparametric” Statistics: Kendall's tau, Somers’ D and Median Differences
Newson, R. (2002) · The Stata Journal: Promoting communications on statistics and Stata · doi:10.1177/1536867x0200200103 · Cited by 394
An Alternative Method for Quantitative Synthesis of Single-Subject Researches
Ma, H.-H. (2006) · Behavior Modification · doi:10.1177/0145445504272974 · Cited by 371
Design and analysis of single-case research
(1997) · Choice Reviews Online · doi:10.5860/choice.35-1208 · Cited by 297
How many discoveries have been lost by ignoring modern statistical methods?
Wilcox, R. R. (1998) · American Psychologist · doi:10.1037/0003-066x.53.3.300 · Cited by 295
Kratochwill, T. R., & Stoiber, K. C. (2002) · School Psychology Quarterly · doi:10.1521/scpq.17.4.341.20872 · Cited by 288
Probabilistic index: an intuitive non‐parametric approach to measuring the size of treatment effects
Ación, L., Peterson, J. J., TEMPLE, S. D., & Arndt, S. V. (2005) · Statistics in Medicine · doi:10.1002/sim.2256 · Cited by 270
INCONSISTENT VISUAL ANALYSES OF INTRASUBJECT DATA
DeProspero, A., & Cohen, S. (1979) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1979.12-573 · Cited by 260
Design and Analysis of Single-Case Research
(2014) · Psychology Press eBooks · doi:10.4324/9781315806402 · Cited by 244
Statistics for the Social Sciences
Reichardt, C. S., Novotny, J. A., & Wilcox, R. R. (1997) · Journal of the American Statistical Association · doi:10.2307/2965738 · Cited by 197
Statistics for the Social Sciences.
Lound, C., & Sirkin, R. M. (1996) · Journal of the Royal Statistical Society Series A (Statistics in Society) · doi:10.2307/2983353 · Cited by 186
INTERRUPTED TIME‐SERIES ANALYSIS AND ITS APPLICATION TO BEHAVIORAL DATA
Hartmann, D. P., Gottman, J. M., Jones, R. R., Gardner, W. P., Kazdin, A. E., & Vaught, R. S. (1980) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1980.13-543 · Cited by 166
Evaluating Single-Case Research Data: A Comparison of Seven Statistical Methods
Parker, R. I., & Brossart, D. F. (2003) · Behavior Therapy · doi:10.1016/s0005-7894(03)80013-8 · Cited by 144
Self-citationColeman, D. E., & Cleveland, W. (1987) · Technometrics · doi:10.1080/00401706.1987.10488193 · Cited by 132
Controlling baseline trend in single-case research.
Parker, R. I., Cryer, J., & Byrns, G. (2006) · School Psychology Quarterly · doi:10.1037/h0084131 · Cited by 130
Self-citationMeasures of strength of association: A comparative examination.
Maxwell, S. E., Camp, C. J., & Arvey, R. D. (1981) · Journal of Applied Psychology · doi:10.1037/0021-9010.66.5.525 · Cited by 126
Effect Sizes in Single Case Research: How Large is Large?
Parker, R. I., Brossart, D. F., Vannest, K. J., Long, J. R., De-Alba, R. G., Baugh, F. G., & Sullivan, J. R. (2005) · School Psychology Review · doi:10.1080/02796015.2005.12086279 · Cited by 122
Self-citationOn the Problem of Confidence Intervals
Neyman, J. (1935) · The Annals of Mathematical Statistics · doi:10.1214/aoms/1177732585 · Cited by 122
Autocorrelation estimation and inference with small samples.
Huitema, B. E., & McKean, J. W. (1991) · Psychological Bulletin · doi:10.1037/0033-2909.110.2.291 · Cited by 114
Visual Inspection and Statistical Analysis in Single-Case Designs
Park, H. S., Marascuilo, L. A., & Gaylord‐Ross, R. (1990) · The Journal of Experimental Education · doi:10.1080/00220973.1990.10806545 · Cited by 110
Steiger, J. H., & Fouladi, R. T. (1992) · Behavior Research Methods, Instruments, & Computers · doi:10.3758/bf03203611 · Cited by 108
Group Overlap as a Basis for Effect Size
Huberty, C. J., & Lowman, L. L. (2000) · Educational and Psychological Measurement · doi:10.1177/0013164400604004 · Cited by 101
Visual inspection of single-subject data: an empirical analysis.
Ottenbacher, K. J. (1990) · PubMed · Cited by 78
Median-Based Overlap Analysis for Single Case Data
Parker, R. I., & Hagan–Burke, S. (2007) · Behavior Modification · doi:10.1177/0145445507303452 · Cited by 63
Self-citationEvidence-based interventions in school psychology: Opportunities, challenges, and cautions.
Christenson, S. L., Carlson, C. I., & Valdez, C. R. (2002) · School Psychology Quarterly · doi:10.1521/scpq.17.4.466.20862 · Cited by 57
Interrater Reliability of Therapists' Judgments of Graphed Data
Harbst, K., Ottenbacher, K. J., & Harris, S. R. (1991) · Physical Therapy · doi:10.1093/ptj/71.2.107 · Cited by 56
Twenty Nonparametric Statistics And Their Large Sample Approximations
Fahoome, G. F. (2002) · Journal of Modern Applied Statistical Methods · doi:10.22237/jmasm/1036110540 · Cited by 51
“Make things as simple as possible, but no simpler.” A rejoinder to Scruggs and Mastropieri
Allison, D. B., & Gorman, B. S. (1994) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)90170-8 · Cited by 49
Increased Reliability for Single-Case Research Results: Is the Bootstrap the Answer?
Parker, R. I. (2006) · Behavior Therapy · doi:10.1016/j.beth.2006.01.007 · Cited by 43
Self-citationSerial Dependency in Single-Case Time Series
Greenwood, M. A. (2014) · doi:10.4324/9781315806402-13 · Cited by 43
Autocorrelation in behavioral data: An alternative perspective.
Sharpley, C. F., & Alavosius, M. P. (1988) · Cited by 38
Making Statistics More Meaningful for Policy Research and Program Evaluation
May, H. (2004) · American Journal of Evaluation · doi:10.1177/109821400402500408 · Cited by 35
Autocorrelation in applied behavior analysis: Myth or reality?
Suen, H. K., & Ary, D. (1987) · Behavioral Assessment · Cited by 29
A cautionary note on the use of omega squared to evaluate the effectiveness of behavioral treatments
Mitchell, C., & Hartmann, D. P. (1981) · Behavioral Assessment · Cited by 28
Comparing two groups by simple graphs.
Darlington, R. B. (1973) · Psychological Bulletin · doi:10.1037/h0033854 · Cited by 23
The Mann–Whitney statistic: continuous use and discovery
D’Agostino, R. B., Campbell, M., & Greenhouse, J. B. (2006) · Statistics in Medicine · doi:10.1002/sim.2508 · Cited by 22
Flagged first, then by citations
2 of the 65 references couldn’t be matched to a known paper.
Research works printed but not linked: 1
- Newson, R. B., 2000. snp15.1: Update to somersd. Stata Technical Bulletin 57: 35. In Stata Technical Bulletin Reprints, vol. 10, 322–323. College Station, TX: Stata Press.
Cited by
Sorted by citations
Combining nonoverlap and trend for single-case research: Tau-U
Parker, R. I., Vannest, K. J., Davis, J. L., & Sauber, S. B. (2011) · Behavior Therapy · doi:10.1016/j.beth.2010.08.006 · Cited by 1,468
Self-citationEffect Size in Single-Case Research: A Review of Nine Nonoverlap Techniques
Parker, R. I., Vannest, K. J., & Davis, J. L. (2011) · Behavior Modification · doi:10.1177/0145445511399147 · Cited by 841
“Although well-documented limitations exist and call for its abandonment (Kratochwill et al., 2010; Parker & Vannest, 2009), PND (Scruggs et al., 1987) is still widely used and thus included here for comparison.”
BackgroundMethodInfluentialSelf-citationEvaluating Intervention Effects in Single‐Case Research Designs
Vannest, K. J., & Ninci, J. (2015) · Journal of Counseling & Development · doi:10.1002/jcad.12038 · Cited by 665
Self-citationSingle-case experimental designs: a systematic review of published research and current standards
Smith, J. D. (2012) · Psychological Methods · doi:10.1037/a0029312 · Cited by 630
“Visual analysis, following systematic procedures such as those provided by Franklin, Gorman, Beasley, and Allison (1997) and Parsonson and Baer (1978), remains the standard by which SCED data are most commonly analyzed (Parker, Cryer, & Byrns, 2006). Visual analysis can arguably be applied to all SCEDs. However, a num…”
MethodKrasny-Pacini, A., & Evans, J. (2018) · Annals of Physical and Rehabilitation Medicine · doi:10.1016/j.rehab.2017.12.002 · Cited by 373
Metadata
- Authors
- Parker, R. I., & Vannest, K.
- Affiliations
- Texas A&M University (US)
- Corresponding author
- Parker, Richard I.
- ORCID
- Vannest, Kimberly
- Venue
- Behavior Therapy
- Publisher
- Elsevier BV
- Published
- 2009-12
- Volume
- 40
- Issue
- 4
- Pages
- 357-367
- ISSN
- 0005-7894 (ISSN-L)
- OpenAlex
- W2028628292
- Cited by
- 1,126
- References
- 65
- Citations by year
- 2026: 61 · 2025: 95 · 2024: 85 · 2023: 97 · 2022: 99 · 2021: 90 · 2020: 87 · 2019: 81 · 2018: 86 · 2017: 84 · 2016: 69 · 2015: 57 · 2014: 49 · 2013: 30 · 2012: 35
- FWCI
- 12.23
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Behavior Therapy (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Developmental and Educational Psychology › Behavioral and Psychological Studies
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Psychology Research and Bibliometrics
- Keywords
- effect size, single subject research, data overlap, R-squared, visual assessment, confidence interval width, statistical power
- MeSH terms
- Humans, Models, Psychological, Probability, Psychology, Research Design, Confidence Intervals, Area Under Curve
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
