Randomization tests as alternative analysis methods for behavior‐analytic data
Open accessThe DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Randomization statistics offer alternatives to many of the statistical methods commonly used in behavior analysis and the psychological sciences, more generally. These methods are more flexible than conventional parametric and nonparametric statistical techniques in that they make no assumptions about the underlying distribution of outcome variables, are relatively robust when applied to small‐n data sets, and are generally applicable to between‐groups, within‐subjects, mixed, and single‐case research designs. In the present article, we first will provide a historical overview of randomization methods. Next, we will discuss the properties of randomization statistics that may make them particularly well suited for analysis of behavior‐analytic data. We will introduce readers to the major assumptions that undergird randomization methods, as well as some practical and computational considerations for their application. Finally, we will demonstrate how randomization statistics may be calculated for mixed and single‐case research designs. Throughout, we will direct readers toward resources that they may find useful in developing randomization tests for their own data.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Normality Tests for Statistical Analysis: A Guide for Non-Statisticians
Ghasemi, A., & Zahediasl, S. (2012) · International Journal of Endocrinology and Metabolism · doi:10.5812/ijem.3505 · Cited by 4,368
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Toward a functional analysis of self-injury
Iwata, B. A., Dorsey, M. F., Slifer, K. J., Bauman, K. E., & Richman, G. S. (1994) · Journal of Applied Behavior Analysis · doi:10.1901/jaba.1994.27-197 · Cited by 2,644
Evidence-based practice in psychology.
American Psychological Association (2006) · American Psychologist · doi:10.1037/0003-066x.61.4.271 · Cited by 2,478
Edgington, E., Edgington, E., & Onghena, P. (2007) · doi:10.1201/9781420011814 · Cited by 2,030
The Importance of the Normality Assumption in Large Public Health Data Sets
Lumley, T., Diehr, P., Emerson, S., & Chen, L. (2002) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.23.100901.140546 · Cited by 1,803
Experimental methods: Between-subject and within-subject design
Charness, G., Gneezy, U., & Kuhn, M. A. (2012) · Journal of Economic Behavior & Organization · doi:10.1016/j.jebo.2011.08.009 · Cited by 1,469
Pickard, K. E., & Ingersoll, B. R. (2016) · Autism · doi:10.1177/1362361315569745 · Cited by 170
Malignant side effects of null-hypothesis significance testing
Branch, M. (2014) · Theory & Psychology · doi:10.1177/0959354314525282 · Cited by 81 · cited 1×
“…embracing statistical methods is a good thing or a bad thing for the science is up to debate (for contrasting views, interested readers may see, e.g., Branch, 1999; 2014; Hopkins, Cole, & Mason, 1998; Huitema, 1986; Kratochwill & Levin, 2014; Michael, 1974; Shadish, Hedges, & Pustejovsky, 2014).”
Cited by
Single‐case experimental designs: Characteristics, changes, and challenges
Kazdin, A. E. (2021) · Journal of the Experimental Analysis of Behavior · doi:10.1002/jeab.638 · Cited by 166
A Priori Justification for Effect Measures in Single-Case Experimental Designs
Manolov, R., Moeyaert, M., & Fingerhut, J. E. (2022) · Perspectives on Behavior Science · doi:10.1007/s40614-021-00282-2 · Cited by 53
“Except for randomization tests (Craig & Fisher, 2019), inferential information requires that the quantification has a known approximate sampling distribution.”
MethodKrasny‐Pacini, A. (2023) · Developmental Medicine & Child Neurology · doi:10.1111/dmcn.15513 · Cited by 20
Meir, E., Pope, D., Abraham, J. K., Kim, K. J., Maruca, S., & Palacio, J. (2024) · CBE—Life Sciences Education · doi:10.1187/cbe.22-08-0158 · Cited by 10
“To check for homogeneity of the samples, we compared the ratio of variance on the preassessment scores to that of the pos-tassessment scores and it was suitably low (ratio = 1.66; Craig and Fisher, 2019).”
InfluentialManolov, R., & Onghena, P. (2023) · Behavior Research Methods · doi:10.3758/s13428-023-02230-1 · Cited by 5
“Rand-omization tests have several advantages in the SCED context (Craig & Fisher, 2019; Jacobs, 2019; Kratochwill & Levin, 2010; Onghena, 1992), including the applicability to all kinds of SCEDs 1 , the absence of parametric assumptions 2 , and the flexibility in choosing the test statistic that…”
Metadata
- Authors
- Craig, A. R., & Fisher, W. W.
- Venue
- Journal of the Experimental Analysis of Behavior
- Publisher
- Wiley
- Published
- 2019-02
- Volume
- 111
- Issue
- 2
- Pages
- 309-328
- ISSN
- 0022-5002 (ISSN-L)
- 1938-3711
- OpenAlex
- W2911875847
- Cited by
- 56
- References
- 9
- FWCI
- 6.39
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.