Rigor, Replication, and Reproducibility: Increasing the Relevance of Behavioral Disorders Research
The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
Abstract
Science is being criticized within scientific domains and society. To address this, we suggest that researchers in special education, broadly, and those working in the behavioral disorders field, specifically, consider three critical facets of research: (a) importance of rigor, (b) value of replication, and (c) transparency of reproducibility in behavioral disorders. In this manuscript, we describe critical features of the three facets. Then, we provide an empirical illustration of all three research facets via a quasi-experimental design analysis of school-wide positive behavior interventions and support (SWPBIS) and its impact on discipline outcomes. Recommendations are then provided for improving the quality of behavioral disorders research.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“The majority of available evaluations originate from the United States, and high‐quality research in other contexts remains scarce.”
From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Gage and Stevens 2018
In the citing paper
“Despite the intervention's broad implementation across approximately 23,000 schools in the United States, the review identified only four studies meeting the threshold for experimental or quasi‐experimental evaluation.”
From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Gage and Stevens 2018
In the citing paper
“Gage and Stevens (2018) actually assess treatment impact by fidelity”
From School‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review (2025) · cites this paper as Gage and Stevens 2018
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Estimating the reproducibility of psychological science
Open Science Collaboration (2015) · Science · doi:10.1126/science.aac4716 · Cited by 8,515
1,500 scientists lift the lid on reproducibility
Baker, M. (2016) · Nature · doi:10.1038/533452a · Cited by 4,209
HARKing: Hypothesizing after the results are known
Kerr, N. L. (1998) · Personality and Social Psychology Review · doi:10.1207/s15327957pspr0203_4 · Cited by 1,989
P-curve: A key to the file-drawer.
Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014) · Journal of Experimental Psychology: General · doi:10.1037/a0033242 · Cited by 1,500
Quality Indicators for Group Experimental and Quasi-Experimental Research in Special Education
Gersten, R., Fuchs, L. S., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. S. (2005) · Exceptional Children · doi:10.1177/001440290507100202 · Cited by 1,023
Bradshaw, C. P., Mitchell, M. M., & Leaf, P. J. (2010) · Journal of Positive Behavior Interventions · doi:10.1177/1098300709334798 · Cited by 868
Optimal Full Matching and Related Designs via Network Flows
Hansen, B. B., & Klopfer, S. O. (2006) · Journal of Computational and Graphical Statistics · doi:10.1198/106186006x137047 · Cited by 703 · cited 3×
“, same percentage of low SES students in each group), permit conclusions that any differences after an intervention are the result of the intervention. However, because there are other unmeasured characteristics, results are less internally valid as those from RCT (i.e., where any differences are considered the produc…”
BackgroundMethodInfluentialFacts Are More Important Than Novelty: Replication in the Education Sciences
Makel, M. C., & Plucker, J. A. (2014) · Educational Researcher · doi:10.3102/0013189x14545513 · Cited by 609 · cited 1×
“For example, Makel and Plucker (2014) reviewed the top 100 education journals to identify how many published studies were true replications.”
BackgroundThe prevalence of statistical reporting errors in psychology (1985–2013)
Nuijten, M. B., Hartgerink, C. H. J., van Assen, M. A. L. M., Epskamp, S., & Wicherts, J. M. (2016) · Behavior Research Methods · doi:10.3758/s13428-015-0664-2 · Cited by 510
Cited by
Lee, A., & Gage, N. A. (2020) · Psychology in the Schools · doi:10.1002/pits.22336 · Cited by 163
Self-citationGage, N. A., Grasley-Boy, N., Lombardo, M., & Anderson, L. (2020) · Behavioral Disorders · doi:10.1177/0198742919896305 · Cited by 26
“…only a handful of schools implementing SWPBIS and do not represent a statewide analysis; SWPBIS scale-up in South Carolina has not been fully financially supported by the state, but instead is supported by a loosely connected group of school- and university-based personnel (Gage & Stevens, 2018).”
BackgroundResultInfluentialSelf-citationSchool‐Based Interventions for Reducing Disciplinary School Exclusion. An Updated Systematic Review
Valdebenito, S., Gaffney, H., Arosemena-Burbano, M. J., Hitchcock, S., Jolliffe, D., & Sutherland, A. (2025) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.70063 · Cited by 8
Metadata
- Authors
- Gage, N. A., & Stevens, R. N.
- Venue
- Education and Treatment of Children
- Publisher
- Project MUSE
- Published
- 2018
- Volume
- 41
- Issue
- 4
- Pages
- 567-588
- ISSN
- 1934-8924
- Cited by
- 13
- References
- 9
- Language
- en
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.