The Implications of “Contamination” for Experimental Design in Education
The DOI matches its Crossref record; 7 of the 7 references it cites were found in a registry or an index.
Abstract
Experimental designs that randomly assign entire clusters of individuals (e.g., schools and classrooms) to treatments are frequently advocated as a way of guarding against contamination of the estimated average causal effect of treatment. However, in the absence of contamination, experimental designs that randomly assign intact clusters to treatments are less efficient than designs that randomly assign individual units within clusters. The current article considers the case of contamination processes that tend to make experimental and control subjects appear more similar than they truly are. The article demonstrates that, for most parameter values of practical interest, the statistical power of a randomized block (RB) design remains higher than the power of a cluster randomized (CR) design even when contamination causes the effect size to decrease by as much as 10%–60%. Furthermore, from the standpoint of point estimation, RB designs will tend to be preferred when true effect sizes are small and when the number of clusters in the experiment is not too large, but CR designs will tend to be preferred when true effect sizes are large or when the number of clusters in the experiment is large.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Statistical analysis and optimal design for cluster randomized trials.
Raudenbush, S. W. (1997) · Psychological Methods · doi:10.1037/1082-989x.2.2.173 · Cited by 636 · cited 3×
“These ANOVA models are the usual models used to compute power for the respective designs under consideration (Raudenbush, 1997; Raudenbush & Liu, 2000).”
MethodStandardized or simple effect size: what should be reported?
Baguley, T. (2009) · British Journal of Psychology · doi:10.1348/000712608x377117 · Cited by 619
Statistical power and optimal design for multisite randomized trials.
Raudenbush, S. W., & Liu, X. (2000) · Psychological Methods · doi:10.1037/1082-989x.5.2.199 · Cited by 593 · cited 2×
“These ANOVA models are the usual models used to compute power for the respective designs under consideration (Raudenbush, 1997; Raudenbush & Liu, 2000).”
MethodSample Size Planning for Statistical Power and Accuracy in Parameter Estimation
Maxwell, S. E., Kelley, K., & Rausch, J. R. (2008) · Annual Review of Psychology · doi:10.1146/annurev.psych.59.103006.093735 · Cited by 542
Contamination in trials: is cluster randomisation the answer?
Torgerson, D. J. (2001) · BMJ · doi:10.1136/bmj.322.7282.355 · Cited by 508
Statistical power for random assignment evaluations of education programs
Schochet, P. Z. (2008) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998607302714 · Cited by 216 · cited 7×
“Schochet (2008) argues that designs that randomize within cluster are appropriate when spillover effects are ‘‘expected to be small’’ but gives no guidance as to what might constitute a small spillover effect.”
BackgroundInfluentialThe Power of the Test for Treatment Effects in Three-Level Cluster Randomized Designs
Konstantopoulos, S. (2008) · Journal of Research on Educational Effectiveness · doi:10.1080/19345740701692522 · Cited by 124 · cited 1×
“…of treatment effects is larger in experimental designs that randomize clusters than it is in designs that randomize units within clusters (Blitstein, Hannan, Murray, & Shadish, 2005; Bloom et al., 2007; Cornfield, 1978; Konstantopoulos, 2008a, 2008b; Raudenbush et al., 2007; Schochet, 2008).”
Background
Cited by
Kraft, M. A., Blazar, D., & Hogan, D. (2018) · Review of Educational Research · doi:10.3102/0034654318759268 · Cited by 1,202
Tominey, S. L., & McClelland, M. M. (2011) · Early Education and Development · doi:10.1080/10409289.2011.574258 · Cited by 312
Jennings, P. A., Doyle, S., Oh, Y., Rasheed, D., Frank, J. L., & Brown, J. L. (2019) · Journal of School Psychology · doi:10.1016/j.jsp.2019.07.009 · Cited by 186
Addressing chronic absenteeism in schools: A meta-analysis of evidence-based interventions.
Eklund, K., Burns, M. K., Oyen, K., DeMarchena, S., & McCollom, E. M. (2022) · School Psychology Review · doi:10.1080/2372966x.2020.1789436 · Cited by 79
Flake, J. K. (2021) · Educational Psychologist · doi:10.1080/00461520.2021.1898962 · Cited by 58
“For example, educational researchers have contributed to understanding issues of implementation fidelity and treatment contamination in large-scale educational experiments (O’Donnell, 2008; Rhoads, 2011).”
Background
Metadata
- Authors
- Rhoads, C. H.
- Venue
- Journal of Educational and Behavioral Statistics
- Published
- 2011-02
- Volume
- 36
- Issue
- 1
- Pages
- 76-104
- ISSN
- 1076-9986 (ISSN-L)
- 1935-1054
- OpenAlex
- W2128246717
- Cited by
- 72
- References
- 7
- FWCI
- 2.96
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.