The consequence of ignoring a nested factor on measures of effect size in analysis of variance.
Abstract
Although the consequences of ignoring a nested factor on decisions to reject the null hypothesis of no treatment effects have been discussed in the literature, typically researchers in applied psychology and education ignore treatment providers (often a nested factor) when comparing the efficacy of treatments. The incorrect analysis, however, not only invalidates tests of hypotheses, but it also overestimates the treatment effect. Formulas were derived and a Monte Carlo study was conducted to estimate the degree to which the F statistic and treatment effect size measures are inflated by ignoring the effects due to providers of treatments. These untoward effects are illustrated with examples from psychotherapeutic treatments.
Cites (4)
- Experimental Design: Procedures for the Behavioral SciencesKirk, Roger · 2013
- The earth is round (p < .05).Cohen, Jacob · 1994 · American Psychologist
- A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, "all must have prizes."Wampold, Bruce E., Mondin, Gregory W., Moody, Marcia, Stich, Frederick, Benson, Kurt, Ahn, Hyun-nie · 1997 · Psychological Bulletin
- Empirically supported individual and group psychological treatments for adult mental disordersDeRubeis, Robert J., Crits-Christoph, Paul · 1998 · Journal of Consulting and Clinical Psychology
Cited by (26)
- Generalized Eta and Omega Squared Statistics: Measures of Effect Size for Some Common Research Designs.Olejnik, Stephen, Algina, James · 2003 · Psychological Methods
- A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior researchBorrelli, Belinda, Sepinwall, Deborah, Ernst, Denise, Bellg, Albert J., Czajkowski, Susan, Breger, Rosemary, DeFrancesco, Carol, Levesque, Chantal, Sharp, Daryl L., Ogedegbe, Gbenga et al. · 2005 · Journal of Consulting and Clinical Psychology
- Dialectical behavior therapy versus comprehensive validation therapy plus 12-step for the treatment of opioid dependent women meeting criteria for borderline personality disorderLinehan, Marsha M, Dimeff, Linda A, Reynolds, Sarah K, Comtois, Katherine Anne, Welch, Stacy Shaw, Heagerty, Patrick, Kivlahan, Daniel R · 2002 · Drug and Alcohol Dependence
- Culturally adapted psychotherapy and the legitimacy of myth: A direct-comparison meta-analysis.Benish, Steven G., Quintana, Stephen, Wampold, Bruce E. · 2011 · Journal of Counseling Psychology
- Meta-analysis of dropout in treatments for posttraumatic stress disorderImel, Zac E., Laska, Kevin, Jakupcak, Matthew, Simpson, Tracy L. · 2013 · Journal of Consulting and Clinical Psychology
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: The consequence of ignoring a nested factor on measures of effect size in analysis of variance. https://app.cochat.ai/papers/10.1037//1082-989x.5.4.425
- The paper
- Wampold, B. E., & Serlin, R. C. (2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods. https://doi.org/10.1037//1082-989x.5.4.425
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.