The consequence of ignoring a nested factor on measures of effect size in analysis of variance.

Wampold, Bruce E., Serlin, Ronald C.·2000·Psychological Methods

Abstract

Although the consequences of ignoring a nested factor on decisions to reject the null hypothesis of no treatment effects have been discussed in the literature, typically researchers in applied psychology and education ignore treatment providers (often a nested factor) when comparing the efficacy of treatments. The incorrect analysis, however, not only invalidates tests of hypotheses, but it also overestimates the treatment effect. Formulas were derived and a Monte Carlo study was conducted to estimate the degree to which the F statistic and treatment effect size measures are inflated by ignoring the effects due to providers of treatments. These untoward effects are illustrated with examples from psychotherapeutic treatments.

Cites (4)

  1. Experimental Design: Procedures for the Behavioral Sciences
    Kirk, Roger · 2013
  2. The earth is round (p < .05).
    Cohen, Jacob · 1994 · American Psychologist
  3. A meta-analysis of outcome studies comparing bona fide psychotherapies: Empiricially, "all must have prizes."
    Wampold, Bruce E., Mondin, Gregory W., Moody, Marcia, Stich, Frederick, Benson, Kurt, Ahn, Hyun-nie · 1997 · Psychological Bulletin
  4. Empirically supported individual and group psychological treatments for adult mental disorders
    DeRubeis, Robert J., Crits-Christoph, Paul · 1998 · Journal of Consulting and Clinical Psychology

Show all 4 references

Cited by (26)

  1. Generalized Eta and Omega Squared Statistics: Measures of Effect Size for Some Common Research Designs.
    Olejnik, Stephen, Algina, James · 2003 · Psychological Methods
  2. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research
    Borrelli, Belinda, Sepinwall, Deborah, Ernst, Denise, Bellg, Albert J., Czajkowski, Susan, Breger, Rosemary, DeFrancesco, Carol, Levesque, Chantal, Sharp, Daryl L., Ogedegbe, Gbenga et al. · 2005 · Journal of Consulting and Clinical Psychology
  3. Dialectical behavior therapy versus comprehensive validation therapy plus 12-step for the treatment of opioid dependent women meeting criteria for borderline personality disorder
    Linehan, Marsha M, Dimeff, Linda A, Reynolds, Sarah K, Comtois, Katherine Anne, Welch, Stacy Shaw, Heagerty, Patrick, Kivlahan, Daniel R · 2002 · Drug and Alcohol Dependence
  4. Culturally adapted psychotherapy and the legitimacy of myth: A direct-comparison meta-analysis.
    Benish, Steven G., Quintana, Stephen, Wampold, Bruce E. · 2011 · Journal of Counseling Psychology
  5. Meta-analysis of dropout in treatments for posttraumatic stress disorder
    Imel, Zac E., Laska, Kevin, Jakupcak, Matthew, Simpson, Tracy L. · 2013 · Journal of Consulting and Clinical Psychology

See all 186

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: The consequence of ignoring a nested factor on measures of effect size in analysis of variance. https://app.cochat.ai/papers/10.1037//1082-989x.5.4.425
The paper
Wampold, B. E., & Serlin, R. C. (2000). The consequence of ignoring a nested factor on measures of effect size in analysis of variance. Psychological Methods. https://doi.org/10.1037//1082-989x.5.4.425

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.