Random sampling, randomization, and equivalence of contrasted groups in psychotherapy outcome research.

Hsu, Louis M.·1989·Journal of Consulting and Clinical Psychology

Abstract

Random sampling and random assignment (randomization) are some of the most popular methods of equating contrasted groups on pre-existing nuisance variables. However, the small samples typically used in psychotherapy outcome studies raise some questions about the extent to which these methods eliminate the pretreatment nonequivalence of groups in this area of research. This article identifies conditions under which equivalence is likely (and unlikely) to be attained with simple random sampling and randomization in psychotherapy efficacy studies of the kind examined in recent meta-analyses. Some consequences of nonequivalence are viewed as manifestations of Simpson's paradox. Misinterpretations of estimates of the relative efficacy of treatments are expected in view of belief in the law of small numbers. The minimum sample sizes needed to protect against nonequivalence are compared with those needed to satisfy several other criteria.

Cites (9)

  1. Belief in the law of small numbers.
    Tversky, Amos, Kahneman, Daniel · 1971 · Psychological Bulletin
  2. Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial
    Pocock, Stuart J., Simon, Richard · 1975 · Biometrics
  3. The interpretation of interaction in contingency tables
    Simpson, E. H. · 1951 · Journal of the Royal Statistical Society Series B: Statistical Methodology
  4. Comparative Studies of Psychotherapies: Is It True That "Everyone Has Won and All Must Have Prizes"?
    Luborsky, Lester · 1975 · Archives of General Psychiatry
  5. Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance
    Jacobson, Neil S., Follette, William C., Revenstorf, Dirk · 1984 · Behavior Therapy
  6. The case against statistical significance testing
    Carver, Ronald · 1978 · Harvard Educational Review
  7. Forcing a sequential experiment to be balanced
    EFRON, BRADLEY · 1971 · Biometrika
  8. A paradox in the interpretation of group comparisons.
    Lord, Frederic M. · 1967 · Psychological Bulletin
  9. Meta-analysis of comparative therapy outcome studies: a replication and refinement.
    Shapiro, David A., Shapiro, Diana · 1982 · Psychological Bulletin

Show all 9 references

Cited by (11)

  1. Evidence-Based Psychosocial Treatments for Self-Injurious Thoughts and Behaviors in Youth
    Glenn, Catherine R., Franklin, Joseph C., Nock, Matthew K. · 2015 · Journal of Clinical Child & Adolescent Psychology
  2. Comparative effects of short-term psychodynamic psychotherapy and cognitive-behavioral therapy in depression
    Leichsenring, Falk · 2001 · Clinical Psychology Review
  3. Psychodynamic therapy: as efficacious as other empirically supported treatments? A meta-analysis testing equivalence of outcomes
    Steinert, Christiane, Munder, Thomas, Rabung, Sven, Hoyer, Jürgen, Leichsenring, Falk · 2017 · American Journal of Psychiatry
  4. Research Design in Clinical Psychology
    Kazdin, Alan E. · 2021
  5. Evidence Base Update of Psychosocial Treatments for Self-Injurious Thoughts and Behaviors in Youth
    Glenn, Catherine R., Esposito, Erika C., Porter, Andrew C., Robinson, Devin J. · 2019 · Journal of Clinical Child & Adolescent Psychology

See all 104

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Random sampling, randomization, and equivalence of contrasted groups in psychotherapy outcome research. https://app.cochat.ai/papers/10.1037//0022-006x.57.1.131
The paper
Hsu, L. M. (1989). Random sampling, randomization, and equivalence of contrasted groups in psychotherapy outcome research. Journal of Consulting and Clinical Psychology. https://doi.org/10.1037//0022-006x.57.1.131

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.