Clinical supervision research from 1981 to 1993: A methodological critique.

Ellis, Michael V., Ladany, Nicholas, Krengel, Maxine, Schult, Deborah·1996·Journal of Counseling Psychology

Abstract

Michael V. Ellis University at Albany, State University of New York Nicholas Ladany Temple University Maxine Krengel Boston Veterans Affairs Medical Center and Boston University School of Medicine Deborah Schult University at Albany, State University of New York The empirical studies in clinical supervision published from 1981 through 1993 were investigated to assess scientific rigor and to test whether the quality of methodology had improved since the review by R. K. Russell, A. M. Crimmings, and R. W. Lent (1984). The 144 studies were evaluated according to 49 threats to validity (T. D. Cook & D. T. Campbell, 1979; R. K. Russell et al., 1984; B. E. Wampold, B. Davis, & R. H. Good III, 1990) and 8 statistical variables (e.g., effect size, statistical power, and Type I and Type II error rates). The data revealed a shift to realistic field studies, unchecked Type I and Type II error rates, medium effect sizes, and inattention to hypothesis validity. Recommendations for designing and conducting a feasible and well-designed supervision study are offered. It can be argued that a primary goal of research in clinical supervision is to test and improve theory and to guide the practice of supervision (Ellis, 1991b). A thorough under- standing of the strengths and weaknesses of supervision research would ostensibly expand supervision theory and provide practitioners with information on how to train ef- fective counselors who, in turn, will provide more effective therapy. Although there have been numerous calls for in- creasing the scientific rigor of research on counselor super- vision and training (e.g., Ellis, 1991b; Hansen & Warner, Michael V. Ellis and Deborah Schult, Department of Counseling Psychology, University at Albany, State University of New York; Nicholas Ladany, Department of Counseling Psychology, Temple University; Maxine Krengel, Psychology Section (1168), Boston Veterans Affairs Medical Center and Department of Psychology, Boston University School of Medicine. Earlier versions of this article were presented at the 96th Annual Convention of the American Psychological Association, Atlanta, Georgia, August 1988, and at the meeting of the North Atlantic Regional Association of Counselor Education and Supervision, Albany, New York, October 1991. Maxine Krengel completed some of this research while a doc- toral student in the Department of Counseling Psychology, Uni- versity at Albany, State University of New York. We are grateful to Micki Friedlander, Richard Haase, and Erica Robbins Ellis for their insightful comments on earlier versions of this article. We express our appreciation to Eric Adams, Mafoozal Ali, Elizabeth Bhargava, Virginia Flander, David Hahn, Gohpa Khan, Michelle Mautner, Deborah Melincoff, Michael Remshard, Greg Savage, Heidi Weiss, Donna Wilson, and Bradley Wolgast for their data coding and entry assistance. Correspondence concerning this article should be addressed to Michael V. Ellis, Department of Counseling Psychology, Educa- tion 220, University at Albany, State University of New York, 1400 Washington Avenue, Albany, New York 12222. Electronic mail may be sent via Internet to me464@cnsibm.albany.edu. 35 1971; Holloway & Hosford, 1983; Russell, Crimmings, & Lent, 1984), a comprehensive and in-depth investigation of the actual state of scientific rigor has yet to be conducted. If supervision research is going to meet the goal of informing theory and practice, then a thorough assessment of its meth- odological limitations and implications is warranted. At least 32 reviews of empirically based articles pertain- ing to clinical supervision and counselor training have ap- peared in the literature. Although these reviews have made substantial contributions to the field, many did not evaluate systematically the methodological or the scientific rigor of the examined studies (e.g., Harkness & Poertner, 1989; Holloway, 1984, 1992; Holloway & Neufeldt, 1995; Kaplan, 1983; Lambert & Arnold, 1987; Leddick & Ber- nard, 1980; Liddle & Halpin, 1978; Matarazzo, 1971, 1978; Matarazzo & Garner, 1992; Matarazzo & Patterson, 1986; Russell & Petrie, 1994; Stoltenberg, McNeill, & Crethar, 1994; Yutrzenka, 1995) or did so in a cursory fashion (i.e., Baker & Daniels, 1989; Baker, Daniels, & Greeley, 1990; Ford, 1979; Hansen, Pound, & Petro, 1976; Robins, & Grimes, 1982; Hansen & Warner, 1971; Holloway & Johnston, 1985; Holloway & Wampold, 1986; Kurtz, Mar- shall, & Banspach, 1985; Loganbill, Hardy, & Delworth, 1982; Stein & Lambert, 1995; Worthington, 1987). Only four reviewers presented details of the methodological flaws encountered in the studies reviewed (Alberts & Edelstein, 1990; Avis & Sprenkle, 1990; Holloway, 1987; Russell et al., 1984). The result of not systematically evaluating the methodological issues may have erroneously led to (a) equating (or even outweighing) the findings of excellent research with poor research (Hogarty, 1989; Kline, 1983), (b) exacerbating the theoretical ambiguity in

Cites (5)

  1. The statistical power of abnormal-social psychological research: A review.
    Cohen, Jacob · 1962 · The Journal of Abnormal and Social Psychology
  2. Do studies of statistical power have an effect on the power of studies?
    Sedlmeier, Peter, Gigerenzer, Gerd · 1989 · Psychological Bulletin
  3. Improved Bonferroni-type multiple testing procedures.
    Holland, Burt S., Copenhaver, Margaret D. · 1988 · Psychological Bulletin
  4. Statistical power of psychological research: What have we gained in 20 years?
    Rossi, Joseph S. · 1990 · Journal of Consulting and Clinical Psychology
  5. The History of Supervision: A Critical Review
    LEDDICK, GEORGE R., BERNARD, JANINE M. · 1980 · Counselor Education and Supervision

Show all 5 references

Cited by (12)

  1. Training therapists in evidence-based practice: A critical review of studies from a systems-contextual perspective.
    Beidas, Rinad S., Kendall, Philip C. · 2010 · Clinical Psychology: Science and Practice
  2. Effective supervision in clinical practice settings: a literature review
    Kilminster, S M, Jolly, B C · 2000 · Medical Education
  3. The role of therapist training in the implementation of psychosocial treatments: A review and critique with recommendations
    Herschell, Amy D., Kolko, David J., Baumann, Barbara L., Davis, Abigail C. · 2010 · Clinical Psychology Review
  4. Toward a definition of intolerance of uncertainty: A review of factor analytical studies of the Intolerance of Uncertainty Scale
    Birrell, Jane, Meares, Kevin, Wilkinson, Andrew, Freeston, Mark · 2011 · Clinical Psychology Review
  5. An empirical definition of clinical supervision
    Milne, Derek · 2007 · British Journal of Clinical Psychology

See all 148

How to cite

Cite CoChat for the checks on this page, and the paper for its own findings.

This page
CoChat. (2026). Citation check: Clinical supervision research from 1981 to 1993: A methodological critique. https://app.cochat.ai/papers/10.1037//0022-0167.43.1.35
The paper
Ellis, M. V., Ladany, N., Krengel, M., & Schult, D. (1996). Clinical supervision research from 1981 to 1993: A methodological critique. Journal of Counseling Psychology. https://doi.org/10.1037//0022-0167.43.1.35

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.