Cited by 162Psychotherapy Techniques and ApplicationsPsychometric Methodologies and TestingReliability and Agreement in Measurement

The DOI matches its Crossref record; 70 of the 70 references it cites were found in a registry or an index.

Reliability Generalization of Working Alliance Inventory Scale Scores

Hanson, William E., Curry, Kyle T., Bandalos, Deborah L.·2002·Educational and Psychological Measurement

DOI 10.1177/0013164402062004008

Open this paper in CoChat — free account

Abstract

Reliability generalization (RG) was used to study five versions of the Working Alliance Inventory (WAI), including scores from 12 different scales. Sixty-seven internal consistency estimates, six interrater reliability estimates, and four study characteristics were analyzed. In general, reliability estimates of WAI scale scores appear to be robust. Mean reliability estimates ranged, in this sample of studies, from .79 to .97, with a modal estimate of .92. Variability in reliability estimates was, based on simple bivariate correlations, associated with client and therapist sample size for WAI total scores (observer version). Implications for measuring alliance using the WAI and conducting future RG studies on psychotherapy process measures are discussed.

Integrity signals

Registry

Crossref · 10.1177/0013164402062004008

Cites

70 of 70 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology.
Cicchetti, D. V. (1994) · Psychological Assessment · doi:10.1037/1040-3590.6.4.284 · Cited by 9,349
The generalizability of the psychoanalytic concept of the working alliance.
Bordin, E. S. (1979) · Psychotherapy: Theory, Research & Practice · doi:10.1037/h0085885 · Cited by 5,238
Introduction to Classical and Modern Test Theory
Crocker, L. M., & Algina, J. J. (1986) · Cited by 5,090
Measurement, Design, and Analysis: An Integrated Approach.
Funk, S. G., Pedhazur, E. J., & Schmelkin, L. P. (1992) · Journal of the American Statistical Association · doi:10.2307/2290248 · Cited by 4,007
Development and validation of the working alliance inventory
Horvath, A. O., & Greenberg, L. S. (1989) · Journal of Counseling Psychology · doi:10.1037//0022-0167.36.2.223 · Cited by 3,520
Relation of the therapeutic alliance with outcome and other variables: a meta-analytic review
Martin, D. J., Garske, J. P., & Davis, M. K. (2000) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.68.3.438 · Cited by 3,355
Educational measurement, 3rd ed.
Linn, R. L. (1989) · Cited by 2,858
Statistical methods in psychology journals: Guidelines and explanations.
Wilkinson, L., & Task Force on Statistical Inference (1999) · American Psychologist · doi:10.1037/0003-066x.54.8.594 · Cited by 2,791
Relation between working alliance and outcome in psychotherapy: A meta-analysis
Horvath, A. O., & Symonds, B. D. (1991) · Journal of Counseling Psychology · doi:10.1037/0022-0167.38.2.139 · Cited by 2,673
American Educational Research Journal, 1964-1968
Vesta, F. J. D., & Grinder, R. E. (1968) · American Educational Research Journal · doi:10.2307/1162008 · Cited by 2,606
Educational measurement.
Douglas, O. B., & Holland, B. F. (2015) · MacMillan Co eBooks · doi:10.1037/14910-020 · Cited by 2,283
The benefits of psychotherapy
Smith, M. L., Glass, G. V., & Miller, T. I. (1980) · Medical Entomology and Zoology · Cited by 2,123
The Great Psychotherapy Debate: Models Methods and Findings
Horvath, A. O. (2002) · Psychotherapy Research · doi:10.1093/ptr/12.1.108 · Cited by 1,705
Journal of Psychotherapy Practice and Research
Terrell, C. D. (1993) · JAMA · doi:10.1001/jama.1993.03510010107047 · Cited by 1,686
Factor structure of the working alliance inventory
Tracey, T. J., & Kokotovic, A. M. (1989) · Psychological Assessment: A Journal of Consulting and Clinical Psychology · doi:10.1037/1040-3590.1.3.207 · Cited by 1,501 · cited 5×
“The WAI-S is a 12-item self-report measure of WA (Tracey & Kokotovic, 1989).”
Influential
The psychotherapeutic process: A research handbook.
Greenberg, L. S., & Pinsof, W. M. (1986) · Cited by 1,386
The role of the therapeutic alliance in psychotherapy.
Horvath, A. O., & Luborsky, L. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.61.4.561 · Cited by 1,331
The role of the therapeutic alliance in psychotherapy.
Horvath, A. O., & Luborsky, L. (1993) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.61.4.561 · Cited by 1,279
Effectiveness of psychotherapy
Puri, B. K., Ho, R., & Treasden, I. (2011) · CRC Press eBooks · doi:10.1201/b13482-76 · Cited by 1,058
The Working Alliance: Theory, Research, and Practice
Bogart, G. (1995) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1995.49.2.306 · Cited by 994
Predicting the effect of cognitive therapy for depression: A study of unique and common factors.
Castonguay, L. G., Goldfried, M. R., Wiser, S. L., Raue, P. J., & Hayes, A. M. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.3.497 · Cited by 713
Untitled work
(1994) · Medical Entomology and Zoology · Cited by 674
The development of the Working Alliance Inventory.
Horvath, A. O., & Greenberg, L. S. (1986) · Cited by 529
The Working Alliance and The Transference Neurosis
Greenson, R. R. (2008) · The Psychoanalytic Quarterly · doi:10.1002/j.2167-4086.2008.tb00334.x · Cited by 473
Reliability Generalization: Exploring Variance in Measurement Error Affecting Score Reliability Across Studies
Vacha‐Haase, T. (1998) · Educational and Psychological Measurement · doi:10.1177/0013164498058001002 · Cited by 451
The Working Alliance and the Transference Neurosis
Greenson, R. R. (1965) · The Psychoanalytic Quarterly · doi:10.1080/21674086.1965.11926343 · Cited by 432
The concept of the alliance and its role in psychotherapy: Theoretical and empirical considerations.
Gaston, L. (1990) · Psychotherapy · doi:10.1037/0033-3204.27.2.143 · Cited by 408
Psychometrics Is Datametrics: The Test Is Not Reliable
Thompson, B., & Vacha‐Haase, T. (2000) · Educational and Psychological Measurement · doi:10.1177/0013164400602002 · Cited by 357
A Reliability Generalization Study of the Teacher Efficacy Scale and Related Instruments
Henson, R. K., Kogan, L. R., & Vacha‐Haase, T. (2001) · Educational and Psychological Measurement · doi:10.1177/00131640121971284 · Cited by 306
Two Helping Alliance Methods for Predicting Outcomes of Psychotherapy
Luborsky, L. B., Crits-Christoph, P. F., Alexander, L. B., Margolis, M., & COHEN, M. (1983) · The Journal of Nervous and Mental Disease · doi:10.1097/00005053-198308000-00005 · Cited by 297
A comparison of six measures of working alliance.
Tichenor, V., & Hill, C. E. (1989) · Psychotherapy · doi:10.1037/h0085419 · Cited by 295
Working alliance in the early phase of counseling.
Kokotovic, A. M., & Tracey, T. J. G. (1990) · Journal of Counseling Psychology · doi:10.1037/0022-0167.37.1.16 · Cited by 277
Scale construction.
Dawis, R. V. (1987) · Journal of Counseling Psychology · doi:10.1037/0022-0167.34.4.481 · Cited by 233
Toward the validation of the California Therapeutic Alliance Rating System.
Marmar, C. R., Weiss, D. S., & Gaston, L. (1989) · Psychological Assessment · doi:10.1037/1040-3590.1.1.46 · Cited by 214
Research on psychotherapy integration: Recommendations and conclusions from an NIMH workshop.
Wolfe, B. E., & Goldfried, M. R. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.56.3.448 · Cited by 200
Assessing the reliability of Beck Depression Inventory scores: reliability generalization across studies
Yin, P., & Fan, X. (2000) · Educational and Psychological Measurement · doi:10.1177/00131640021970466 · Cited by 197 · cited 1×
“Nevertheless, RG studies have been conducted on a number of wellknown, clinically-oriented psychological tests, including, to name a few, symptom-based tests, such as the Beck Depression Inventory (BDI) ( Yin & Fan, 2000 ), and personality tests, such as the Minnesota Multiphasic Personality Inventory (MMPI/MMPI-2) (V…”
Background
Sample Compositions and Variabilities in Published Studies versus Those in Test Manuals: Validity of Score Reliability Inductions
Vacha‐Haase, T., Kogan, L. R., & Thompson, B. (2000) · Educational and Psychological Measurement · doi:10.1177/00131640021970682 · Cited by 187
Educational and psychological measurement
Silverstein, A. B., Brownlee, L., Hubbell, M., & Mclain, R. E. (1982) · Serials Review · doi:10.1016/0098-7913(82)90053-3 · Cited by 186
Role of the Working Alliance in the Treatment of Delinquent Boys in Community-Based Programs
Florsheim, P., Shotorbani, S., Guest-Warnick, G., Barratt, T., & Hwang, W. (2000) · Journal of Clinical Child Psychology · doi:10.1207/s15374424jccp2901_10 · Cited by 183
Reliability Generalization of the Neo Personality Scales
Caruso, J. (2000) · Educational and Psychological Measurement · doi:10.1177/00131640021970484 · Cited by 179
The factor structure of the working alliance inventory in cognitive-behavioral therapy.
Andrusyna, T. P., Tang, T., DeRubeis, R. J., & Luborsky, L. B. (2001) · PubMed Central · Cited by 174
The working alliance and consumer case management
Solomon, P., Draine, J., & Delaney, M. A. (1995) · The Journal of Behavioral Health Services & Research · doi:10.1007/bf02518753 · Cited by 171
Relevance of the supervisory alliance to the counseling alliance and to treatment adherence in counselor training.
Patton, M. J., & Jr. Kivlighan Dennis, M. (1997) · Journal of Counseling Psychology · doi:10.1037/0022-0167.44.1.108 · Cited by 152
Perspective is everything: the predictive validity of six working alliance instruments.
Fenton, L. R., Cecero, J. J., Nich, C., Frankforter, T. L., & Carroll, K. M. (2001) · PubMed Central · Cited by 151
The therapeutic alliance in psychodynamic-interpersonal and cognitive-behavioral therapy.
Raue, P. J., Goldfried, M. R., & Barkham, M. (1997) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.65.4.582 · Cited by 132
The Therapeutic Bond Scales: Psychometric characteristics and relationship to treatment effectiveness.
Saunders, S. M., Howard, K. I., & Orlinsky, D. E. (1989) · Psychological Assessment · doi:10.1037/1040-3590.1.4.323 · Cited by 129
Research on psychotherapy integration: Recommendations and conclusions from an NIMH workshop.
Wolfe, B. E., & Goldfried, M. R. (1988) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.56.3.448 · Cited by 113
Prediction of outcome of brief psychotherapy from therapist interpretive interventions.
Marziali, E. (1984) · Archives of General Psychiatry · doi:10.1001/archpsyc.1984.01790140091011 · Cited by 108
Attachment Patterns and Working Alliance in Trauma Therapy for Victims of Political Violence
Kanninen, K. M., Salo, J., & Punamäki, R. (2000) · Psychotherapy Research · doi:10.1093/ptr/10.4.435 · Cited by 105
Practices Regarding Reporting of Reliability Coefficients: A Review of Three Journals
Vacha‐Haase, T., Ness, C. M., Nilsson, J. E., & Reetz, D. R. (1999) · The Journal of Experimental Education · doi:10.1080/00220979909598487 · Cited by 97
The therapeutic alliance in psychodynamic-interpersonal and cognitive-behavioral therapy.
Raue, P. J., Goldfried, M. R., & Barkham, M. (1997) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.65.4.582 · Cited by 86
Notes and Comments: The Reliability of Observational Measures
ROWLEY, G. L. (1976) · American Educational Research Journal · doi:10.2307/1162553 · Cited by 80
Trends in reporting psychometric properties of scales used in counseling psychology research.
Meier, S. T., & Davis, S. R. (1990) · Journal of Counseling Psychology · doi:10.1037/0022-0167.37.1.113 · Cited by 78
Client Involvement, Working Alliance, and Type of Therapy Termination
Tryon, G. S., & Kane, A. S. (1995) · Psychotherapy Research · doi:10.1080/10503309512331331306 · Cited by 67
Focus on therapeutic alliance: The psychometric properties of six measures across three treatments.
Cecero, J. J., Fenton, L. R., Frankforter, T. L., Nich, C., & Carroll, K. M. (2001) · Psychotherapy · doi:10.1037/0033-3204.38.1.1 · Cited by 67
The Working Alliance: A Comparison of Two Therapies
Raue, P. J., Castonguay, L. G., & Goldfried, M. R. (1993) · Psychotherapy Research · doi:10.1080/10503309312331333789 · Cited by 64
An exploratory study of the working alliance : its measurement and relationship to therapy outcome
Horvath, A. O. (2010) · cIRcle (University of British Columbia) · doi:10.14288/1.0054327 · Cited by 57
White Counselor Trainees' Racial Identity and Working Alliance Perceptions
Burkard, A. W., Ponterotto, J. G., Reynolds, A. L., & Alfonso, V. C. (1999) · Journal of Counseling & Development · doi:10.1002/j.1556-6676.1999.tb02455.x · Cited by 46
Reliability Generalization: Exploring Variation of Reliability Coefficients of MMPI Clinical Scales Scores
Vacha‐Haase, T., Kogan, L. R., Tani, C. R., & Woodall, R. A. (2001) · Educational and Psychological Measurement · doi:10.1177/00131640121971059 · Cited by 45
Changes in counselor adherence over the course of training.
Multon, K. D., Jr. Kivlighan Dennis, M., & Gold, P. B. (1996) · Journal of Counseling Psychology · doi:10.1037/0022-0167.43.3.356 · Cited by 42
Bias in participant ratings of psychotherapy process: An initial generalizability study.
Hoyt, W. T. (2002) · Journal of Counseling Psychology · doi:10.1037/0022-0167.49.1.35 · Cited by 38
Goal agreement and early therapeutic change.
Long, J. R. (2001) · Psychotherapy · doi:10.1037/0033-3204.38.2.219 · Cited by 32
The process of individual therapy.
Hill, C. E., & Williams, E. N. (2000) · Cited by 31
Counselor Sex Role Orientation, Flexibility, and Working Alliance Formation
Kivlighan, D. M., Clements, L., Blake, C., Arnzen, A., & Brady, L. (1993) · Journal of Counseling & Development · doi:10.1002/j.1556-6676.1993.tb02284.x · Cited by 28
Consumer experience with payeeship provided by a community mental health center
Rosen, M. I., Desai, R., Bailey, M., Davidson, L., & Rosenheck, R. (2001) · Psychiatric Rehabilitation Journal · doi:10.1037/h0095025 · Cited by 25 · cited 1×
“Schmelkin, 1991; Rowley, 1976; Thompson, 1994; Thompson & VachaHaase, 2000; Vacha-Haase, 1998; Wilkinson & American Psychological Association Task Force on Statistical Inference, 1999). To this end, Vacha-Haase (1998) introduced a meta-analytic method that she called RG.”
Clients' Expectations About Counseling and Perceptions of the Working Alliance
Tokar, D. M., Hardin, S. I., Adams, E. M., & Brandel, I. W. (1996) · Journal of College Student Psychotherapy · doi:10.1300/j035v11n02_03 · Cited by 25
Process, life events, and symptomatic change in brief eclectic psychotherapy.
Sexton, H. C. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.6.1358 · Cited by 22
Therapeutic relationship and its association with outcome
Moseley, D. (2010) · cIRcle (University of British Columbia) · doi:10.14288/1.0054293 · Cited by 17
Effect of Rater Orientation on the Evaluation of Therapeutic Alliance
Raue, P. J., Putterman, J., Goldfried, M. R., & Wolitzky, D. L. (1995) · Psychotherapy Research · doi:10.1080/10503309512331331446 · Cited by 11
Development of Nurse-Client Relationships: What Helps?
Forchuk, C. (1995) · Journal of the American Psychiatric Nurses Association · doi:10.1177/107839039500100503 · Cited by 8

Research works printed but not linked: 7

Fenton, L. R., 2001, Journal of Psychotherapy Practice and Research, 10, 262
Hartley, D., 1983, Empirical studies of psychoanalytic theories, 1
Hill, C. E., 2001, Handbookof counseling psychology, 670
10.1037/0022-0167.36.2.223 · DOI · Paper page
Lambert, M. J., 1994, Handbookof psychotherapy and behavior change, 143
10.1037/0022-006X.68.3.438 · DOI · Paper page
Rowley, G. L., 1976, American Educational Research Journal, 13, 51

Sorted by citations

A meta-analytic examination of client–therapist perspectives of the working alliance
Tryon, G. S., Blackwell, S. C., & Hammel, E. F. (2007) · Psychotherapy Research · doi:10.1080/10503300701320611 · Cited by 286
Does Acceptance and Relationship Focused Behavior Therapy Contribute to Bupropion Outcomes? A Randomized Controlled Trial of Functional Analytic Psychotherapy and Acceptance and Commitment Therapy for Smoking Cessation
Gifford, E. V., Kohlenberg, B. S., Hayes, S. C., Pierson, H. M., Piasecki, M. P., Antonuccio, D. O., & Palm, K. M. (2011) · Behavior Therapy · doi:10.1016/j.beth.2011.03.002 · Cited by 200
Asking the right questions in the right way: the need for a shift in research on psychological treatments for addiction
Orford, J. (2008) · Addiction · doi:10.1111/j.1360-0443.2007.02092.x · Cited by 184
Reliability Generalization: Moving Toward Improved Understanding and Use of Score Reliability
Vacha‐Haase, T., Henson, R. K., & Caruso, J. C. (2002) · Educational and Psychological Measurement · doi:10.1177/0013164402062004002 · Cited by 157
Two aspects of the therapeutic alliance: Differential relations with depressive symptom change.
Webb, C. A., DeRubeis, R. J., Amsterdam, J. D., Shelton, R. C., Hollon, S. D., & Dimidjian, S. (2011) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0023252 · Cited by 150

See all 162

Authors: Hanson, W. E., Curry, K. T., & Bandalos, D. L.
Affiliations: University of Nebraska–Lincoln (US)
Corresponding author: Hanson, William E.
ORCID: Bandalos, Deborah L.
Venue: Educational and Psychological Measurement
Publisher: SAGE Publications
Published: 2002-08
Volume: 62
Issue: 4
Pages: 659-673
ISSN: 0013-1644 (ISSN-L)
1552-3888
DOI: 10.1177/0013164402062004008
OpenAlex: W2122193551
Cited by: 161
References: 69
Citations by year: 2026: 3 · 2025: 4 · 2024: 8 · 2023: 4 · 2022: 13 · 2021: 7 · 2020: 6 · 2019: 9 · 2018: 11 · 2017: 9 · 2016: 11 · 2015: 14 · 2014: 11 · 2013: 8 · 2012: 8
FWCI: 2.21
Type: journal article
Language: en
Open access: closed
Licence: journals.sagepub.com/page/policies/text-and-data-mining-license (tdm)
Locations: Educational and Psychological Measurement (publishedVersion)
Topics: Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
Keywords: Working Alliance Inventory, therapeutic relationship, reliability generalization, internal consistency, inter-rater reliability, scale scores, observer ratings

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Reliability Generalization of Working Alliance Inventory Scale Scores

Abstract

Overview

Integrity signals

Verification

Claims the paper makes

Claims other papers make about this one

Citations

Cites

Cited by

Related

Metadata

Record

About these checks