Bias in participant ratings of psychotherapy process: An initial generalizability study.
Abstract
Rater bias has long been considered a source of error in observer ratings but has been ignored by process researchers using participant ratings. In particular, rater variance, or differences in generalized favorable or unfavorable perceptions of others, represents a neglected source of error in studies using participant ratings. The author presents an initial generalizability study of 2 commonly used psychotherapy process measures and discusses the implications for statistical power of studies using these measures and for interpretation of research findings. Results suggest that bias may be a substantial source of error in process ratings, accounting for 4 to 8 times as much variance in observed scores as intrarater error.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Self-reported practices may not necessarily indicate what clinicians actually do.
“Another limitation of this study is that self-reported practices may not necessarily indicate what clinicians actually do (Hoyt, 2002).”
From What do psychotherapists really do in practice? An internet study of over 2,000 practitioners (2010) · cites this paper as Hoyt, 2002
Cited by
Sorted by citations
Baldwin, S. A., Wampold, B. E., & Imel, Z. E. (2007) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.75.6.842 · Cited by 646 · see citing works
Del Re, A. C., Flückiger, C., Horvath, A. O., Symonds, D., & Wampold, B. E. (2012) · Clinical Psychology Review · doi:10.1016/j.cpr.2012.07.002 · Cited by 397 · see citing works
What do psychotherapists really do in practice? An internet study of over 2,000 practitioners
Cook, J. M., Biyanova, T., Elhai, J., Schnurr, P. P., & Coyne, J. C. (2010) · Psychotherapy: Theory, Research, Practice, Training · doi:10.1037/a0019788 · Cited by 168 · see citing works
“Another limitation of this study is that selfreported practices may not necessarily indicate what clinicians actually do (Hoyt, 2002).”
Reliability Generalization of Working Alliance Inventory Scale Scores
Hanson, W. E., Curry, K. T., & Bandalos, D. L. (2002) · Educational and Psychological Measurement · doi:10.1177/0013164402062004008 · Cited by 161 · see citing works
Perceived support and relational effects in psychotherapy process constructs.
Lakey, B., Cohen, J. L., & Neely, L. C. (2008) · Journal of Counseling Psychology · doi:10.1037/0022-0167.55.2.209 · Cited by 23 · see citing works
- Authors
- Hoyt, W. T.
- Affiliations
- University of Wisconsin–Madison (US)
- Corresponding author
- Hoyt, William T.
- ORCID
- Hoyt, William T.
- Venue
- Journal of Counseling Psychology
- Published
- 2002-01
- Volume
- 49
- Issue
- 1
- Pages
- 35-46
- ISSN
- 0022-0167 (ISSN-L)
- 1939-2168
- OpenAlex
- W2039317503
- Cited by
- 38
- References
- 0
- Citations by year
- 2024: 1 · 2023: 1 · 2022: 3 · 2021: 1 · 2020: 1 · 2018: 1 · 2016: 2 · 2015: 3 · 2014: 2 · 2012: 4
- FWCI
- 1.77
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Counseling Psychology (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Reliability and Agreement in Measurement
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Keywords
- rater bias, therapeutic process, observer ratings, measurement error, generalizability, statistical power, process indicators
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
