An administratively created reality: Some problems with the use of self-report questionnaire measures of adjustment in counselling/psychotherapy outcome research
Abstract
Outcome research in counselling and psychotherapy has come to rely heavily on the use of self‐report questionnaire measures of anxiety, depression and general symptomatology. The validity of these measures in the context of psychotherapy research has, in general, been taken for granted. This paper proposes that, despite the success of self‐report questionnaires in personality research and personnel selection, there are significant problems involved in the use of these methods in research into therapeutic change. Specifically, three areas of difficulty are reviewed. First, questionnaires are completed in a social setting that itself changes as a result of therapy. Second, the experience of psychotherapy changes the way that clients make sense of questionnaire items (the ‘response shift’ phenomenon). Third, the conceptualisation of the person implicit in questionnaire design is not compatible with the conceptualisation of persons espoused by most contemporary therapeutic approaches. It is argued that the continuing popularity of self‐report questionnaires can be understood in terms of Kurt Danziger's model of the social construction of psychological knowledge.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Reliance on self-report questionnaires as a method of data collection can be affected by bias.
“Another limitation of this study is the reliance on self-report questionnaires as a method of data collection, which can be affected by bias [93].”
From Psychological interventions for generalized anxiety disorder: Effects and predictors in a naturalistic outpatient setting (2023) · cites this paper as [93]
Cites
The Discovery of Grounded Theory: Strategies for Qualitative Research
Glaser, B. G., & Strauss, A. L. (2017) · Aldine (Routledge reprint) · doi:10.4324/9780203793206 · Cited by 25,789 · see citing works
Convergent and discriminant validation by the multitrait-multimethod matrix.
Campbell, D. T., & Fiske, D. W. (1959) · Psychological Bulletin · doi:10.1037/h0046016 · Cited by 17,116 · see citing works
The Discovery of Grounded Theory
Glaser, B. G., & Strauss, A. L. (2017) · Routledge eBooks · doi:10.4324/9780203793206-1 · Cited by 8,342 · see citing works
Madison, P., & Mischel, W. (1968) · The American Journal of Psychology · doi:10.2307/1421067 · Cited by 4,395 · see citing works
Orne, M. T. (1962) · American Psychologist · doi:10.1037/h0043424 · Cited by 4,041 · see citing works
Handbook of Psychotherapy and Behavior Change
Garfield, S. L., Bergin, A. E., & Dryden, W. (1987) · Journal of Cognitive Psychotherapy · doi:10.1891/0889-8391.1.4.264a · Cited by 3,998 · see citing works
Handbook of Psychotherapy and Behavior Change
Sanford, E. F. (1988) · American Journal of Psychotherapy · doi:10.1176/appi.psychotherapy.1988.42.1.153 · Cited by 2,581 · see citing works
Danziger, K. (1990) · Cambridge University Press eBooks · doi:10.1017/cbo9780511524059 · Cited by 1,752 · see citing works
The effectiveness of psychotherapy: the Consumer Reports study
Seligman, M. E. P. (1995) · American Psychologist · doi:10.1037/0003-066x.50.12.965 · Cited by 1,608 · see citing works
The effectiveness of psychotherapy: The Consumer Reports study.
Seligman, M. E. P. (1995) · American Psychologist · doi:10.1037//0003-066x.50.12.965 · Cited by 1,591 · see citing works
The Social Desirability Variable in Personality Assessment and Research.
Sprott, D. A., & Edwards, A. L. (1959) · Journal of the American Statistical Association · doi:10.2307/2282171 · Cited by 1,500 · see citing works
The Social Desirability Variable in Personality Assessment and Research
Barry, R. M., & Edwards, A. L. (1958) · The American Catholic Sociological Review · doi:10.2307/3709409 · Cited by 1,262 · see citing works
Personality, Modernity, and the Storied Self: A Contemporary Framework for Studying Persons
McAdams, D. P. (1996) · Psychological Inquiry · doi:10.1207/s15327965pli0704_1 · Cited by 1,210 · see citing works
Qualitative Research in Counselling and Psychotherapy
McLeod, J. (2001) · doi:10.4135/9781849209663 · Cited by 1,166 · see citing works
Self-citationNaming the Mind: How Psychology Found its Language
Danziger, K. (1997) · SAGE Publications Ltd eBooks · doi:10.4135/9781446221815 · Cited by 910 · see citing works
The illusion of mental health.
Shedler, J., Mayman, M., & Manis, M. (1993) · American Psychologist · doi:10.1037//0003-066x.48.11.1117 · Cited by 806 · see citing works
Social desirability scales: More substance than style.
McCrae, R. R., & Costa, P. T. (1983) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.51.6.882 · Cited by 735 · see citing works
Measuring Change and Persistence in Human Affairs: Types of Change Generated by OD Designs
Golembiewski, R. T., Billingsley, K. R., & Yeager, S. J. (1976) · The Journal of Applied Behavioral Science · doi:10.1177/002188637601200201 · Cited by 659 · see citing works
The illusion of mental health.
Shedler, J., Mayman, M., & Manis, M. (1993) · American Psychologist · doi:10.1037/0003-066x.48.11.1117 · Cited by 563 · see citing works
Cognition and Communication: Judgmental Biases, Research Methods, and the Logic of Conversation
Schwarz, N. (2014) · Medical Entomology and Zoology · Cited by 537 · see citing works
Self-Deception and Impression Management in Test Responses
Paulhus, D. L. (1986) · Springer eBooks · doi:10.1007/978-3-642-70751-3_8 · Cited by 468 · see citing works
Howard, G. S., Ralph, K., Gulanick, N. A., Maxwell, S. E., Nance, D. W., & Gerber, S. K. (1979) · Applied Psychological Measurement · doi:10.1177/014662167900300101 · Cited by 462 · see citing works
Adaptation to changing health: Response shift in quality-of-life research.
(2000) · American Psychological Association eBooks · doi:10.1037/10382-000 · Cited by 301 · see citing works
How relevant is an individual difference theory of personality?1
Shweder, R. A. (1975) · Journal of Personality · doi:10.1111/j.1467-6494.1975.tb00716.x · Cited by 244 · see citing works
Self-Identification: Toward an Integration of the Private and Public Self
Schlenker, B. R. (1986) · Springer eBooks · doi:10.1007/978-1-4613-9564-5_2 · Cited by 227 · see citing works
Asking questions and influencing answers.
Clark, H. H., & Schober, M. F. (1992) · Cited by 210 · see citing works
Abuse of the drug metaphor in psychotherapy process-outcome research
Stiles, W. B., & Shapiro, D. A. (1989) · Clinical Psychology Review · doi:10.1016/0272-7358(89)90007-x · Cited by 202 · see citing works
Thurstone, L. L., & Thurstone, T. G. (1930) · The Journal of Social Psychology · doi:10.1080/00224545.1930.9714128 · Cited by 170 · see citing works
Henry, W. P. (1998) · Psychotherapy Research · doi:10.1093/ptr/8.2.126 · Cited by 125 · see citing works
Helping construct desirable identities: A self-presentational view of psychotherapy.
Kelly, A. E. (2000) · Psychological Bulletin · doi:10.1037/0033-2909.126.4.475 · Cited by 99 · see citing works
(1994) · Cited by 98 · see citing works
A review of practices of psychotherapy outcome measurement
Froyd, J. E. (1996) · Journal of Mental Health · doi:10.1080/09638239650037144 · Cited by 94 · see citing works
Method factors in changes associated with psychotherapy.
Cartwright, D. S., Kirtner, W. L., & Fiske, D. W. (1963) · Journal of Abnormal & Social Psychology · doi:10.1037/h0049388 · Cited by 70 · see citing works
Memory and mismemory for health events.
Loftus, E. F., Smith, K. D., Klinger, M. R., & Fiedler, J. (1992) · Cited by 66 · see citing works
Learning from clients : interpersonal helping as viewed by clients and social workers
Maluccio, A. N. (1979) · Free Press eBooks · Cited by 64 · see citing works
Laird, D. A. (1925) · Journal of Abnormal & Social Psychology · doi:10.1037/h0072769 · Cited by 48 · see citing works
The Retrospective Pretest and the Role of Pretest Information in Evaluative Studies
Goedhart, H., & Hoogstraten, J. (1992) · Psychological Reports · doi:10.2466/pr0.1992.70.3.699 · Cited by 46 · see citing works
Āllport, G. (1921) · The Emotional Mind · doi:10.1037/h0066265 · Cited by 39 · see citing works
The ecology of psychotherapy research.
Anderson, T., & Strupp, H. H. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.64.4.776 · Cited by 36 · see citing works
The ecology of psychotherapy research.
Anderson, T., & Strupp, H. H. (1996) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.64.4.776 · Cited by 30 · see citing works
The contribution of qualitative research to evidence-based counselling and psychotherapy
(2013) · Routledge eBooks · doi:10.4324/9780203131602-16 · Cited by 18 · see citing works
The Alpha, Beta, Gamma Change Typology
Golembiewski, R. T. (1989) · Group & Organization Studies · doi:10.1177/105960118901400204 · Cited by 16 · see citing works
Client self-presentations at intake.
Kelly, A. E., Kahn, J. H., & Coulter, R. G. (1996) · Journal of Counseling Psychology · doi:10.1037/0022-0167.43.3.300 · Cited by 15 · see citing works
The Impact of Research on the Practice of Psychotherapy
Firth, J., Shapiro, D., & Parry, G. (1986) · British Journal of Psychotherapy · doi:10.1111/j.1752-0118.1986.tb00938.x · Cited by 13 · see citing works
The response-shift bias in a counsellor education programme
Manthei, R. J. (1997) · British Journal of Guidance and Counselling · doi:10.1080/03069889708253804 · Cited by 11 · see citing works
Whither or Wither Personality Research
Holzman, P. S., & Kagan, J. (2014) · Psychology Press eBooks · doi:10.4324/9781315806815-1 · Cited by 5 · see citing works
Kuncel, R. B., & Kuncel, N. R. (2014) · Psychology Press eBooks · doi:10.4324/9781315806815-12 · Cited by 5 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Krzikalla, C., Morina, N., Andor, T., Nohr, L., & Buhlmann, U. (2023) · PLoS ONE · doi:10.1371/journal.pone.0282902 · Cited by 3 · see citing works
“Another limitation of this study is the reliance on self-report questionnaires as a method of data collection, which can be affected by bias [93].”
Background
- Authors
- McLeod, J.
- Affiliations
- Abertay University (GB)
- Corresponding author
- McLeod, John
- Venue
- Counselling and Psychotherapy Research
- Publisher
- Wiley
- Published
- 2001-12
- Volume
- 1
- Issue
- 3
- Pages
- 215-226
- ISSN
- 1473-3145 (ISSN-L)
- 1746-1405
- OpenAlex
- W2088783638
- Cited by
- 48
- References
- 47
- Citations by year
- 2026: 1 · 2025: 1 · 2024: 7 · 2023: 4 · 2021: 8 · 2020: 2 · 2019: 2 · 2018: 3 · 2017: 3 · 2016: 4 · 2015: 2 · 2014: 2 · 2013: 1 · 2012: 1
- FWCI
- 0.00
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Counselling and Psychotherapy Research (publishedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- self-report questionnaire, psychotherapy outcome research, response shift, questionnaire validity, therapeutic change, client experience of therapy
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
