How consistent are clinical significance classifications when calculation methods and outcome measures differ?
Abstract
Outcome studies report the percentage of clinically significant outcomes; however, the reliability of these classifications is unclear. The current study explored the extent to which inconsistencies arise in classifying patient outcomes using five clinical significance calculation methods and three outcome measures. Adult inpatients (N = 2,676) treated for depression completed the three outcome measures pre- and post-treatment. Their outcomes were classified as recovered, improved, unchanged, or deteriorated using selected clinical significance calculation methods and outcome measures. The choice of outcome measure used in calculating clinical significance had a greater impact on the classification than the choice of clinical significance calculation method. Guidelines for reporting recovery rates are presented to ensure that appropriate conclusions are drawn.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
An exploratory analysis of clinically significant change for the DASS-21 was conducted using an adaptation of the Ronk et al. classification system for indicating recovery (deteriorated, improved, recovered).
“An exploratory analysis of clinically significant change was conducted for the DASS-21 using descriptive, chi-square and odds ratio (OR) statistics and based on an adaptation of the Ronk et al. classification system (64) for indicating recovery (deteriorated, improved, recovered).”
Cites
Lovibond, P. F., & Lovibond, S. H. (1995) · Behaviour Research and Therapy · doi:10.1016/0005-7967(94)00075-u · Cited by 14,523 · see citing works
Rozeboom, W. W., Lord, F. M., Novick, M. R., & Birnbaum, A. (1969) · American Educational Research Journal · doi:10.2307/1162101 · Cited by 8,012 · see citing works
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770 · see citing works
(2010) · Cited by 7,034 · see citing works
(1995) · Cited by 6,909 · see citing works
Validating the SF-36 health survey questionnaire: new outcome measure for primary care.
Brazier, J. E., Harper, R., Jones, N. M., O’Cathain, A., Thomas, K. J., Usherwood, T. P., & Westlake, L. (1992) · BMJ · doi:10.1136/bmj.305.6846.160 · Cited by 4,660 · see citing works
Hierarchical Linear Models: Applications and Data Analysis Methods.
Goldstein, H., Bryk, A. S., & Raudenbush, S. W. (1993) · Journal of the American Statistical Association · doi:10.2307/2290750 · Cited by 3,320 · see citing works
Interpreting scores on the Kessler Psychological Distress Scale (K10)
Andrews, G., & Slade, T. (2001) · Australian and New Zealand Journal of Public Health · doi:10.1111/j.1467-842x.2001.tb00310.x · Cited by 2,021 · see citing works
Regression to the mean: what it is and how to deal with it
Barnett, A. G. (2004) · International Journal of Epidemiology · doi:10.1093/ije/dyh299 · Cited by 1,988 · see citing works
(1993) · PubMed · Cited by 1,900 · see citing works
Clinical versus mechanical prediction: a meta-analysis
Grove, W. M., Zald, D. H., Lebow, B. S., Snitz, B. E., & Nelson, C. (2000) · Psychological Assessment · doi:10.1037/1040-3590.12.1.19 · Cited by 1,640 · see citing works
Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984) · Behavior Therapy · doi:10.1016/s0005-7894(84)80002-7 · Cited by 1,294 · see citing works
Quality-of-Life Impairment in Depressive and Anxiety Disorders
Rapaport, M. H., Clary, C., Fayyad, R., & Endicott, J. (2005) · American Journal of Psychiatry · doi:10.1176/appi.ajp.162.6.1171 · Cited by 914 · see citing works
A lab test and algorithms for identifying clients at risk for treatment failure.
Hannan, C. R., Lambert, M., Harmon, C., Nielsen, S. L., Smart, D., Shimokawa, K., & Sutton, S. (2005) · Journal of Clinical Psychology · doi:10.1002/jclp.20108 · Cited by 550 · see citing works
Lambert, M. J., & Shimokawa, K. (2011) · Psychotherapy · doi:10.1037/a0022238 · Cited by 464 · see citing works
The Determination of the Significance of Change Between Pre- and Posttesting Periods
Linn, R. L., & Slinde, J. A. (1977) · Review of Educational Research · doi:10.3102/00346543047001121 · Cited by 336 · see citing works
Psychometric properties of the Depression Anxiety Stress Scales (DASS) in depressed clinical samples
Page, A. C., Hooke, G. R., & Morrison, D. L. (2007) · British Journal of Clinical Psychology · doi:10.1348/014466506x158996 · Cited by 301 · see citing works
Self-citationClinical Significance Methods: A Comparison of Statistical Techniques
Bauer, S., Lambert, M. J., & Nielsen, S. L. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8201_11 · Cited by 258 · see citing works
Clinically significant change: Jacobson and Truax (1991) revisited.
Speer, D. C. (1992) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.60.3.402 · Cited by 219 · see citing works
Speer, D. C., & Greenbaum, P. E. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.63.6.1044 · Cited by 216 · see citing works
Speer, D. C., & Greenbaum, P. E. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.6.1044 · Cited by 216 · see citing works
Clinically significant change: Jacobson and Truax (1991) revisited.
Speer, D. C. (1992) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.60.3.402 · Cited by 207 · see citing works
Assessing Clinical Significance: Proposed Extensions to Method
Tingey, R., Lambert, M. J., Burlingame, G. M., & Hansen, N. B. (1996) · Psychotherapy Research · doi:10.1080/10503309612331331638 · Cited by 198 · see citing works
Hageman, W. J. J. M., & Arrindell, W. A. (1999) · Behaviour Research and Therapy · doi:10.1016/s0005-7967(99)00032-7 · Cited by 164 · see citing works
Outcome assessment: From conceptualization to implementation.
Lambert, M. J., Okiishi, J. C., Finch, A. E., & Johnson, L. D. (1998) · Professional Psychology Research and Practice · doi:10.1037/0735-7028.29.1.63 · Cited by 129 · see citing works
Oil and Water or Oil and Vinegar? Evidence-Based Medicine Meets Recovery
Davidson, L., Drake, R. E., Schmutte, T. J., Dinzeo, T. J., & Andrés-Hyman, R. C. (2009) · Community Mental Health Journal · doi:10.1007/s10597-009-9228-1 · Cited by 106 · see citing works
Clinical significance of the Outcome Questionnaire (OQ-45.2).
Beckstead, D. J., Hatch, A. L., Lambert, M. J., Eggett, D. L., Goates-Jones, M. K., & Vermeersch, D. A. (2003) · The Behavior Analyst Today · doi:10.1037/h0100015 · Cited by 97 · see citing works
Newnham, E. A., Harwood, K. E., & Page, A. C. (2006) · Journal of Affective Disorders · doi:10.1016/j.jad.2006.07.001 · Cited by 97 · see citing works
Self-citationReliable changes in psychotherapy: Taking into account regression toward the mean.
Hsu, L. M. (1989) · Behavioral Assessment · Cited by 95 · see citing works
Subjective quality of life in severely mentally ill patients: A comparison of two instruments
Ritsner, M., Kurs, R., Kostizky, H., Ponizovsky, A. M., & Modai, I. (2002) · Quality of Life Research · doi:10.1023/a:1016323009671 · Cited by 75 · see citing works
Agoraphobia outcome measurement: A review and meta-analysis.
Ogles, B. M., Lambert, M. J., Weight, D. G., & Payne, I. R. (1990) · Psychological Assessment · doi:10.1037/1040-3590.2.3.317 · Cited by 70 · see citing works
Studies of individual subjects: Logic and methods of analysis
Nunnally, J. C., & Kotsch, W. E. (1983) · British Journal of Clinical Psychology · doi:10.1111/j.2044-8260.1983.tb00582.x · Cited by 49 · see citing works
Hsu, L. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.63.1.141 · Cited by 46 · see citing works
Hsu, L. M. (1995) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.63.1.141 · Cited by 44 · see citing works
Hsu, L. M. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037//0022-006x.67.4.594 · Cited by 38 · see citing works
Hsu, L. (1996) · Journal of Psychopathology and Behavioral Assessment · doi:10.1007/bf02229141 · Cited by 32 · see citing works
Hsu, L. M. (1999) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.67.4.594 · Cited by 30 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.010 · Cited by 279 · see citing works
Research Design in Clinical Psychology
Kazdin, A. E. (2021) · doi:10.1017/9781108993647 · Cited by 240 · see citing works
Assessing clinical significance of treatment outcomes using the DASS-21.
Ronk, F. R., Korman, J. R., Hooke, G. R., & Page, A. C. (2013) · Psychological Assessment · doi:10.1037/a0033100 · Cited by 165 · see citing works
Self-citationFrancis, S. E. B., Shawyer, F., Cayoun, B., Enticott, J., & Meadows, G. N. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2022.815170 · Cited by 45 · see citing works
“classification system (64) for indicating recovery (deteriorated, improved, recovered).”
BackgroundEkeroth, K., & Birgegård, A. (2014) · Psychiatry Research · doi:10.1016/j.psychres.2014.02.008 · Cited by 42 · see citing works
Single-case experimental research designs.
Kazdin, A. E. (2015) · Methodological issues and strategies in clinical research (4th ed.). · doi:10.1037/14805-029 · Cited by 39 · see citing works
Restifo, E., Kashyap, S., Hooke, G. R., & Page, A. C. (2015) · Psychotherapy Research · doi:10.1080/10503307.2015.1006707 · Cited by 33 · see citing works
Self-citationde Beurs, E., Barendregt, M., de Heer, A., van Duijn, E., Goeree, B., Kloos, M., Kooiman, K., Lionarons, H., & Merks, A. (2015) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.1954 · Cited by 29 · see citing works
Ronk, F. R., Hooke, G., & Page, A. C. (2016) · BMC Psychiatry · doi:10.1186/s12888-016-0895-5 · Cited by 26 · see citing works
Self-citationBlais, M. A., Malone, J. C., Stein, M. B., Slavin‐Mulford, J. M., O’Keefe, S. M., Renna, M. E., & Sinclair, S. J. (2013) · Psychotherapy · doi:10.1037/a0031385 · Cited by 25 · see citing works
Seow, L. L. Y., Collins, K. R. L., Page, A. C., & Hooke, G. R. (2021) · Psychotherapy Research · doi:10.1080/10503307.2021.1933240 · Cited by 12 · see citing works
Self-citationHooke, G. R., Sng, A., Cunningham, N. K., & Page, A. C. (2017) · Cognitive Therapy and Research · doi:10.1007/s10608-017-9851-z · Cited by 10 · see citing works
Self-citationEfectos de un Programa de Intervención sobre las habilidades emocionales en niños preescolares
Muchotrigo, M. P. G., & Merino‐Soto, C. (2020) · REOP - Revista Española de Orientación y Psicopedagogía · doi:10.5944/reop.vol.31.num.1.2020.27290 · Cited by 9 · see citing works
Use of outcome measurements in clinical practice: How specific should one be?
Nugter, A., Hermens, M. L., Robbers, S. C. C., van Son, G. E., Theunissen, J., & Engelsbel, F. (2017) · Psychotherapy Research · doi:10.1080/10503307.2017.1408975 · Cited by 9 · see citing works
Mather, S., Fisher, P., Nevitt, S. J., Cherry, M. G., Maturana, C., Warren, J. G., & Noble, A. J. (2022) · Journal of Affective Disorders · doi:10.1016/j.jad.2022.04.132 · Cited by 8 · see citing works
Hooke, G. R., Savani, P., Stewart, B., Araujo, S., & Page, A. C. (2022) · Journal of Clinical Psychology · doi:10.1002/jclp.23426 · Cited by 8 · see citing works
Self-citationBadcock, J., Graham, M., & Paulik, G. (2019) · Clinical Psychology and Psychotherapy · doi:10.1002/cpp.2410 · Cited by 8 · see citing works
Ferrer-Urbina, R., Pardo, A., Arrindell, W., & Puddu-Gallardo, G. (2023) · Frontiers in Psychology · doi:10.3389/fpsyg.2023.1132128 · Cited by 5 · see citing works
The Depression Anxiety Stress Scale: Features and applications
Gillies, J. C. P., & Dozois, D. J. A. (2021) · Elsevier eBooks · doi:10.1016/b978-0-12-817933-8.00025-6 · Cited by 6 · see citing works
Flood, N. K., Page, A. C., & Hooke, G. R. (2018) · Psychotherapy Research · doi:10.1080/10503307.2018.1469803 · Cited by 4 · see citing works
Self-citationFollette, W. C., & Callaghan, G. M. (2015) · The Encyclopedia of Clinical Psychology · doi:10.1002/9781118625392.wbecp574 · Cited by 2 · see citing works
Group Level Clinical Significance: An Analysis of Current Practice
Cribbie, R. A., Arpin‐Cribbie, C. A., Vendittelli, R., & Tucciarone, E. (2014) · Current Psychology · doi:10.1007/s12144-014-9280-z · Cited by 1 · see citing works
Internal and External Validity
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.003 · Cited by 1 · see citing works
Case-Control and Cohort Designs
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.008 · Cited by 1 · see citing works
Null Hypothesis Significance Testing
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.014 · Cited by 1 · see citing works
Temple, J. (2019) · University of Liverpool · doi:10.17638/03033174
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.001
Construct and Data-Evaluation Validity
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.004
Single-Case Experimental Research Designs
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.009
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.007
Cautions, Negative Effects, and Replication
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.016
Selecting Measures for Research
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.011
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.021
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.013
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.022
Ideas that Begin the Research Process
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.005
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.020
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.025
Presenting and Analyzing the Data
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.015
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.024
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.018
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.002
Assessment: Types of Measures and Their Use
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.012
Experimental Research Using Group Designs
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.006
Ethical Issues and Guidelines for Research
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.017
Communication of Research Findings
Kazdin, A. E. (2021) · Cambridge University Press eBooks · doi:10.1017/9781108993647.019
Feedback in the Treatment of Suicidality
Page, A. C., Kyron, M. J., & Hooke, G. R. (2025) · CBT: science into practice · doi:10.1007/978-3-031-90428-8_9
Self-citation
- Authors
- Ronk, F. R., Hooke, G. R., & Page, A. C.
- Affiliations
- The University of Western Australia (AU)
- Venue
- Clinical Psychology Science and Practice
- Published
- 2012-06
- Volume
- 19
- Issue
- 2
- Pages
- 167-179
- ISSN
- 0969-5893 (ISSN-L)
- 1468-2850
- OpenAlex
- W2080834572
- Cited by
- 47
- References
- 35
- Citations by year
- 2025: 1 · 2023: 1 · 2022: 3 · 2021: 27 · 2020: 1 · 2019: 2 · 2018: 1 · 2017: 2 · 2016: 1 · 2015: 4 · 2014: 2 · 2013: 2
- FWCI
- 2.90
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Clinical Psychology Science and Practice (publishedVersion)
- UWA Profiles and Research Repository (UWA) (submittedVersion)
- Topics
- Social Sciences › Psychology › Clinical Psychology › Psychotherapy Techniques and Applications
- Social Sciences › Decision Sciences › Management Science and Operations Research › Psychometric Methodologies and Testing
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Keywords
- clinical significance, outcome classification, depression, inpatient care, recovery rate, outcome measures, classification consistency, reporting guidelines, treatment outcomes
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review