The DOI matches its Crossref record; 9 of the 9 references it cites were found in a registry or an index.
The contribution of reliable and clinically significant change methods to evidence-based mental health
Open accessAbstract
Where outcomes are unequivocal (life or death; being able to walk v being paralysed) clinicians, researchers, and patients find it easy to speak the same language in evaluating results. However, in much of mental health work initial states and outcomes of treatments are measured on continuous scales and the distribution of the “normal” often overlaps with the range of the “abnormal.” In this situation, clinicians and researchers often talk different languages about change data, and both are probably poor at conveying their thoughts to patients. Researchers traditionally compare means between groups. Their statistical methods, using distributions of the scores before and after treatment to suggest whether change is a sampling artefact or a chance finding, have been known for many years.1 By contrast, clinicians are more often concerned with changes in particular individuals they are treating and often dichotomise outcome as “success” or “failure.” The number needed to treat (NNT) method of presenting results has gone some way to bridge this gap but often uses arbitrary criteria on which to dichotomise change into “success” and “failure.” A typical example is the criterion of a 50% drop on the Hamilton Depression Rating Scale score. A method bridging these approaches would assist the translation of research results into clinical practice. Jacobson et al proposed a method of determining reliable and clinically significant change (RCSC) that summarises changes at the level of the individual in the context of observed changes for the whole sample.2, 3–5 Their methods are applicable, in one form or another, to the measurement of change on any continuous scale for any clinical problem, although they have been reported primarily in the psychotherapy research literature. The broad concept of reliable and clinically significant change rests on 2 questions being addressed at the level of each …
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Jacobson, N. S., & Truax, P. (1991) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.59.1.12 · Cited by 7,770
Jacobson, N. S., Follette, W. C., & Revenstorf, D. (1984) · Behavior Therapy · doi:10.1016/s0005-7894(84)80002-7 · Cited by 1,294
A method of assessing change in a single subject: An alteration of the RC index
Christensen, L., & Mendoza, J. L. (1986) · Behavior Therapy · doi:10.1016/s0005-7894(86)80060-0 · Cited by 508
Kwek, L. C., & Oh, C. H. (1998) · The European Physical Journal C · doi:10.1007/s100520050261 · Cited by 324
BARKHAM CHRIS EVANS FRANK MARGISON, M. (1998) · Journal of Mental Health · doi:10.1080/09638239818328 · Cited by 202
Assessing Clinical Significance: Proposed Extensions to Method
Tingey, R., Lambert, M., Burlingame, G., & Hansen, N. (1996) · Psychotherapy Research · doi:10.1080/10503309612331331638 · Cited by 197
Hageman, W. J., & Arrindell, W. (1993) · Behaviour Research and Therapy · doi:10.1016/0005-7967(93)90122-b · Cited by 92
Reliable changes in psychotherapy: Taking into account regression toward the mean.
Hsu, L. M. (1989) · Behavioral Assessment · Cited by 91
Some Comments on “Assessing Clinical Significance”
Martinovich, Z., Saunders, S. M., & Howard, K. I. (1996) · Psychotherapy Research · doi:10.1080/10503309612331331648 · Cited by 83
Research works printed but not linked: 2
- 1988, Behavioural Assessment, 10, 133
- 1989, Behavioral Assessment, 11, 459
Cited by
Towards a standardised brief outcome measure: Psychometric properties and utility of the CORE–OM
Evans, C., Connell, J., Barkham, M., Margison, F., McGrath, G., Mellor-Clark, J., & Audin, K. (2002) · British Journal of Psychiatry · doi:10.1192/bjp.180.1.51 · Cited by 934
Self-citationBiegel, G. M., Brown, K. W., Shapiro, S. L., & Schubert, C. M. (2009) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0016241 · Cited by 656
CORE: Clinical Outcomes in Routine Evaluation
Evans, C., Mellor-Clark, J., Margison, F., Barkham, M., Audin, K., Connell, J., & McGrath, G. (2000) · Journal of Mental Health · doi:10.1080/713680250 · Cited by 560
Self-citationBarkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., Benson, L., Connell, J., Audin, K., & McGrath, G. (2001) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.69.2.184 · Cited by 532
Self-citationBorckardt, J. J., Nash, M. R., Murphy, M. D., Moore, M., Shaw, D., & O'Neil, P. (2008) · American Psychologist · doi:10.1037/0003-066x.63.2.77 · Cited by 407
Whittingham, K., Sofronoff, K., Sheffield, J., & Sanders, M. R. (2009) · Journal of Abnormal Child Psychology · doi:10.1007/s10802-008-9285-x · Cited by 357
Measurement and psychotherapy: Evidence-based practice and practice-based evidence
Margison, F. R., Barkham, M., Evans, C., McGrath, G., Clark, J. M., Audin, K., & Connell, J. (2000) · British Journal of Psychiatry · doi:10.1192/bjp.177.2.123 · Cited by 326
Self-citationFISHER, P. L., & DURHAM, R. C. (1999) · Psychological Medicine · doi:10.1017/s0033291799001336 · Cited by 324
van Aalderen, J. R., Donders, A. R. T., Giommi, F., Spinhoven, P., Barendregt, H. P., & Speckens, A. E. M. (2012) · Psychological Medicine · doi:10.1017/s0033291711002054 · Cited by 262
Defining successful treatment outcome in depression using the PHQ-9: A comparison of methods
McMillan, D., Gilbody, S., & Richards, D. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2010.04.030 · Cited by 247
Behavioural activation delivered by the non-specialist: phase II randomised controlled trial
Ekers, D., Richards, D., McMillan, D., Bland, J. M., & Gilbody, S. (2011) · British Journal of Psychiatry · doi:10.1192/bjp.bp.110.079111 · Cited by 217
Early sudden gains in psychotherapy under routine clinic conditions: Practice-based evidence.
Stiles, W. B., Leach, C., Barkham, M., Lucock, M., Iveson, S., Shapiro, D. A., Iveson, M., & Hardy, G. E. (2003) · Journal of Consulting and Clinical Psychology · doi:10.1037/0022-006x.71.1.14 · Cited by 220
Self-citationKarterud, S., Pedersen, G., Bjordal, E., Brabrand, J., Friis, S., Haaseth, Ø., Haavaldsen, G., Irion, T., Leirvåg, H., Tørum, E., et al. (2003) · Journal of Personality Disorders · doi:10.1521/pedi.17.3.243.22151 · Cited by 178
McCracken, L. M., MacKichan, F., & Eccleston, C. (2007) · European Journal of Pain · doi:10.1016/j.ejpain.2006.05.004 · Cited by 167
van den Berg, D. P. G., de Bont, P. A. J. M., van der Vleugel, B. M., de Roos, C., de Jongh, A., van Minnen, A., & van der Gaag, M. (2016) · Schizophrenia Bulletin · doi:10.1093/schbul/sbv172 · Cited by 153
Rozental, A., Magnusson, K., Boettcher, J., Andersson, G., & Carlbring, P. (2017) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000158 · Cited by 137
Magiati, I., Moss, J., Charman, T., & Howlin, P. (2011) · Research in Autism Spectrum Disorders · doi:10.1016/j.rasd.2010.11.007 · Cited by 108
Evans, C., Connell, J., Barkham, M., Marshall, C., & Mellor‐Clark, J. (2003) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.384 · Cited by 107
Self-citationInternet-based cognitive—behavior therapy for procrastination: A randomized controlled trial.
Rozental, A., Forsell, E., Svensson, A., Andersson, G., & Carlbring, P. (2015) · Journal of Consulting and Clinical Psychology · doi:10.1037/ccp0000023 · Cited by 103
Preliminary Evaluation of the Engagement and Effectiveness of a Mental Health Chatbot
Daley, K., Hungerbuehler, I., Cavanagh, K., Claro, H. G., Swinton, P. A., & Kapps, M. (2020) · Frontiers in Digital Health · doi:10.3389/fdgth.2020.576361 · Cited by 101
Sarkadi, A., Ådahl, K., Stenvall, E., Ssegonja, R., Batti, H., Gavra, P., Fängström, K., & Salari, R. (2018) · European Child & Adolescent Psychiatry · doi:10.1007/s00787-017-1093-9 · Cited by 94
Sofronoff, K., Jahnel, D., & Sanders, M. (2011) · Research in Developmental Disabilities · doi:10.1016/j.ridd.2011.07.046 · Cited by 92
(2003) · PubMed Central · Cited by 84
Pybis, J., Saxon, D., Hill, A., & Barkham, M. (2017) · BMC Psychiatry · doi:10.1186/s12888-017-1370-7 · Cited by 83
Self-citationThe MYRIAD team, Baer, R., Crane, C., Montero-Marin, J., Phillips, A., Taylor, L., Tickell, A., & Kuyken, W. (2021) · Mindfulness · doi:10.1007/s12671-020-01547-8 · Cited by 78
Deterioration rates in Virtual Reality Therapy: An individual patient data level meta-analysis
Fernández-Álvarez, J., Rozental, A., Carlbring, P., Colombo, D., Riva, G., Anderson, P. L., Baños, R. M., Benbow, A. A., Bouchard, S., Bretón-López, J. M., et al. (2019) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2018.06.005 · Cited by 79
Rozental, A., Shafran, R., Wade, T., Egan, S., Nordgren, L. B., Carlbring, P., Landström, A., Roos, S., Skoglund, M., Thelander, E., et al. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.05.015 · Cited by 76
Rozental, A., Andersson, G., & Carlbring, P. (2019) · Frontiers in Psychology · doi:10.3389/fpsyg.2019.00589 · Cited by 74
Andersson, G., Carlbring, P., & Rozental, A. (2019) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2019.00749 · Cited by 71
“In the formula, SD1 corresponds to the standard deviation of a condition at pre-treatment, and r is the reliability estimate (38).”
BackgroundWuthrich, V. M., & Rapee, R. M. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.09.002 · Cited by 71
DePrince, A. P., Labus, J., Belknap, J., Buckingham, S., & Gover, A. (2012) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0027224 · Cited by 68
Gaigg, S. B., Flaxman, P. E., McLaven, G., Shah, R., Bowler, D. M., Meyer, B., Roestorf, A., Haenschel, C., Rodgers, J., & South, M. (2020) · Autism · doi:10.1177/1362361320909184 · Cited by 80
“…aim of examining longitudinal changes in primary and secondary outcome measures through analyses of variance (ANOVAs) and the calculation of indices of reliable change (RC) and clinically significant change (CSC) at an individual participant level (Evans et al., 1998; Jacobson & Truax, 1991).”
MethodAshworth, M., Evans, C., & Clement, S. (2009) · Journal of Mental Health · doi:10.1080/09638230701879144 · Cited by 63
Self-citationShah, R., Hunt, J., Webb, T. L., & Thompson, A. R. (2014) · British Journal of Dermatology · doi:10.1111/bjd.12990 · Cited by 59
Efficacy of hypnosis in adults undergoing surgical procedures: a meta-analytic update
Holler, M., Koranyi, S., Strauss, B., & Rosendahl, J. (2021) · Clinical Psychology Review · doi:10.1016/j.cpr.2021.102001 · Cited by 53
Keen, N., Hunter, E. C. M., & Peters, E. (2017) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2017.00092 · Cited by 53
Personalised computerised feedback in e-mental health
Musiat, P., Hoffmann, L., & Schmidt, U. (2012) · Journal of Mental Health · doi:10.3109/09638237.2011.648347 · Cited by 47
Cognitive Analytic Therapy for Bipolar Disorder: A Pilot Randomized Controlled Trial
Evans, M., Kellett, S., Heyland, S., Hall, J., & Majid, S. (2017) · Clinical Psychology & Psychotherapy · doi:10.1002/cpp.2065 · Cited by 48
Rozental, A., Forsström, D., Lindner, P., Nilsson, S., Mårtensson, L., Rizzo, A., Andersson, G., & Carlbring, P. (2018) · Behavior Therapy · doi:10.1016/j.beth.2017.08.002 · Cited by 47
Turner, J. C., Gray, D. L., Anderman, L. H., Dawson, H. S., & Anderman, E. M. (2013) · Contemporary Educational Psychology · doi:10.1016/j.cedpsych.2013.06.003 · Cited by 46
Lemmens, L. H., Arntz, A., Peeters, F. P., Hollon, S. D., Roefs, A., & Huibers, M. J. (2011) · Trials · doi:10.1186/1745-6215-12-150 · Cited by 37
“Our calculations will be based on the method of Jacobson and Truax [130] which prescribes that Clinical Improvement (CI) is based on both Reliable Change (RC), the extent to which the pre-to-post-difference score is reliable; and on Clinical Significant change (CSC), the extent to which post-treatment scores are clini…”
MethodRichards, P., & Simpson, S. (2015) · British Journal of Guidance & Counselling · doi:10.1080/03069885.2014.936824 · Cited by 32
Kothari, R., Barker, C., Pistrang, N., Rozental, A., Egan, S., Wade, T., Allcott-Watson, H., Andersson, G., & Shafran, R. (2019) · Journal of Behavior Therapy and Experimental Psychiatry · doi:10.1016/j.jbtep.2019.03.007 · Cited by 32
Hallberg, J., Kaldo, V., Arver, S., Dhejne, C., Piwowar, M., Jokinen, J., & Öberg, K. G. (2020) · The Journal of Sexual Medicine · doi:10.1016/j.jsxm.2020.07.018 · Cited by 28
Victor, M., Lau, B., & Ruud, T. (2018) · BMC Public Health · doi:10.1186/s12889-017-4581-4 · Cited by 27
Brown, F. L., Whittingham, K., McKinlay, L., Boyd, R., & Sofronoff, K. (2013) · Brain Impairment · doi:10.1017/brimp.2013.18 · Cited by 26
Marshall, J. M., Dunstan, D. A., & Bartik, W. (2020) · JMIR Research Protocols · doi:10.2196/17159 · Cited by 24
Self-compassion in Acceptance and Commitment Therapy for chronic pain: A pilot study
Carvalho, S. A., Gillanders, D., Forte, T., Trindade, I. A., Pinto-Gouveia, J., Lapa, T., Valentim, A., Santos, E., Paciência, J., Guiomar, R., et al. (2022) · Scandinavian Journal of Pain · doi:10.1515/sjpain-2021-0214 · Cited by 21
Burns, P., Kellett, S., & Donohoe, G. (2016) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465815000491 · Cited by 20
Pahnke, J., Jansson-Fröjmark, M., Andersson, G., Bjureberg, J., Jokinen, J., Bohman, B., & Lundgren, T. (2023) · Autism · doi:10.1177/13623613221140749 · Cited by 19
“Clinically significant changes (Evans et al., 1998; Jacobson & Truax, 1991) in the primary outcome measures were calculated using normal population data of the PSS ( M = 24.8, SD = 11.1) (Eklund et al., 2014) and the SWLS ( M = 24.1, SD = 6.9) (Pavot & Diener, 2008) along with clinical data from…”
InfluentialBaas, M. A. M., van Pampus, M. G., Stramrood, C. A. I., Dijksman, L. M., Vanhommerig, J. W., & de Jongh, A. (2022) · Frontiers in Psychiatry · doi:10.3389/fpsyt.2021.798249 · Cited by 18
Farrand, P., Confue, P., Byng, R., & Shaw, S. (2009) · Health & Social Care in the Community · doi:10.1111/j.1365-2524.2008.00792.x · Cited by 15
How much is enough? Considering minimally important change in youth mental health outcomes
Krause, K. R., Hetrick, S. E., Courtney, D. B., Cost, K. T., Butcher, N. J., Offringa, M., Monga, S., Henderson, J., & Szatmari, P. (2022) · The Lancet Psychiatry · doi:10.1016/s2215-0366(22)00338-8 · Cited by 15
O'Dell, S. M., Hosterman, S. J., Hostutler, C. A., Nordberg, C., Klinger, T., Petts, R. A., Whitehead, M. R., Faulkner, S. A., Garcia, A., & Massura, C. A. (2020) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2020.03.009 · Cited by 15
Leigh, E., Creswell, C., Stallard, P., Waite, P., Violato, M., Pearcey, S., Brooks, E., Taylor, L., Warnock-Parkes, E., & Clark, D. M. (2021) · Behavioural and Cognitive Psychotherapy · doi:10.1017/s1352465821000035 · Cited by 11
An exploration of group cognitive analytic therapy for anxiety and depression
Martin, E., Byrne, G., Connon, G., & Power, L. (2021) · Psychology and Psychotherapy: Theory, Research and Practice · doi:10.1111/papt.12299 · Cited by 11
An evaluation of large group cognitive behaviour therapy with mindfulness (CBTm) classes
Thakur, V. K., Wong, J. Y., Randall, J. R., Bolton, J. M., Parikh, S. V., Mota, N., Whitney, D., Palay, J., Kinley, J., Diocee, S., et al. (2019) · BMC Psychiatry · doi:10.1186/s12888-019-2124-5 · Cited by 7
Stockton, D., Kellett, S., Wilkinson, N., Hague, J., Bliss, P., Heaver, M., & Devine, A. (2021) · International Journal of Cognitive Therapy · doi:10.1007/s41811-021-00118-6 · Cited by 5
Johns, E. K., Kokokyi, S., Neufeld, D., Krysanski, V., & Meek, B. P. (2024) · Journal of Contextual Behavioral Science · doi:10.1016/j.jcbs.2024.100812 · Cited by 2
Ooi, J., & Steverson, T. (2023) · The Cognitive Behaviour Therapist · doi:10.1017/s1754470x23000260 · Cited by 2
“(2021) (i.e. .91 and 20.79, respectively), we calculated a reliable change index (RCI) of 17 (95% CI) and for the CompACT total score (Evans et al ., 1998; Jacobson and Truax, 1991).”
MethodInfluential(2016) · Research Online (University of Wollongong) · Cited by 2
Metadata
- Authors
- Evans, C., Margison, F., & Barkham, M.
- Venue
- Evidence Based Mental Health
- Publisher
- BMJ
- Published
- 1998-08
- Volume
- 1
- Issue
- 3
- Pages
- 70-72
- ISSN
- 1362-0347 (ISSN-L)
- 1468-960X
- OpenAlex
- W2037131080
- Cited by
- 470
- References
- 9
- FWCI
- 5.94
- Type
- journal article
- Language
- en
- Open access
- bronze
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.