Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change
Open accessPublished version of a preprint
The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 8 of the 32 sources its claims cite were checked for support.
Abstract
Background: Accurate measurement of treatment-related change is a key part of psychotherapy research and the investigation of treatment efficacy. For this reason, the ability to measure change with accurate and valid methods is critical for psychotherapy. Objective: The aims of this study were to (1) explore the underlying characteristics of depressive symptom change, measured with the nine-item Patient Health Questionnaire (PHQ-9), following psychotherapy, and (2) compare the suitability of different ways to measure and interpret symptom change. A treatment sample of Web-based psychotherapy participants (n=1098) and a waitlist sample (n=96) were used to (1) explore the statistical characteristics of depressive symptom change, and (2) compare the suitability of two common types of change functions: linear and proportional change. Methods: These objectives were explored using hypotheses that tested (1) the relationship between baseline symptoms and the rate of change, (2) the shape of symptom score distribution following treatment, and (3) measurement error associated with linear and proportional measurement models. Results: Findings demonstrated that (1) individuals with severe depressive baseline symptoms had greater reductions in symptom scores than individuals with mild baseline symptoms (11.4 vs 3.7); however, as a percentage measurement, change remained similar across individuals with mild, moderate, or severe baseline symptoms (50%-55%); (2) positive skewness was observed in PHQ-9 score distributions following treatment; and (3) models that measured symptom change as a proportional function resulted in greater model fit and reduced measurement error (<30%). Conclusions: This study suggests that symptom scales, sharing an implicit feature of score bounding, are associated with a proportional function of change. Selecting statistics that overlook this proportional change (eg, Cohen d) is problematic and leads to (1) artificially increased estimates of change with higher baseline symptoms, (2) increased measurement error, and (3) confounded estimates of treatment efficacy and clinical change. Implications, limitations, and idiosyncrasies from these results are discussed.
Overview
CoChat Summary
TL;DR
This study argues that PHQ-9 symptom change after web-based psychotherapy is better characterized as proportional rather than linear: participants with more severe baseline symptoms changed more in raw points, but changed similarly in percentage terms. Across a treatment sample of 1,098 and a waitlist sample of 96, proportional models fit better and reduced measurement error.
Key findings
- In the treatment sample, participants with severe baseline symptoms reduced PHQ-9 scores more than those with mild symptoms in raw points: 11.4 vs 3.7; yet proportional change was similar across baseline-severity bands at 50%–55%.
- In the waitlist condition, symptom change also depended on baseline severity; severe participants showed 35% proportional improvement, and severe waitlist participants reduced more than moderate treatment participants (7 points vs 6 points).
- PHQ-9 posttreatment distributions showed positive skewness, which increased after treatment.
- Models assuming proportional change had better fit and lower error: the paper reports reduced QIC, AIC, and BIC and a drop in PHQ-9 error variance from σ²=24.122 to σ²=16.716, described as more than 30% less measurement error.
Methods
- Combined clinical data from three published randomized controlled trials of internet-delivered cognitive behavioral therapy (ICBT) conducted in the same clinic; total participants N=1262, with 1098 assessed at both pretreatment and posttreatment. A waitlist sample included n=96.
- Primary outcome was the PHQ-9 (scores 0–27); participants were adults 18–65 years with at least mild depression or anxiety symptoms (minimum ≥5 on PHQ-9 or GAD-7).
- Tested three hypotheses using GEE models, multilevel/mixed models, and raw means, then compared linear vs proportional change models using QIC, AIC, BIC, and residual error/measurement error.
Limitations
- The authors state the findings are preliminary, specific to the PHQ-9, and specific to one treatment model (the Macquarie University online model).
- The paper says it is unclear how far the 50%–55% change generalizes to other symptom scales or treatment models.
- Evident from the text: the sample comes from self-selecting Australian adults in web-based CBT trials, so generalizability beyond this setting is limited.
Model-written summary. Check it against the paper before relying on it.
Integrity signals
Registry
Cites
Statistics
Figures





Verification
Claims the paper makes
5 Valid · 0 Not valid · 12 Not checked · 20 Own results and interpretations
The sources with a verdict support this claim; the rest weren’t checked yet, have no full text we could read, or their verdicts aren’t rated yet.
In the paper
“common symptom scales, such as the PHQ-9 [29], the Generalized Anxiety Disorder seven-item scale (GAD-7) [30], and prominent others (eg, Kessler Psychological Distress scale) [31], often demonstrate an implicit design feature of score bounding at minimal symptoms.”
§ Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change > Introduction
What the sources say
- Supportedcites [29] source
The METHODS section states that 'the PHQ-9 score can range from 0 to 27, since each of the 9 items can be scored from 0 (not at all) to 3 (nearly every day)', i.e. the scale is anchored/bounded at the minimal-symmetry end, and the Abstract repeats the 0 ('not at all') to 3 scoring. The RESULTS (Table 2, Distribution of PHQ-9 Scores According to Depression Diagnostic Status) formalize this with a lowest severity band labelled 'Minimal, 0–4', with 73.4% of patients with no depressive disorder falling in it. Our copy does not use the words 'bounding' or 'floor' — that framing is the citing authors' — but the instrument described is exactly the exemplar the citation is offered for. (LLM, conf=0.82)
- Not checked yetcites [30] source
- Not checked yetcites [31] source
hedged- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“These interventions were almost identical in structure and therapeutic content.”
§ Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change > Methods > The Sample
What the sources say
- Partly supportedcites [39] sourceExperimental — accuracy not yet measured
Our copy supports similarity in structure and dose: Section 2.3 (Interventions) states 'the two courses comprise a similar structure and similar amounts and forms of content' (5 lessons over 8 weeks, ~60 slides each, similar case stories and resources), and the Introduction says the TD-CBT and DS-CBT interventions were designed to 'comprise a similar structure, present similar amounts of information, and require a similar amount of therapist contact'. However, the source never calls them 'almost identical', and it explicitly builds therapeutic differences into the comparison: Section 2.3 notes DS-CBT presented all information and skills in the context of MDD with MDD-specific vignettes and no mention of other diagnoses, whereas TD-CBT avoided diagnostic labels and covered distress generally. Also, Section 2.2 describes a 2x2 design of two treatment approaches crossed with two support formats, not three interventions, so the count in the citing sentence is not this paper's description. (LLM, conf=0.78)
- Not checked yetcites [40] source
plain- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“First, psychological treatment studies often describe an increased rate of clinical change within samples of increased baseline symptom severity [20,28].”
§ Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change > Introduction
What the sources say
- Supportedcites [20] source
The results section ('Is the effect of low intensity interventions on depression moderated by baseline depression severity?') reports a significant negative interaction between baseline severity and treatment effect (coefficient -0.1, 95% CI -0.19 to -0.002), stating that 'patients who are more severely depressed at baseline demonstrate larger treatment effects than those who are less severely depressed', and the 'Strengths and weaknesses in relation to other studies' section describes this as the prevailing pattern across individual studies and across high intensity psychological therapies ('higher initial severity was associated with greater treatment effects'). This supports the claim's directional generalisation. The one qualification worth noting is that the paper repeatedly stresses the additional benefit is small (about one extra point on BDI/CES-D per SD of severity) and 'may not be clinically significant', and Box 2 adds that more severe patients remain more symptomatic afterwards. (LLM, conf=0.80)
- Could not verifycites [28] sourceExperimental — accuracy not yet measured
Our copy of Boettcher et al. (2014) reports clinically significant change rates for its own sample (Results > Clinical change, Table 3) and defines CSC on the LSAS-SR (Statistical analyses), but no passage states or examines that higher baseline symptom severity is associated with greater rate of clinical change. The closest material is the Limitations section noting this trial's pre-post effect sizes (d = 1.39) exceeded five previous trials of the same ICBT programme (mean d = 1.04), which the authors attribute to dose (11 vs 9 modules) rather than baseline severity; the only baseline-value-as-predictor analysis concerns initial attention bias score predicting attention bias change (Results > Change in attention bias), not symptom severity predicting clinical change. Searched: baseline symptom severity, initial severity predictor of outcome, clinically significant change, sample more severely symptomatic — none yield a statement of the attributed generalisation. (LLM, conf=0.62)
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Supported
None of the sources cited for this claim have been checked yet.
In the paper
“the use of effect sizes, such as Cohen d, is based on statistical assumptions that change is linear.”
§ Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change > Introduction
What the sources say
- Could not verifycites [18] sourceExperimental — accuracy not yet measured
Our copy of Lakens (2013) explains how Cohen's d is calculated and which standardizer to use (between-subjects ds, dz, drm, dav, Glass's Δ), the bias correction to Hedges's g, and the availability of bootstrapping when data are not normally distributed (Cohen's d in between-subjects designs; Cohen's d in one-sample or correlated samples comparisons). No passage states or implies that Cohen's d rests on a statistical assumption that change is linear; searches for 'linear change/linearity', 'assumptions underlying Cohen's d', and distributional assumptions returned nothing on that point. Since our copy does not address the claim's specific assertion, it can neither confirm nor contradict it. (LLM, conf=0.70)
- Not checked yetcites [27] Ellis, PD, 2010, The Essential Guide to Effect Sizes: Statistical Power, Meta-Analysis, and the Interpretation of Research Results source
plainstates a cause- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: passes — The paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Could not verify
The cited sources support only part of this claim — read each source’s verdict below.
In the paper
“This finding is broadly consistent with the data within previous psychotherapy studies showing increased effect sizes with samples of increased symptoms, even when similar treatments are applied [20,29,32].”
§ Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change > Discussion
What the sources say
- Partly supportedcites [20] sourceExperimental — accuracy not yet measured
Split into 2 atoms — 1 confirmed, 1 partially supported, 0 disputed, 0 misattributed, 0 unverifiable. Expand the claim to see per-atom verdicts.
- Could not verifycites [29] sourceExperimental — accuracy not yet measured
Split into 2 atoms — 0 confirmed, 0 partially supported, 0 disputed, 0 misattributed, 2 unverifiable. Expand the claim to see per-atom verdicts.
- Not checked yetcites [32] source
plainstates an association- Asserted: passes — The authors put it forward as true.
- Checkable: passes — It can be checked against evidence.
- Load-bearing: unclear — It is unclear whether the paper’s conclusions depend on it.
- Based: passes — The paper gives a reason why it holds.
- Partly supported
13 more claims
Create a free account to see all 37 claims and to verify the 24 references still open.
- Probably not a substantial claim. 10
- It may state two separate claims as one. 11
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
The source backs part of the claim, not all of it — or backs it for a different population, condition or comparison.
In the citing paper
“Generalized estimating equation (GEE) models were run in R and examined symptoms from assessment to post-treatment to follow-up after controlling for enrolment year and therapist location. 45”
From Utilization, Patient Characteristics, and Longitudinal Improvements among Patients from a Provincially Funded Transdiagnostic Internet-delivered Cognitive Behavioural Therapy Program: Observational Study of Trends over 6 Years (2022) · cites this paper as [45]
Experimental — accuracy not yet measured
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
Rated major: it would touch a result — a value a result rests on, reported and recomputed p-values on opposite sides of the significance level, two passages that conflict with no sign which is right, or a cited work retracted before this paper was published.
In the paper
“For example, in this study, individuals with severe baseline symptoms demonstrated effect sizes that increased by nearly threefold (3.9) when compared to individuals with mild symptoms (1.6), even when the same treatment was applied.”
§ Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change > Discussion
Reported 3 · recomputed 2.4375
Citations
Cites
The PHQ-9: Validity of a brief depression severity measure
Kroenke, K., Spitzer, R. L., & Williams, J. B. W. (2001) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2001.016009606.x · Cited by 44,910
A Brief Measure for Assessing Generalized Anxiety Disorder: The GAD-7
Spitzer, R. L., Kroenke, K., Williams, J. B. W., & Löwe, B. (2006) · Archives of Internal Medicine · doi:10.1001/archinte.166.10.1092 · Cited by 31,765
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., Vandenbroucke, J. P., & Initiative, F. T. S. (2007) · PLoS Medicine · doi:10.1371/journal.pmed.0040296 · Cited by 23,020
Longitudinal data analysis using generalized linear models
LIANG, K.-Y., & ZEGER, S. L. (1986) · Biometrika · doi:10.1093/biomet/73.1.13 · Cited by 18,550
Information Theory and an Extension of the Maximum Likelihood Principle
Akaike, H. (1998) · Springer Series in Statistics · doi:10.1007/978-1-4612-1694-0_15 · Cited by 17,781
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
KESSLER, R. C., ANDREWS, G., COLPE, L. J., HIRIPI, E., MROCZEK, D. K., NORMAND, S.-L. T., WALTERS, E. E., & ZASLAVSKY, A. M. (2002) · Psychological Medicine · doi:10.1017/s0033291702006074 · Cited by 10,327
Lakens, D. (2013) · Frontiers in Psychology · doi:10.3389/fpsyg.2013.00863 · Cited by 10,303
von Elm, E., Altman, D. G., Egger, M., Pocock, S. J., Gøtzsche, P. C., & Vandenbroucke, J. P. (2014) · International Journal of Surgery · doi:10.1016/j.ijsu.2014.07.013 · Cited by 9,909
Anxiety disorders in primary care: prevalence, impairment, comorbidity, and detection
Kroenke, K., Spitzer, R. L., Williams, J. B. W., Monahan, P. O., & Löwe, B. (2007) · Annals of Internal Medicine · doi:10.7326/0003-4819-146-5-200703060-00004 · Cited by 4,741
Fitzmaurice, G. M., Laird, N. M., & Ware, J. H. (2011) · Wiley Series in Probability and Statistics · doi:10.1002/9781119513469 · Cited by 4,639
Des Jarlais, D. C., Lyles, C., Crepaz, N., & the TREND Group (2004) · American Journal of Public Health · doi:10.2105/ajph.94.3.361 · Cited by 2,817
Akaike's Information Criterion in Generalized Estimating Equations
Pan, W. (2001) · Biometrics · doi:10.1111/j.0006-341x.2001.00120.x · Cited by 2,493
Ellis, P. D. (2010) · doi:10.1017/cbo9780511761676 · Cited by 2,461
Standards of Evidence: Criteria for Efficacy, Effectiveness and Dissemination
Flay, B. R., Biglan, A., Boruch, R. F., Castro, F. G., Gottfredson, D., Kellam, S., Mościcki, E. K., Schinke, S., Valentine, J. C., & Ji, P. (2005) · Prevention Science · doi:10.1007/s11121-005-5553-y · Cited by 1,338 · cited 3×
“The most common methodology in psychotherapy research is the combined use of standardized scales, such as standardized symptom scales of anxiety [12] or depression [1,13], and the use of statistical analyses, such as Cohen d effect sizes, that measure and interpret the rate of change in treatment [4-6].”
Longitudinal Data Analysis Using Generalized Linear Models
Liang, K., & Zeger, S. L. (1986) · Biometrika · doi:10.2307/2336267 · Cited by 1,201
Hubbard (2010) · doi:10.1097/ede.0b013e3181caeb90 · Cited by 1,139
Snyder, C. F., Aaronson, N. K., Choucair, A. K., Elliott, T. E., Greenhalgh, J., Halyard, M. Y., Hess, R., Miller, D. M., Reeve, B. B., & Santana, M. (2012) · Quality of Life Research · doi:10.1007/s11136-011-0054-x · Cited by 910
Clark, D. M. (2011) · International Review of Psychiatry · doi:10.3109/09540261.2011.606803 · Cited by 699 · cited 1×
“For example, symptom scales and effect sizes are used to evaluate treatment-related change and treatment efficacy within psychotherapy trials [17-19], epidemiological studies [20,21], meta-analytic studies of various treatments [22], and are even mandated within clinical guidelines for reporting in clinical trials, su…”
Gottfredson, D. C., Cook, T. D., Gardner, F. E. M., Gorman-Smith, D., Howe, G. W., Sandler, I. N., & Zafft, K. M. (2015) · Prevention Science · doi:10.1007/s11121-015-0555-x · Cited by 627
Newby, J. M., McKinnon, A., Kuyken, W., Gilbody, S., & Dalgleish, T. (2015) · Clinical Psychology Review · doi:10.1016/j.cpr.2015.06.002 · Cited by 545
Standardising outcomes for clinical trials and systematic reviews
Clarke, M. (2007) · Trials · doi:10.1186/1745-6215-8-39 · Cited by 483
Choi, S. W., Schalet, B., Cook, K. F., & Cella, D. (2014) · Psychological Assessment · doi:10.1037/a0035768 · Cited by 480 · cited 1×
“The most common methodology in psychotherapy research is the combined use of standardized scales, such as standardized symptom scales of anxiety [12] or depression [1,13], and the use of statistical analyses, such as Cohen d effect sizes, that measure and interpret the rate of change in treatment [4-6].”
How Should Remission From Depression Be Defined? The Depressed Patient’s Perspective
Zimmerman, M., McGlinchey, J. B., Posternak, M. A., Friedman, M., Attiullah, N., & Boerescu, D. (2006) · American Journal of Psychiatry · doi:10.1176/appi.ajp.163.1.148 · Cited by 431
Field, A. P., & Wilcox, R. R. (2017) · Behaviour Research and Therapy · doi:10.1016/j.brat.2017.05.013 · Cited by 418
Wise, E. A. (2004) · Journal of Personality Assessment · doi:10.1207/s15327752jpa8201_10 · Cited by 399 · cited 1×
“Accurate measurement of treatment-related change is a key part of psychotherapy research [1-3] and the investigation of treatment efficacy [4-6].”
Evidence‐based practice in clinical psychology: What it is, why it matters; what you need to know
Spring, B. (2007) · Journal of Clinical Psychology · doi:10.1002/jclp.20373 · Cited by 388
Keller, M. B. (2003) · JAMA · doi:10.1001/jama.289.23.3152 · Cited by 388
Titov, N., Dear, B. F., Johnston, L., Lorian, C., Zou, J., Wootton, B., Spence, J., McEvoy, P. M., & Rapee, R. M. (2013) · PLoS ONE · doi:10.1371/journal.pone.0062873 · Cited by 364
Self-citationEnhancing recovery rates: Lessons from year one of IAPT
Gyani, A., Shafran, R., Layard, R., & Clark, D. M. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.06.004 · Cited by 345
Bower, P., Kontopantelis, E., Sutton, A., Kendrick, T., Richards, D. A., Gilbody, S., Knowles, S., Cuijpers, P., Andersson, G., Christensen, H., et al. (2013) · BMJ · doi:10.1136/bmj.f540 · Cited by 341
Cognitive Behavioral Therapy for Mood Disorders: Efficacy, Moderators and Mediators
Driessen, E., & Hollon, S. D. (2010) · Psychiatric Clinics of North America · doi:10.1016/j.psc.2010.04.005 · Cited by 291
Practice-Based Evidence Study Design for Comparative Effectiveness Research
Horn, S. D., & Gassaway, J. (2007) · Medical Care · doi:10.1097/mlr.0b013e318070c07b · Cited by 257
Schalet, B. D., Cook, K. F., Choi, S. W., & Cella, D. (2014) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2013.11.006 · Cited by 257
Defining successful treatment outcome in depression using the PHQ-9: A comparison of methods
McMillan, D., Gilbody, S., & Richards, D. (2010) · Journal of Affective Disorders · doi:10.1016/j.jad.2010.04.030 · Cited by 247 · cited 2×
“This study aims to (1) explore the fundamental statistical characteristics of treatment-related depressive symptom change and (2) compare the implications from measuring and interpreting clinical change through effect sizes, such as Cohen d, against emerging alternatives, such as percentage improvement (proportional,…”
Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Zou, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., McEvoy, P. M., et al. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2015.09.003 · Cited by 230
Self-citationSome methodological and statistical issues in the study of change processes in psychotherapy
Laurenceau, J.-P., Hayes, A. M., & Feldman, G. C. (2007) · Clinical Psychology Review · doi:10.1016/j.cpr.2007.01.007 · Cited by 225
Titov, N., Dear, B. F., Staples, L. G., Terides, M. D., Karin, E., Sheehan, J., Johnston, L., Gandy, M., Fogliati, V. J., Wootton, B. M., et al. (2015) · Journal of Anxiety Disorders · doi:10.1016/j.janxdis.2015.08.002 · Cited by 223
Self-citationDriessen, E., Cuijpers, P., Hollon, S. D., & Dekker, J. J. M. (2010) · Journal of Consulting and Clinical Psychology · doi:10.1037/a0020570 · Cited by 208
Dear (2015) · doi:10.1097/j.pain.0000000000000251 · Cited by 187
Titov, N., Dear, B. F., Staples, L. G., Bennett-Levy, J., Klein, B., Rapee, R. M., Shann, C., Richards, D., Andersson, G., Ritterband, L., et al. (2015) · Psychiatric Services · doi:10.1176/appi.ps.201400477 · Cited by 181
Self-citationAltman, D. G., & Simera, I. (2016) · Journal of the Royal Society of Medicine · doi:10.1177/0141076815625599 · Cited by 173
Mixed and Mixture Regression Models for Continuous Bounded Responses Using the Beta Distribution
Verkuilen, J., & Smithson, M. J. (2011) · Journal of Educational and Behavioral Statistics · doi:10.3102/1076998610396895 · Cited by 146
Kroenke, K., Monahan, P. O., & Kean, J. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.023 · Cited by 100
Severity of depression and response to cognitive behavior therapy
(1991) · American Journal of Psychiatry · doi:10.1176/ajp.148.6.784 · Cited by 100 · cited 1×
“This bounding within symptom scales should theoretically imply that, under effective treatment, all individuals would reduce their symptoms down to the same endpoint of minimal levels [1,9] and that the rate of change would systematically depend on an individual’s symptoms at baseline [32,33].”
SOBOCKI, P., EKMAN, M., ÅGREN, H., RUNESON, B., & JÖNSSON, B. (2008) · International Journal of Clinical Practice · doi:10.1111/j.1742-1241.2006.00997.x · Cited by 83
Hiller, W., Schindler, A. C., & Lambert, M. J. (2012) · Psychotherapy Research · doi:10.1080/10503307.2011.616237 · Cited by 82
Baldwin, S. A., Fellingham, G. W., & Baldwin, A. S. (2016) · Health Psychology · doi:10.1037/hea0000292 · Cited by 82
Challenging Issues in Clinical Trial Design
Pocock, S., Clayton, T. C., & Stone, G. W. (2015) · Journal of the American College of Cardiology · doi:10.1016/j.jacc.2015.10.051 · Cited by 77
Boettcher, J., Hasselrot, J., Sund, E., Andersson, G., & Carlbring, P. (2014) · Cognitive Behaviour Therapy · doi:10.1080/16506073.2013.809141 · Cited by 68 · cited 1×
“First, psychological treatment studies often describe an increased rate of clinical change within samples of increased baseline symptom severity [20,28].”
Gunn, J. M., Elliott, P., Densley, K., Middleton, A., Ambresin, G., Dowrick, C. F., Herrman, H., Hegarty, K. L., Gilchrist, G., & Griffiths, F. E. (2013) · Journal of Affective Disorders · doi:10.1016/j.jad.2012.12.021 · Cited by 62
A Contextual Data Mining Approach Toward Assisting the Treatment of Anxiety Disorders
Panagiotakopoulos, T. C., Lyras, D. P., Livaditis, M., Sgarbas, K. N., Anastassopoulos, G. C., & Lymberopoulos, D. K. (2010) · IEEE Transactions on Information Technology in Biomedicine · doi:10.1109/titb.2009.2038905 · Cited by 60
Predictors of outcome in Internet-based cognitive behavior therapy for severe health anxiety
Hedman‐Lagerlöf, E., Lindefors, N., Andersson, G., Andersson, E., Lekander, M., Rück, C., & Ljótsson, B. (2013) · Behaviour Research and Therapy · doi:10.1016/j.brat.2013.07.009 · Cited by 59
Judd, L. L., Schettler, P. J., Rush, A. J., Coryell, W. H., Fiedorowicz, J. G., & Solomon, D. A. (2016) · The Journal of Clinical Psychiatry · doi:10.4088/jcp.15m09918 · Cited by 40 · cited 1×
“For example, a symptom reduction demonstrated by individuals with severe baseline symptoms could be interpreted as a more substantive clinical effect than an equivalent symptom reduction achieved with individuals with mild or moderate symptoms [47].”
Cases, clusters, densities: Modeling the nonlinear dynamics of complex health trajectories
Castellani, B. C., Rajaram, R., Gunn, J. M., & Griffiths, F. E. (2015) · Complexity · doi:10.1002/cplx.21728 · Cited by 32
Cited by
Karamians, R., Proffitt, R. M., Kline, D., & Gauthier, L. V. (2019) · Archives of Physical Medicine and Rehabilitation · doi:10.1016/j.apmr.2019.10.195 · Cited by 183
Predictive models for clinical decision making: Deep dives in practical machine learning
Eloranta, S., & Boman, M. (2022) · Journal of Internal Medicine · doi:10.1111/joim.13483 · Cited by 163
Titov, N., Dear, B. F., Nielssen, O., Wootton, B., Kayrouz, R., Karin, E., Genest, B., Bennett-Levy, J., Purtell, C., Bezuidenhout, G., et al. (2020) · The Lancet Digital Health · doi:10.1016/s2589-7500(20)30224-7 · Cited by 108
Self-citationHadjistavropoulos, H. D., Peynenburg, V., Thiessen, D. L., Nugent, M., Karin, E., Staples, L., Dear, B. F., & Titov, N. (2022) · The Canadian Journal of Psychiatry · doi:10.1177/07067437211006873 · Cited by 52
Self-citationDear, B. F., Gandy, M., Karin, E., Fogliati, R., Fogliati, V. J., Staples, L. G., Wootton, B. M., Sharpe, L., & Titov, N. (2018) · The Journal of Pain · doi:10.1016/j.jpain.2018.07.005 · Cited by 51
Self-citation
Metadata
- Authors
- Karin, E., Dear, B. F., Heller, G. Z., Gandy, M., & Titov, N.
- Affiliations
- Macquarie University (AU)
- Corresponding author
- Karin, Eyal
- Venue
- JMIR Mental Health
- Publisher
- JMIR Publications Inc.
- Published
- 2018-07-12
- Volume
- 5
- Issue
- 3
- Pages
- e10200
- ISSN
- 2368-7959 (ISSN-L)
- PMC
- PMC6062691
- OpenAlex
- W2801744434
- Cited by
- 69
- References
- 54
- Citations by year
- 2026: 6 · 2025: 1 · 2024: 9 · 2023: 9 · 2022: 7 · 2021: 9 · 2020: 18 · 2019: 9 · 2018: 1
- FWCI
- 11.67
- Type
- journal article
- Language
- en
- Open access
- gold
- doi.org
- Licence
- cc-by
- Locations
- JMIR Mental Health (publishedVersion, cc-by)
- Europe PMC (PubMed Central) (publishedVersion, cc-by)
- PubMed Central (publishedVersion, cc-by)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
- Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
- Keywords
- PHQ-9, measurement error, baseline symptom severity, treatment efficacy, waitlist control
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
The claim verdicts this report counts are experimental: their accuracy is not yet measured.
A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).
A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).
Cited sources are checked with a language model’s help (qwen3.8-omni-flash).
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.