Student or educator? Sign up with your school email and get 6 months of Pro free.
CoChat

The DOI matches its Crossref record; 55 of the 55 references it cites were found in a registry or an index; 8 of the 32 sources its claims cite were checked for support.

Measurement of Symptom Change Following Web-Based Psychotherapy: Statistical Characteristics and Analytical Methods for Measuring and Interpreting Change

Open access

Published version of a preprint

Abstract

Background: Accurate measurement of treatment-related change is a key part of psychotherapy research and the investigation of treatment efficacy. For this reason, the ability to measure change with accurate and valid methods is critical for psychotherapy. Objective: The aims of this study were to (1) explore the underlying characteristics of depressive symptom change, measured with the nine-item Patient Health Questionnaire (PHQ-9), following psychotherapy, and (2) compare the suitability of different ways to measure and interpret symptom change. A treatment sample of Web-based psychotherapy participants (n=1098) and a waitlist sample (n=96) were used to (1) explore the statistical characteristics of depressive symptom change, and (2) compare the suitability of two common types of change functions: linear and proportional change. Methods: These objectives were explored using hypotheses that tested (1) the relationship between baseline symptoms and the rate of change, (2) the shape of symptom score distribution following treatment, and (3) measurement error associated with linear and proportional measurement models. Results: Findings demonstrated that (1) individuals with severe depressive baseline symptoms had greater reductions in symptom scores than individuals with mild baseline symptoms (11.4 vs 3.7); however, as a percentage measurement, change remained similar across individuals with mild, moderate, or severe baseline symptoms (50%-55%); (2) positive skewness was observed in PHQ-9 score distributions following treatment; and (3) models that measured symptom change as a proportional function resulted in greater model fit and reduced measurement error (<30%). Conclusions: This study suggests that symptom scales, sharing an implicit feature of score bounding, are associated with a proportional function of change. Selecting statistics that overlook this proportional change (eg, Cohen d) is problematic and leads to (1) artificially increased estimates of change with higher baseline symptoms, (2) increased measurement error, and (3) confounded estimates of treatment efficacy and clinical change. Implications, limitations, and idiosyncrasies from these results are discussed.

Overview

CoChat Summary

TL;DR

This study argues that PHQ-9 symptom change after web-based psychotherapy is better characterized as proportional rather than linear: participants with more severe baseline symptoms changed more in raw points, but changed similarly in percentage terms. Across a treatment sample of 1,098 and a waitlist sample of 96, proportional models fit better and reduced measurement error.

Key findings

  • In the treatment sample, participants with severe baseline symptoms reduced PHQ-9 scores more than those with mild symptoms in raw points: 11.4 vs 3.7; yet proportional change was similar across baseline-severity bands at 50%–55%.
  • In the waitlist condition, symptom change also depended on baseline severity; severe participants showed 35% proportional improvement, and severe waitlist participants reduced more than moderate treatment participants (7 points vs 6 points).
  • PHQ-9 posttreatment distributions showed positive skewness, which increased after treatment.
  • Models assuming proportional change had better fit and lower error: the paper reports reduced QIC, AIC, and BIC and a drop in PHQ-9 error variance from σ²=24.122 to σ²=16.716, described as more than 30% less measurement error.

Methods

  • Combined clinical data from three published randomized controlled trials of internet-delivered cognitive behavioral therapy (ICBT) conducted in the same clinic; total participants N=1262, with 1098 assessed at both pretreatment and posttreatment. A waitlist sample included n=96.
  • Primary outcome was the PHQ-9 (scores 0–27); participants were adults 18–65 years with at least mild depression or anxiety symptoms (minimum ≥5 on PHQ-9 or GAD-7).
  • Tested three hypotheses using GEE models, multilevel/mixed models, and raw means, then compared linear vs proportional change models using QIC, AIC, BIC, and residual error/measurement error.

Limitations

  • The authors state the findings are preliminary, specific to the PHQ-9, and specific to one treatment model (the Macquarie University online model).
  • The paper says it is unclear how far the 50%–55% change generalizes to other symptom scales or treatment models.
  • Evident from the text: the sample comes from self-selecting Australian adults in web-based CBT trials, so generalizability beyond this setting is limited.

Model-written summary. Check it against the paper before relying on it.

Integrity signals

Registry

Crossref · 10.2196/10200

Cites

55 of 55 references verified0 citation errors · 0 not checkable

Statistics

Nothing to recompute.

Figures

Figure 1 Equations 1.1-1.8.
Figure 1 Equations 1.1-1.8.CC BY · PubMed Central
Figure 2 Measurement of mean treatment-related PHQ-9 symptom change per initial pretreatment symptom severity band; whiskers represent 95% CI s. Symptom change observed under control conditions indicated by a solid trend line.
Figure 2 Measurement of mean treatment-related PHQ-9 symptom change per initial pretreatment symptom severity band; whiskers represent 95% CI s. Symptom change observed under control conditions indicated by a solid trend line.CC BY · PubMed Central
Figure 3 Measurement of mean treatment-related PHQ-9 symptom change as a proportional pattern of remission (52%); per initial pretreatment symptom severity; whiskers represent 95% CIs. Symptom change observed under control conditions indicated by a solid trend line.
Figure 3 Measurement of mean treatment-related PHQ-9 symptom change as a proportional pattern of remission (52%); per initial pretreatment symptom severity; whiskers represent 95% CIs. Symptom change observed under control conditions indicated by a solid trend line.CC BY · PubMed Central
Figure 4 Dispersion of symptom scores (nine-item Patient Health Questionnaire, PHQ-9) at pretreatment (in light bars) and posttreatment scores (in dark bars). The dotted trend lines are indicative of the shape of each distribution.
Figure 4 Dispersion of symptom scores (nine-item Patient Health Questionnaire, PHQ-9) at pretreatment (in light bars) and posttreatment scores (in dark bars). The dotted trend lines are indicative of the shape of each distribution.CC BY · PubMed Central
Figure 5 PHQ-9 estimation error (residual) following fixed (linear) and relative (proportional) change assumption.
Figure 5 PHQ-9 estimation error (residual) following fixed (linear) and relative (proportional) change assumption.CC BY · PubMed Central

Verification

Claims the paper makes

5 Valid · 0 Not valid · 12 Not checked · 20 Own results and interpretations

13 more claims

Create a free account to see all 37 claims and to verify the 24 references still open.

See all claims — free account
  • Probably not a substantial claim. 10
  • It may state two separate claims as one. 11

Create a free account to see these claims.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

1 of 1

Citations

Cites

Show all 54 references

Cited by

Metadata

Authors
Karin, E., Dear, B. F., Heller, G. Z., Gandy, M., & Titov, N.
Affiliations
Macquarie University (AU)
Corresponding author
Karin, Eyal
Venue
JMIR Mental Health
Published
2018-07-12
Volume
5
Issue
3
Pages
e10200
ISSN
2368-7959 (ISSN-L)
OpenAlex
W2801744434
Cited by
69
References
54
Citations by year
2026: 6 · 2025: 1 · 2024: 9 · 2023: 9 · 2022: 7 · 2021: 9 · 2020: 18 · 2019: 9 · 2018: 1
FWCI
11.67
Type
journal article
Language
en
Open access
Licence
cc-by
Locations
  • JMIR Mental Health (publishedVersion, cc-by)
  • Europe PMC (PubMed Central) (publishedVersion, cc-by)
  • PubMed Central (publishedVersion, cc-by)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Psychology › Applied Psychology › Digital Mental Health Interventions
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Anxiety, Depression, Psychometrics, Treatment, Cognitive Processes
  • Social Sciences › Psychology › Experimental and Cognitive Psychology › Mental Health Research Topics
Keywords
PHQ-9, measurement error, baseline symptom severity, treatment efficacy, waitlist control

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

The claim verdicts this report counts are experimental: their accuracy is not yet measured.

A language model drew the claims from the paper on 2026-10-11, under version 3 of the claim definition (model deepseek-v4.1-flash, prompt extract-v3).

A second model labelled each claim on 2026-10-11 (model jev-1.13-20260917, definition revision gold-guideline-v0.2).

Cited sources are checked with a language model’s help (qwen3.8-omni-flash).

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review

Data sources

Bibliographic data: Semantic Scholar, OpenAlex, Crossref.

Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).

Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.