CoChat
Sign Up
Cited by 54Neurobiology of Language and BilingualismStroke Rehabilitation and RecoveryDementia and Cognitive Impairment Research

Benchmarks of Significant Change After Aphasia Rehabilitation

Open access
Gilmore, Natalie, Dwyer, Michaela, Kiran, Swathi·2019·Archives of Physical Medicine and Rehabilitation
DOI 10.1016/j.apmr.2018.08.177PMC PMC6422764
Open this paper in CoChat — free account

Abstract

Objective To establish benchmarks of significant change for aphasia rehabilitation outcome measures (ie, Western Aphasia Battery-Aphasia Quotient [WAB-AQ], Communicative Effectiveness Index [CETI], Boston Naming Test [BNT]) and assess if those benchmarks significantly differ across subgroups (ie, time post onset, dose frequency, treatment type).Data sources A comprehensive literature search of 12 databases, reference lists of previous reviews, and evidence-based practice materials was conducted.Study selection Randomized controlled trials, quasi-experimental studies, single-subject design, and case studies that used a standardized outcome measure to assess change were included. Titles and full-text articles were screened using a dual review process. Seventy-eight studies met criteria for inclusion.Data extraction Data were extracted independently, and 25% of extractions were checked for reliability. All included studies were assigned quality indicator ratings and an evidence level.Data synthesis Random-effects meta-analyses were conducted separately for each study design group (ie, within-/between-group comparisons). For within-group designs, the summary effect size after aphasia rehabilitation was 5.03 points (95% confidence interval, 3.95-6.10, P<.001) on the WAB-AQ, 10.37 points (6.08-14.66, P<.001) on the CETI, and 3.30 points (2.43-4.18, P<.001) on the BNT. For between-group designs, the summary effect size was 5.05 points (1.64-8.46, P=.004) on the WAB-AQ and 0.55 points (-1.33 to 2.43, P=.564) on the BNT, the latter of which was not significant. Subgroup analyses for the within-group designs showed no significant differences in the summary effect size as a function of dose frequency or treatment type.Conclusions This study established benchmarks of significant change on 3 standardized outcome measures used in aphasia rehabilitation.

Integrity signals

Registry

Crossref · 10.1016/j.apmr.2018.08.177

Cites

113 of 113 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

0 of 26 cited sources checked · 0 supported · 0 flagged

  • Findingown data

    For within-group designs, the WAB-AQ summary effect size was 5.03 points (95% CI 3.95-6.10, p < .001) across 53 studies with 522 participants.

    “On the WAB-AQ (53 studies, n = 522), the summary ES on the raw unstandardized mean difference was 5.03 points, (95% confidence interval [CI]: 3.95-6.10, p < .001).”
    Quote in the paperSubstantial claim
  • Findingown data

    For within-group designs, the CETI summary effect size was 10.37 points (95% CI 6.08-14.66, p < .001) across 17 studies with 208 participants.

    “The CETI summary ES (17 studies, n = 208), was 10.37 points (6.08-14.66, p < .001).”
    Quote in the paperSubstantial claim
  • Findingown data

    For the within-group BNT meta-analysis, Duval and Tweedie's Trim and Fill indicated publication bias with missing negative studies.

    “For the BNT, the Duval and Tweedie’s Trim and Fill revealed the presence of publication bias (Observed point estimate = 3.30(2.43, 4.18); Imputed point estimate = 2.97(2.02, 3.92)) (i.e., missing negative studies).”
    Quote in the paperNot a substantial claim
  • Findingown data

    No statistically significant differences were found between summary effect sizes for any completed within-group subgroup analyses (dose frequency for WAB-AQ, CETI, and BNT; treatment type for WAB-AQ and BNT).

    “There were no statistically significant differences between summary ESs for any of the within group study design subgroup analyses completed (i.e., dose frequency for WAB-AQ, CETI, and BNT; treatment type for WAB-AQ and BNT).”
    Quote in the paperSubstantial claim
  • Findingown data

    No significant heterogeneity was found in the between-group BNT meta-analysis (Q = .86, df = 4, p = .93; I2 = 0).

    “There was no significant heterogeneity between included studies (Q = .86, df = 4, p = .93; I2 = 0).”
    Quote in the paperNot a substantial claim
  • May combine two claims. 18
  • Classified as probably not a substantial claim. 20
  • The quote only partly states this claim. 1

  • May combine two claims.

    Findingown data

    78 studies met criteria for inclusion in the meta-analyses, with 70 within-group and 8 between-group studies.

    “78 studies met criteria for inclusion in the meta-analyses (i.e., within group: 70; between group: 8).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingown data

    For within-group studies, most had summative quality scores of 3.

    “For within group studies, most studies had summative scores of 3, with higher scores indicating better quality.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For between-group designs, the BNT summary effect size was .55 points (95% CI −1.33 to 2.43, p = .56), which was not significant, across 5 studies with 66 experimental and 35 control participants.

    “On the BNT (5 studies, Experimental n = 66; Control n = 35), the raw unstandardized mean difference between the experimental and control groups at post-treatment was .55 (−1.33-2.43, p = .56).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Findingreasoning

    None of the 78 studies selected for meta-analysis were excluded based on quality.

    “None of the 78 studies selected for meta-analysis were excluded from the analysis based on their quality, which is unsurprising as studies of poorer quality were likely excluded during the two initial screening phases.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingown data

    For between-group study designs, 50% were classified as level IB, 38% as level IIA, and 13% as level IIB.

    “For between group study designs, 50% were classified as IB, 38% as IIA, and 13% as IIB level evidence.”
    Quote in the paperNot a substantial claim

  • May combine two claims.

    Findingcitation

    For within-group study designs, 73% of included studies were level III evidence, 26% were level IIB, and 1% were level IIA.

    “For within group study designs, 73% of studies included in the meta-analyses were level III evidence,29,31 26% were IIB, and 1% were IIA.”
    Quote in the paperNot a substantial claim0 of 2 sources checked
    • cites [29]

      Not checked in the free run.

    • cites [31] source

      Not checked in the free run.

  • May combine two claims.

    Study design and conductcitation

    Because more than 5 studies per subgroup are needed for a valid subgroup analysis, not all planned subgroup analyses were feasible.

    “As > 5 studies per subgroup are required to conduct a valid subgroup analysis,7 the same subgroup analyses were not feasible for all outcome measures and study design groups.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [7]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    A comprehensive search was conducted across 12 databases, reference lists of previous reviews, and evidence-based practice materials.

    “A comprehensive literature search of 12 databases, reference lists of previous reviews, and evidence-based practice materials was conducted.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Two reviewers independently screened 9,285 titles and abstracts against inclusion criteria, with 96% inter-reviewer reliability.

    “Two reviewers (first two authors) independently screened 9,285 titles and abstracts against inclusionary criteria (96% inter-reviewer reliability).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Funnel plots were examined for asymmetry in meta-analyses with more than 10 studies.

    “Funnel plots for meta-analyses including > 10 studies were examined for asymmetry (i.e., within group meta-analyses only).”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Random-effects meta-analyses were run separately for each study design group.

    “Random-effects meta-analyses were conducted separately for each study design group (i.e., within/between group comparisons).”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductown data

    Both reviewers screened 858 full-text articles against inclusion criteria, with 90% inter-reviewer reliability.

    “Both reviewers screened 858 full-text articles against the inclusionary criteria (90% inter-reviewer reliability).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    Data extraction was performed independently, and 25% of extractions were checked for reliability.

    “Data were extracted independently and 25% of extractions were checked for reliability.”
    Quote in the paperNot a substantial claim

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    The same two reviewers independently appraised study quality using the ASHA level of evidence scheme.

    “The same two reviewers independently appraised included studies’ quality using indicators identified by the American Speech-Language Hearing Association (ASHA) level of evidence scheme.29,30”
    Quote in the paperSubstantial claim0 of 2 sources checked
    • cites [29]

      Not checked in the free run.

    • cites [30] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    To achieve 80% power to detect an effect size of at least .50 with a random-effects model, outcome measures with cumulative within-group sample sizes below 100 or used in fewer than 10 studies were excluded.

    “To have a power of .80 to detect an effect size of ≥.50 using a random-effects model, outcome measures with cumulative sample sizes across within group studies < than 100 were excluded and/or if the measure was used in less than < 10 studies.25”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [25] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductcitation

    Meta-analyses were run separately for within- and between-group designs to avoid methodological concerns from transforming to a common metric.

    “Meta-analyses were conducted independently for within and between group study designs to avoid methodological concerns involved in transforming to a common metric.35”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [35]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    When multiple publications used the same dataset, only the one with the largest sample was included.

    “When results from the same dataset were included in multiple publications, only the publication with the greatest sample size was included.”
    Quote in the paperSubstantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    Subgroup analyses were corrected for multiple comparisons using the Bonferroni method.

    “Sub-group analyses were corrected for multiple comparisons using the Bonferroni correction method.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductnone given

    All included studies received quality indicator ratings and an evidence level.

    “All included studies were assigned quality indicator ratings and an evidence level.”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductnone given

    The same two reviewers divided data extraction, each extracting 25% of the other's studies, with 98% inter-reviewer reliability.

    “The same two reviewers responsible for screening divided the data extraction. Each reviewer extracted data for 25% of the others' studies (98% inter-reviewer reliability).”
    Quote in the paperNot a substantial claim

  • Probably not a substantial claim: may not pass S3 (load-bearing). May combine two claims.

    Study design and conductcitation

    The study adhered to PRISMA guidelines and was registered at PROSPERO under CRD42016039393.

    “This study followed the Preferred Reporting Items for Systematic Review and Meta-analyses: the PRISMA Statement19 guidelines and was registered at the International prospective register of systematic reviews, PROSPERO, under the identification number CRD42016039393.”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [19] source

      Not checked in the free run.

  • Probably not a substantial claim: may not pass S3 (load-bearing).

    Study design and conductcitation

    Publication bias was assessed using Begg and Mazumdar rank correlation, Egger's regression intercept, and Duval and Tweedie's Trim and Fill.

    “Publication bias was objectively assessed using Begg and Mazumdar rank correlation, Egger’s regression intercept and Duval and Tweedie’s Trim and Fill.7”
    Quote in the paperNot a substantial claim0 of 1 sources checked
    • cites [7]

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Study design and conductreasoning

    A random-effects model combined individual study results into a summary effect size.

    “As heterogeneity between studies was anticipated, a random-effects model was used to combine individual study results into a summary ES (i.e., raw unstandardized mean difference).”
    Quote in the paperSubstantial claim

  • Classified as probably not a substantial claim.

    Study design and conductnone given

    Studies with quality indicator summative scores of 1 or less for within-group designs and 2 or less for between-group designs were excluded for poor quality.

    “Quality indicator summative scores ≤ 1 for within group studies [Post-treatment Mean vs. Pre-treatment mean for the same group] and ≤ 2 for between group studies [Experimental group Post-treatment Change vs. Control group Post-treatment Change] were excluded for poor quality.”
    Quote in the paperSubstantial claim

  • The quote only partly states this claim.

    Interpretationown data

    The study produced benchmarks for meaningful change on three standard aphasia rehabilitation outcome measures: WAB-AQ, CETI, and BNT.

    “This study established benchmarks of significant change on three standardized outcome measures used in aphasia rehabilitation.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationown data

    The within-group summary effect sizes for WAB-AQ and BNT remained valid and can be used with confidence despite detected publication bias, because point estimates shifted minimally (less than 1 point, within the confidence interval).

    “In both cases where publication bias, was indicated, the SES shifted only minimally (i.e., < 1 point, within the confidence interval), verifying that the within group SESs reported for all three outcome measures are valid and can be utilized with confidence.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Interpretationcitation

    The WAB-AQ summary effect sizes (within-group 5.03; between-group 5.05) matched its SEM of 5, a value previously described as clinically meaningful improvement.

    “WAB-AQ summary ESs (Within group: 5.03; Between group: 5.05), were equivalent to its SEM of 5, which has been framed as a metric of clinically meaningful improvement.40–42”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [40] source

      Not checked in the free run.

    • cites [41] source

      Not checked in the free run.

    • cites [42] source

      Not checked in the free run.

  • Classified as probably not a substantial claim.

    Premisecitation

    Wallace and colleagues proposed a core outcome set for aphasia that defines a minimum group of outcomes for routine administration.

    “Wallace and colleagues proposed a core outcome set (COS)13,15–18 for aphasia, specifying a minimum set of outcomes that should be administered to persons with aphasia as standard practice (i.e., WAB, The Scenario Test, General Health Questionnaire-12, SAQOL-39g) to increase consistency.”
    Quote in the paperSubstantial claim0 of 5 sources checked
    • cites [13] source

      Not checked in the free run.

    • cites [15] source

      Not checked in the free run.

    • cites [16] source

      Not checked in the free run.

    • cites [17]

      Not checked in the free run.

    • cites [18]

      Not checked in the free run.

  • May combine two claims.

    Interpretationcitation

    The CETI summary effect size of 10.37 exceeded its SEM of 5.87, indicating that improvements were unlikely to result from measurement variation alone.

    “The CETI’s summary ES of 10.37 was well above its SEM of 5.87,23 suggesting that those improvements were not due to variations inherent to measurement alone.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [23]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    The latest Cochrane review on speech therapy efficacy pooled only randomized controlled trials, leaving out much aphasia treatment evidence.

    “Similarly, the most recent Cochrane review3 demonstrating speech therapy efficacy, synthesized data from randomized controlled trials only, excluding a wealth of aphasia treatment data.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [3]

      Not checked in the free run.

  • May combine two claims.

    Premisecitation

    Robey's meta-analyses found a positive treatment effect, but they separated studies by design and aimed to identify effect sizes for different conditions.

    “Robey’s hallmark meta-analyses2,4,5 showed a positive aphasia treatment effect, but were segregated by study design and focused on identifying the effect size for different conditions (e.g., treated vs untreated recovery).”
    Quote in the paperSubstantial claim0 of 3 sources checked
    • cites [2] source

      Not checked in the free run.

    • cites [4] source

      Not checked in the free run.

    • cites [5] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    The BNT summary effect size of 3.30 exceeded its SEM of 2.04, which supports its validity for measuring intervention-related improvement.

    “Lastly, the summary ES for the BNT of 3.30 was also higher than its SEM of 2.04,43 supporting its validity as a metric of intervention-related improvement.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [43] source

      Not checked in the free run.

  • May combine two claims.

    Interpretationown data

    Aphasia rehabilitation is effective, as confirmed by the meta-analyses.

    “Furthermore, it confirms that aphasia rehabilitation is indeed effective.”
    Quote in the paperSubstantial claim

  • May combine two claims.

    Premisecitation

    Hula et al. (2010) found that the WAB-AQ's SEM was about 2 for AQs from 28 to 68, but rose to as high as 12 outside that interval.

    “However, the seminal work of Hula, Donovan, Kendall & Gonzalez-Rothi, 2010,42 demonstrating that the WAB-AQ’s SEM was actually closer to 2 for AQs between 28-68, but much higher (i.e., up to 12) for scores outside that range (i.e., AQs of 0-27, 69-100) serves to clearly distinguish the summary ES established in this study from measurement error.”
    Quote in the paperSubstantial claim0 of 1 sources checked
    • cites [42] source

      Not checked in the free run.

34 of 34

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • The CT-R group's mean improvement of 6.75 points on the WAB-AQ is above the 5 point threshold for clinically meaningful improvement in speech-language ability.

    “a change that is above the 5 point threshold for clinically meaningful improvement in speech-language ability (46–48)”

    From A Virtual, Randomized, Control Trial of a Digital Therapeutic for Speech, Language, and Cognitive Intervention in Post-stroke Persons With Aphasia (2021) · cites this paper as [46]

    The Discussion (page unavailable) says a WAB-AQ standard error of measurement of 5 points has been framed as a metric of clinically meaningful improvement. It also cautions that this value varies with baseline aphasia severity—closer to 2 points for some scores and up to 12 for others—so the paper does not establish 5 points as a universal threshold. The CT-R group's 6.75-point improvement is the citing paper's result, not a result attributed to this source. (LLM, conf=0.84)

    Experimental — accuracy not yet measured

  • A 5-point change on the WAB-R AQ represents a conservative, clinically meaningful effect.

    “Based on benchmarking of aphasia intervention studies, a 5-point change on the WAB-R AQ represents a conservative, clinically meaningful effect. 29”

    From A randomized control trial of intensive aphasia therapy after acute stroke: The Very Early Rehabilitation for SpEech (VERSE) study (2021) · cites this paper as [29]

Statistics recomputed

Every reported test result, recomputed from its test statistic and degrees of freedom. A result shows as not adding up only once an independent check has confirmed it.

  • Not confirmedt(51) = 1.86reported p = .04 · recomputed p = .069

    “t (51) =1.86, p = .04”

1 of 1

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

Each contradiction between two passages listed here was confirmed by an independent check.

  • Dose frequency is defined in two different units

    Minor

    The subgroup classification measures dose frequency in hours per week, but the paper then defines it as the number of intervention sessions per day or week. Those are different quantities, so it is unclear which was used for the subgroup analysis.

1 of 1

Cites

Flagged first, then by citations

Research works printed but not linked: 5

  1. Wallace S, Worrall L, Rose T, Le Dorze G. Improving research outcome measurement in aphasia: development of a core outcome set. Paper presented at: International Aphasia Rehabilitation Conference. December 14, 2016; London.
  2. Cherney, 2010, Oral reading for language in aphasia: Impact of aphasia severity on cross-modal outcomes in chronic nonfluent aphasia, Seminars in Speech and Language, 31, 42, 10.1055/s-0029-1244952 · DOI · Paper page
  3. Mozeiko, 2016, The role of intensity in constraint-induced language therapy for people with chronic aphasia, Aphasiology, 30, 339, 10.1080/02687038.2015.1070949 · DOI
  4. Des Roches, 2015, Effectiveness of an impairment-based individualized rehabilitation program using an iPad-based software platform, Frontiers in Human Neuroscience, 8, 10.3389/fnhum.2014.01015 · DOI · Paper page
  5. Katz, 1997, The efficacy of computer-provided reading treatment for chronic aphasic adults, Journal of Speech & Hearing Research, 40, 493, 10.1044/jslhr.4003.493 · DOI · Paper page

Cited by

Sorted by citations

Related

Authors
Gilmore, N., Dwyer, M., & Kiran, S.
Affiliations
Boston University (US)
Corresponding author
Gilmore, Natalie
ORCID
Venue
Archives of Physical Medicine and Rehabilitation
Publisher
Elsevier BV
Published
2019-06
Volume
100
Issue
6
Pages
1131-1139.e87
ISSN
0003-9993 (ISSN-L)
DOI
10.1016/j.apmr.2018.08.177
PMC
PMC6422764
OpenAlex
W2890104961
Cited by
54
References
113
Citations by year
2026: 11 · 2025: 7 · 2024: 7 · 2023: 5 · 2022: 8 · 2021: 10 · 2020: 6
FWCI
0.54
Type
journal article
Language
en
Open access
Licence
Locations
  • Archives of Physical Medicine and Rehabilitation (publishedVersion)
  • PubMed (publishedVersion)
  • PubMed Central (acceptedVersion)
Funders
  • National Institutes of Health
  • National Institute on Deafness and Other Communication Disorders (T32DC013017)
Topics
  • Life Sciences › Neuroscience › Cognitive Neuroscience › Neurobiology of Language and Bilingualism
  • Health Sciences › Medicine › Rehabilitation › Stroke Rehabilitation and Recovery
  • Health Sciences › Medicine › Psychiatry and Mental health › Dementia and Cognitive Impairment Research
Keywords
aphasia rehabilitation, Boston Naming Test, outcome measures, treatment response, dosing frequency, treatment type
MeSH terms
Aphasia, Humans, Language Tests, Severity of Illness Index, Treatment Outcome, Outcome Assessment, Health Care, Benchmarking

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Claims read under definition version 3 (extract-v3) on 2026-09-28.

Classified by jev-1.13-20260917 (gold-guideline-v0.2) on 2026-09-28.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review