Change score or follow-up score? Choice of mean difference estimates could impact meta-analysis conclusions
Abstract
Objectives In randomized controlled clinical trials, continuous outcomes are typically measured at both baseline and follow-up, and mean difference could be estimated using the change scores from baseline or the follow-up scores. This study assesses the impact of using change score vs. follow-up score on the conclusions of meta-analyses.Study design and setting A total of 63 meta-analyses from six comparative effectiveness reviews were included. The combined mean difference was estimated using a random-effects model, and we also evaluated whether the impact qualitatively varied by alternative random-effects estimates.Results Based on the Dersimonian-Laird (DL) method, using the change vs. the follow-up score led to five meta-analyses (7.9%) showing discrepancy in conclusions. Based on the profile likelihood (PL) method, nine (14.3%) showed discrepancy in conclusions. Using change score was more likely to show a significant difference in effects between interventions (DL method: 4 of 5; PL method: 7 of 9). A significant difference in baseline scores did not necessarily lead to discrepancies in conclusions.Conclusions Using the change vs. the follow-up score could lead to important discrepancies in conclusions. Sensitivity analyses should be conducted to check the robustness of results to the choice of mean difference estimates.
Integrity signals
Registry
Cites
Statistics
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
Follow-up values were used here, whereas Greff et al used mean change values (delta) between baseline and post-treatment, which tend to produce larger effect sizes and thus more significant results in meta-analyses.
“Moreover, follow-up values were used here, whereas Greff et al used mean change values (delta) between baseline and post-treatment, which tend to produce larger effect sizes and thus more significant results in meta-analyses (63).”
From Inositol for Polycystic Ovary Syndrome: A Systematic Review and Meta-analysis to Inform the 2023 Update of the International Evidence-based PCOS Guidelines (2024) · cites this paper as [63]
Comparing post-treatment means only, instead of a change score between baseline and post means, is less prone to bias.
“As stated, comparing the post treatment means only, instead of a change score between baseline and post means, is less prone to bias [22].”
From Cognitive behavioral therapy for insomnia in patients with mental disorders and comorbid insomnia: A systematic review and meta-analysis (2022) · cites this paper as [22]
Cites
Measuring inconsistency in meta-analyses
Higgins, J. P. T., Thompson, S. G., Deeks, J. J., & Altman, D. G. (2003) · BMJ · doi:10.1136/bmj.327.7414.557 · Cited by 63,139 · see citing works
IntHout, J., Ioannidis, J. P., & Borm, G. F. (2014) · BMC Medical Research Methodology · doi:10.1186/1471-2288-14-25 · Cited by 2,419 · see citing works
Analysing controlled trials with baseline and follow up measurements
Vickers, A. J., & Altman, D. G. (2001) · BMJ · doi:10.1136/bmj.323.7321.1123 · Cited by 2,116 · see citing works
Improved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075 · see citing works
The Agency for Healthcare Research and Quality
Baine, W. B. (2006) · Italian Journal of Public Health · doi:10.2427/5924 · Cited by 1,191 · see citing works
Allocation concealment in randomised trials: defending against deciphering
Schulz, K. F., & Grimes, D. A. (2002) · The Lancet · doi:10.1016/s0140-6736(02)07750-4 · Cited by 923 · see citing works
Testing for baseline balance in clinical trials
Senn, S. (1994) · Statistics in Medicine · doi:10.1002/sim.4780131703 · Cited by 503 · see citing works
Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change
Cornell, J. E., Mulrow, C. D., Localio, R., Stack, C. B., Meibohm, A. R., Guallar, E., & Goodman, S. N. (2014) · Annals of Internal Medicine · doi:10.7326/m13-2886 · Cited by 453 · see citing works
Meta-analysis of skewed data: Combining results reported on log-transformed or raw scales
Higgins, J., White, I., & Anzures-Cabrera, J. (2008) · Statistics in Medicine · doi:10.1002/sim.3427 · Cited by 441 · see citing works
Understanding controlled trials: Baseline imbalance in randomised controlled trials
Roberts, C., & Torgerson, D. J. (1999) · BMJ · doi:10.1136/bmj.319.7203.185 · Cited by 287 · see citing works
Covariate imbalance and random allocation in clinical trials
Senn, S. J. (1989) · Statistics in Medicine · doi:10.1002/sim.4780080410 · Cited by 281 · see citing works
Kontopantelis, E., & Reeves, D. (2010) · Statistical Methods in Medical Research · doi:10.1177/0962280210392008 · Cited by 264 · see citing works
da Costa, B. R., Nüesch, E., Rutjes, A. W., Johnston, B. C., Reichenbach, S., Trelle, S., Guyatt, G. H., & Jüni, P. (2013) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2013.03.009 · Cited by 148 · see citing works
Assessing the impact of attrition in randomized controlled trials
Hewitt, C. E., Kumaravel, B., Dumville, J. C., & Torgerson, D. J. (2010) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2010.01.010 · Cited by 139 · see citing works
Comparing groups in a before–after design: When t test and ANCOVA produce different results
Wright, D. B. (2006) · British Journal of Educational Psychology · doi:10.1348/000709905x52210 · Cited by 135 · see citing works
Kontopantelis, E., & Reeves, D. J. (2012) · Statistical Methods in Medical Research · doi:10.1177/0962280211413451 · Cited by 117 · see citing works
(2012) · Europe PMC (PubMed Central) · Cited by 100 · see citing works
Riley, R. D., Kauser, I., Bland, M., Thijs, L., Staessen, J. A., Wang, J., Gueyffier, F., & Deeks, J. J. (2013) · Statistics in Medicine · doi:10.1002/sim.5726 · Cited by 100 · see citing works
Amador‐Licona, N., Guízar-Mendoza, J., Vargas, E., Sánchez-Camargo, G., & Zamora-Mata, L. (2000) · Archives of Medical Research · doi:10.1016/s0188-4409(00)00241-1 · Cited by 78 · see citing works
Analysis of Covariance in Parallel-Group Clinical Trials with Pretreatment Baselines
Crager, M. (1987) · Biometrics · doi:10.2307/2531543 · Cited by 76 · see citing works
Gartlehner, G. (2014) · Brown Digital Repository · doi:10.7301/z0np22cc · Cited by 73 · see citing works
Significance tests of covariate imbalance in clinical trials
Begg, C. B. (1990) · Controlled Clinical Trials · doi:10.1016/0197-2456(90)90037-3 · Cited by 73 · see citing works
Diabetes Medications for Adults With Type 2 Diabetes: An Update
Bolen, S. D., Tseng, E., Hutfless, S. M., Segal, J. B., Suarez‐Cuervo, C., Berger, Z. D., Wilson, L. M., Chu, Y., Iyoha, E., & Maruthur, N. M. (2016) · Europe PMC (PubMed Central) · Cited by 69 · see citing works
Trowman, R., Dumville, J. C., Torgerson, D. J., & Cranny, G. (2007) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.03.014 · Cited by 59 · see citing works
Quantifying Bias in Randomized Controlled Trials in Child Health: A Meta-Epidemiological Study
Hartling, L., Hamm, M. P., Fernandes, R., Dryden, D., & Vandermeer, B. (2014) · PLoS ONE · doi:10.1371/journal.pone.0088008 · Cited by 39 · see citing works
Nonpharmacologic Interventions for Treatment-Resistant Depression in Adults
Gaynes, B. N., Lux, L. J., Lloyd, S., Hansen, R. A., Gartlehner, G., Keener, P., Brode, S., Evans, T. S., Jonas, D. E., Crotty, K., et al. (2011) · Cited by 34 · see citing works
Fu, R., & Holmer, H. K. (2015) · Europe PMC (PubMed Central) · Cited by 33 · see citing works
Self-citationPain Management Interventions for Hip Fracture
Abou-Setta, A. M., Beaupré, L. A., Jones, C. A., Rashiq, S., Hamm, M. P., Sadowski, C. A., Menon, M. R., Majumdar, S. R., Wilson, D. M., Karkhaneh, M., et al. (2011) · Cited by 27 · see citing works
Flagged first, then by citations
Cited by
Sorted by citations
Hertenstein, E., Trinca, E., Wunderlin, M., Schneider, C. L., Züst, M. A., Fehér, K. D., Su, T., Straten, A. V., Berger, T., Baglioni, C., et al. (2022) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2022.101597 · Cited by 391 · see citing works
Rüsch, N., & Kösters, M. (2021) · Social Psychiatry and Psychiatric Epidemiology · doi:10.1007/s00127-021-02076-y · Cited by 90 · see citing works
The effect of sleep on novel word learning in healthy adults: a systematic review and meta-analysis
Schimke, E. A. E., Angwin, A. J., Cheng, B. B. Y., & Copland, D. A. (2021) · Psychonomic Bulletin & Review · doi:10.3758/s13423-021-01980-3 · Cited by 41 · see citing works
- Authors
- Fu, R., & Holmer, H. K.
- Affiliations
- Oregon Health & Science University (US)
- Corresponding author
- Fu, Rongwei
- Venue
- Journal of Clinical Epidemiology
- Publisher
- Elsevier BV
- Published
- 2016-08
- Volume
- 76
- Pages
- 108-117
- ISSN
- 0895-4356 (ISSN-L)
- OpenAlex
- W2284084005
- Cited by
- 71
- References
- 28
- Citations by year
- 2026: 14 · 2025: 5 · 2024: 16 · 2023: 11 · 2022: 7 · 2021: 8 · 2020: 2 · 2019: 3 · 2018: 3 · 2017: 1 · 2016: 1
- FWCI
- 2.51
- Type
- journal article
- Language
- en
- Open access
- closed
- Locations
- Journal of Clinical Epidemiology (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- change scores, baseline correction, continuous outcomes, random effects models, DerSimonian-Laird method, sensitivity analysis
- MeSH terms
- Data Interpretation, Statistical, Follow-Up Studies, Humans, Meta-Analysis as Topic, Randomized Controlled Trials as Topic, Treatment Outcome, Biomedical Research
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
