Opportunities and challenges in using studies without a control group in comparative effectiveness reviews: Single group studies in comparative effectiveness reviews
The DOI matches its Crossref record; 15 of the 15 references it cites were found in a registry or an index.
Abstract
When examining the evidence on therapeutic interventions to answer a comparative effectiveness research question, one should consider all studies that are informative on the interventions' causal effects. "Single group studies" evaluate outcomes longitudinally in cohorts of subjects who are managed with a single treatment strategy. Because these studies are "missing" a direct, concurrent comparison group, they are typically deemed non-informative on comparative effectiveness. However, in principle, single group studies can provide information on causal treatment effects by extrapolating expected outcomes in the "missing" untreated arm. Single group studies rely on before-after, implicit, or historical comparisons as a proxy for an ideal comparison group. The validity of these comparisons must be carefully examined on a case-by-case basis. While in many cases, researchers will disagree on whether such extrapolations are reasonable; circumstances exist where such studies are generally acceptable as a source of evidence. This article provides an overview of issues related to the interpretation of single group studies with a focus on the assumptions required to support their consideration in comparative effectiveness reviews. We discuss the various settings in which single group studies are employed, common research designs that systematic reviewers need to interpret, and challenges associated with using these designs to inform comparative effectiveness questions.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Evaluating non-randomised intervention studies
Deeks, J., Dinnes, J., D'Amico, R., Sowden, A., Sakarovitch, C., Song, F., Petticrew, M., & Altman, D. (2003) · Health Technology Assessment · doi:10.3310/hta7270 · Cited by 2,907 · cited 2×
“…focusing on historically controlled studies of cancer treatments (Diehl and Perry, 1986) andmore recent research on vascular disease populations (Deeks et al., 2003) also support the contention that such studies often favor recently introduced treatments because of underlying time trends in…”
MethodA structural approach to selection bias
Hernán, M. A., Hernández-Díaz, S., & Robins, J. M. (2004) · Epidemiology · doi:10.1097/01.ede.0000135174.63482.43 · Cited by 2,844 · cited 2×
“Also, if patients with either more or less favorable outcomes are lost to follow-up [a form of selection bias (Hernan et al., 2004), corresponding to the concept of “attrition”] (Shadish et al., 2002), the observed effect of the intervention will be respectively underestimated or overestimated,…”
BackgroundLipsey, M. W., & Wilson, D. B. (1993) · American Psychologist · doi:10.1037/0003-066x.48.12.1181 · Cited by 2,033 · cited 1×
“Across 45 meta-analyses of psychological, educational, and behavioral treatments, before–after studies produced more extreme effect sizes compared with controlled studies of the same intervention (Lipsey and Wilson, 1993); we are not aware of a similar systematic comparison in medicine.”
BackgroundSanderson, S., Tatt, I. D., & Higgins, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym018 · Cited by 1,531 · cited 1×
“Several tools exist for the evaluation of risk of bias (or “quality”) in observational studies but none are specifically tailored or validated for use with single group studies (Deeks et al., 2003; Effective Practice and Organisation of Care, 2013; Ramsay et al., 2003; Sanderson et al., 2007).”
MethodMeta-analysis Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Randomized versus historical controls for clinical trials
Sacks, H., Chalmers, T. C., & Smith, H. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90815-4 · Cited by 834
Stijnen, T., Hamza, T. H., & Özdemir, P. (2010) · Statistics in Medicine · doi:10.1002/sim.4040 · Cited by 757 · cited 1×
“In addition, most systematic reviewers are unfamiliar with the statistical tools to combine single group and comparative studies (Begg and Pilote, 1991; Brumback et al., 1999; Dempster et al., 1983; Li and Begg, 1994; Neuenschwander et al., 2010; Pocock, 1976; Stijnen et al., 2010; Stram, 1996).”
BackgroundLittle, R. J., & Rubin, D. B. (2000) · Annual Review of Public Health · doi:10.1146/annurev.publhealth.21.1.121 · Cited by 721
Ramsay, C. R., Matowe, L., Grilli, R., Grimshaw, J. M., & Thomas, R. E. (2003) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462303000576 · Cited by 703
Model Choice in Time Series Studies of Air Pollution and Mortality
Peng, R. D., Dominici, F., & Louis, T. A. (2006) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2006.00410.x · Cited by 583
The combination of randomized and historical controls in clinical trials
Pocock, S. J. (1976) · Journal of Chronic Diseases · doi:10.1016/0021-9681(76)90044-8 · Cited by 514
MacLehose, Reeves, Harvey, Sheldon, Russell, & Black (2000) · Health Technology Assessment · doi:10.3310/hta4340 · Cited by 311
Dahabreh, I. J., Sheldrick, R. C., Paulus, J. K., Chung, M., Varvarigou, V., Jafri, H., Rassen, J. A., Trikalinos, T. A., & Kitsios, G. D. (2012) · European Heart Journal · doi:10.1093/eurheartj/ehs114 · Cited by 204 · cited 1×
“Furthermore, empirical evidence indicates that in many cases, different study designs reach similar conclusions (Britton et al., 1998; Dahabreh et al., 2012; Dalziel et al., 2005; Ioannidis et al., 2001; MacLehose et al., 2000; Vandenbroucke, 2011), offering another rationale for quantitatively…”
ResultHiggins, J. P., Ramsay, C., Reeves, B. C., Deeks, J. J., Shea, B., Valentine, J. C., Tugwell, P., & Wells, G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1056 · Cited by 183 · cited 1×
“Implementation of this guidance may result in increased consistency among reviews in the near future (Higgins et al., 2013).”
Background
Cited by
Jones, V. K., Hanus, M., Yan, C., Shade, M. Y., Blaskewicz Boron, J., & Maschieri Bicudo, R. (2021) · Frontiers in Public Health · doi:10.3389/fpubh.2021.750736 · Cited by 111
A Systematic Review of Self-Compassion in Chronic Pain: From Correlation to Efficacy
Lanzaro, C., Carvalho, S. A., Lapa, T. A., Valentim, A., & Gago, B. (2021) · The Spanish Journal of Psychology · doi:10.1017/sjp.2021.22 · Cited by 29
Sanchez-Autet, M., Arranz, B., Sierra, P., Safont, G., Garcia-Blanco, A., de la Fuente, L., Garriga, M., Marín, L., & García-Portilla, M. P. (2022) · The World Journal of Biological Psychiatry · doi:10.1080/15622975.2021.2013089 · Cited by 15
Fitzpatrick, D., & Burns, J. (2018) · Campbell Systematic Reviews: Better Evidence for a Better World · doi:10.1002/cl2.194 · Cited by 1
Metadata
- Authors
- Paulus, J. K., Dahabreh, I. J., Balk, E. M., Avendano, E. E., Lau, J., & Ip, S.
- Venue
- Research Synthesis Methods
- Publisher
- Wiley
- Published
- 2013-10-23
- Volume
- 5
- Issue
- 2
- Pages
- 152-161
- ISSN
- 1759-2879 (ISSN-L)
- OpenAlex
- W1899417405
- Cited by
- 62
- References
- 15
- FWCI
- 1.08
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.