Bias from historical control groups used in orthodontic research: a meta-epidemiological study
Open accessThe DOI matches its Crossref record; 12 of the 12 references it cites were found in a registry or an index.
Abstract
AIM: The validity of meta-analysis is dependent upon the quality of included studies. Here, we investigated whether the design of untreated control groups (i.e. source and timing of data collection) influences the results of clinical trials in orthodontic research. MATERIALS AND METHODS: This meta-epidemiological study used unrestricted literature searching for meta-analyses in orthodontics including clinical trials with untreated control groups. Differences in standardized mean differences (ΔSMD) and their 95% confidence intervals (CIs) were calculated according to the untreated control group through multivariable random-effects meta-regression controlling for nature of the interventional group and study sample size. Effects were pooled with random-effects synthesis, followed by mixed-effect subgroup and sensitivity analyses. RESULTS: Studies with historical control groups reported deflated treatment effects compared to studies with concurrent control groups (13 meta-analyses; ΔSMD = -0.31; 95% CI = -0.53, -0.10; P = 0.004). Significant differences were found according to the type of historical control group (based either on growth study or clinical archive; 11 meta-analyses; ΔSMD = 0.40; 95% CI = 0.21, 0.59; P < 0.001). CONCLUSIONS: The use of historical control groups in orthodontic clinical research was associated with deflation of treatment effects, which was independent from whether the interventional group was prospective or retrospective and from the study's sample size. Caution is warranted when interpreting clinical studies with historical untreated control groups or when interpreting systematic reviews that include such studies. REGISTRATION: PROSPERO (CRD42015024179). CONFLICT OF INTEREST: None.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
A Re-Evaluation of Random-Effects Meta-Analysis
Higgins, J. P. T., Thompson, S. G., & Spiegelhalter, D. J. (2009) · Journal of the Royal Statistical Society Series A: Statistics in Society · doi:10.1111/j.1467-985x.2008.00552.x · Cited by 2,569 · cited 1×
“95% predictive intervals were calculated for the ΔSMD, which incorporate existing heterogeneity and provide a range of possible effects for a future meta-analysis (19).”
MethodImproved tests for a random effects meta‐regression with a single covariate
Knapp, G., & Hartung, J. (2003) · Statistics in Medicine · doi:10.1002/sim.1482 · Cited by 2,075
Systematic Reviews in Health Care: Meta‐Analysis in Context
(2001) · doi:10.1002/9780470693926 · Cited by 1,693
Controlling the risk of spurious findings from meta-regression
Higgins, J. P. T., & Thompson, S. G. (2004) · Statistics in Medicine · doi:10.1002/sim.1752 · Cited by 1,267 · cited 1×
“An iterative residual maximum likelihood algorithm was used for the estimation of betweenstudy variance, due to its performance (15), and the Knapp–Hartung modification (16) was used for the calculation of the ΔSMDs, which accounts for the uncertainty in the heterogeneity estimate (17).”
MethodComparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Randomized versus historical controls for clinical trials
Sacks, H., Chalmers, T. C., & Smith, H. (1982) · The American Journal of Medicine · doi:10.1016/0002-9343(82)90815-4 · Cited by 834
Interpretation of tests of heterogeneity and bias in meta-analysis
Ioannidis, J. P. A. (2008) · Journal of Evaluation in Clinical Practice · doi:10.1111/j.1365-2753.2008.00986.x · Cited by 696 · cited 1×
“10 used for the test of heterogeneity and reporting biases (20).”
MethodSavović, J., Jones, H., Altman, D., Harris, R., Jűni, P., Pildal, J., Als-Nielsen, B., Balk, E., Gluud, C., Gluud, L., et al. (2012) · Health Technology Assessment · doi:10.3310/hta16350 · Cited by 668 · cited 1×
“As an example, it has been shown that inadequacies in the generation of a randomization sequence, allocation concealment or blinding in RCTs can lead to biased estimates (12).”
BackgroundStatistics notes. Treatment allocation in controlled trials: why randomise?
Altman, D. G., & Bland, J. M. (1999) · BMJ · doi:10.1136/bmj.318.7192.1209 · Cited by 362 · cited 2×
“Altman and Bland (25) likewise T ab le 2 .”
Valentine, J. C., & Thompson, S. G. (2013) · Research Synthesis Methods · doi:10.1002/jrsm.1064 · Cited by 130 · cited 1×
“When incorporating non-randomized clinical trials in systematic reviews, it will always make sense to explore potential sources of heterogeneity as well as adopt a random-effects approach to acknowledge the unexplained heterogeneity (30, 31).”
Method
Cited by
Papageorgiou, S. N., Antonoglou, G. N., Michelogiannakis, D., Kakali, L., Eliades, T., & Madianos, P. (2022) · Journal of Clinical Periodontology · doi:10.1111/jcpe.13529 · Cited by 75
Self-citationPavoni, C., Cretella Lombardo, E., Franchi, L., Lione, R., & Cozza, P. (2017) · International Journal of Pediatric Otorhinolaryngology · doi:10.1016/j.ijporl.2017.07.032 · Cited by 44
Pavoni, C., Lombardo, E. C., Lione, R., Faltin, K., McNamara, J. A., Cozza, P., & Franchi, L. (2018) · European Journal of Orthodontics · doi:10.1093/ejo/cjx078 · Cited by 41
“A recent investigation (24) showed that trials with historical controls showed smaller treatment effects compared to trials with concurrent controls.”
BackgroundOral Interventions for Obstructive Sleep Apnea
Koretsi, V., Eliades, T., & Papageorgiou, S. N. (2018) · Deutsches Ärzteblatt international · doi:10.3238/arztebl.2018.0200 · Cited by 7
Self-citation
Metadata
- Authors
- Papageorgiou, S. N., Koretsi, V., & Jäger, A.
- Venue
- The European Journal of Orthodontics
- Publisher
- Oxford University Press (OUP)
- Published
- 2016-04-29
- Volume
- 39
- Issue
- 1
- Pages
- 98-105
- ISSN
- 0141-5387 (ISSN-L)
- 1460-2210
- OpenAlex
- W2344014949
- Cited by
- 80
- References
- 12
- FWCI
- 9.78
- Type
- journal article
- Language
- en
- Open access
- green
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.