Reporting characteristics of meta-analyses in orthodontics: methodological assessment and statistical recommendations
The DOI matches its Crossref record; 31 of the 31 references it cites were found in a registry or an index.
Abstract
Ideally meta-analyses (MAs) should consolidate the characteristics of orthodontic research in order to produce an evidence-based answer. However severe flaws are frequently observed in most of them. The aim of this study was to evaluate the statistical methods, the methodology, and the quality characteristics of orthodontic MAs and to assess their reporting quality during the last years. Electronic databases were searched for MAs (with or without a proper systematic review) in the field of orthodontics, indexed up to 2011. The AMSTAR tool was used for quality assessment of the included articles. Data were analyzed with Student's t-test, one-way ANOVA, and generalized linear modelling. Risk ratios with 95% confidence intervals were calculated to represent changes during the years in reporting of key items associated with quality. A total of 80 MAs with 1086 primary studies were included in this evaluation. Using the AMSTAR tool, 25 (27.3%) of the MAs were found to be of low quality, 37 (46.3%) of medium quality, and 18 (22.5%) of high quality. Specific characteristics like explicit protocol definition, extensive searches, and quality assessment of included trials were associated with a higher AMSTAR score. Model selection and dealing with heterogeneity or publication bias were often problematic in the identified reviews. The number of published orthodontic MAs is constantly increasing, while their overall quality is considered to range from low to medium. Although the number of MAs of medium and high level seems lately to rise, several other aspects need improvement to increase their overall quality.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
GRADE: an emerging consensus on rating quality of evidence and strength of recommendations
Guyatt, G. H., Oxman, A. D., Vist, G. E., Kunz, R., Falck-Ytter, Y., Alonso-Coello, P., & Schünemann, H. J. (2008) · BMJ · doi:10.1136/bmj.39489.470347.ad · Cited by 23,305
A modified test for small‐study effects in meta‐analyses of controlled trials with binary endpoints
Harbord, R. M., Egger, M., & Sterne, J. A. C. (2006) · Statistics in Medicine · doi:10.1002/sim.2380 · Cited by 2,267
The Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
Jüni, P. (1999) · JAMA · doi:10.1001/jama.282.11.1054 · Cited by 1,981 · cited 1×
“, 2011), although the use of scales has been cautioned (Jüni et al., 1999; Greenland and O’Rourke, 2001).”
BackgroundShea, B. J., Hamel, C., Wells, G. A., Bouter, L. M., Kristjansson, E., Grimshaw, J., Henry, D. A., & Boers, M. (2009) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.10.009 · Cited by 1,755
Systematic Review of the Empirical Evidence of Study Publication Bias and Outcome Reporting Bias
Dwan, K., Altman, D. G., Arnaiz, J. A., Bloom, J., Chan, A.-W., Cronin, E., Decullier, E., Easterbrook, P. J., Von Elm, E., Gamble, C., et al. (2008) · PLoS ONE · doi:10.1371/journal.pone.0003081 · Cited by 1,546
Sanderson, S., Tatt, I. D., & Higgins, J. P. (2007) · International Journal of Epidemiology · doi:10.1093/ije/dym018 · Cited by 1,531 · cited 1×
“A wide variety of checklists and scales exist (Moher et al., 1995; Sanderson et al., 2007; Higgins et al., 2011), although the use of scales has been cautioned (Jüni et al., 1999; Greenland and O’Rourke, 2001).”
BackgroundMoher, D., Schulz, K. F., Altman, D. G., & for the CONSORT Group* (2001) · Annals of Internal Medicine · doi:10.7326/0003-4819-134-8-200104170-00011 · Cited by 1,426
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308 · cited 1×
“A wide variety of checklists and scales exist (Moher et al., 1995; Sanderson et al., 2007; Higgins et al., 2011), although the use of scales has been cautioned (Jüni et al.”
BackgroundMeho, L. I., & Yang, K. (2007) · Journal of the American Society for Information Science and Technology · doi:10.1002/asi.20677 · Cited by 974
Plint, A. C., Moher, D., Morrison, A., Schulz, K., Altman, D. G., Hill, C., & Gaboury, I. (2006) · Medical Journal of Australia · doi:10.5694/j.1326-5377.2006.tb00557.x · Cited by 872
Subgroup analyses in randomized trials: risks of subgroup-specific analyses;
Brookes, S. T., Whitely, E., Egger, M., Smith, G. D., Mulheran, P. A., & Peters, T. J. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2003.08.009 · Cited by 775 · cited 1×
“Sensitivity analyses, subgroup analyses, meta-regressions can be prespecified to identify causes for heterogeneity, but potential pitfalls of these approaches should be reported (Brookes et al., 2004; Lambert et al., 2002).”
Grey literature in meta-analyses of randomized trials of health care interventions
Hopewell, S., McDonald, S., Clarke, M. J., & Egger, M. (2007) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000010.pub3 · Cited by 598 · cited 1×
“Poor primary study quality has been possibly associated with effect overestimation or underestimation (Hopewell et al., 2007), although D ow naded rom http/academ ic.”
BackgroundExternal Validation of a Measurement Tool to Assess Systematic Reviews (AMSTAR)
Shea, B. J., Bouter, L. M., Peterson, J., Boers, M., Andersson, N., Ortiz, Z., Ramsay, T., Bai, A., Shukla, V. K., & Grimshaw, J. M. (2007) · PLoS ONE · doi:10.1371/journal.pone.0001350 · Cited by 569
Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D. (1998) · JAMA · doi:10.1001/jama.280.3.278 · Cited by 534
Hopewell, S., Dutton, S., Yu, L.-M., Chan, A.-W., & Altman, D. G. (2010) · BMJ · doi:10.1136/bmj.c723 · Cited by 504 · cited 1×
“The years 2001, 2006, and 2011, indicating the start, middle, and end of the last decade, were selected to investigate possible improvements (Hopewell et al., 2010) in reporting of key items associated with quality of orthodontic MAs.”
BackgroundHolty, J.-E. C., & Guilleminault, C. (2010) · Sleep Medicine Reviews · doi:10.1016/j.smrv.2009.11.003 · Cited by 500
Balk, E. M. (2002) · JAMA · doi:10.1001/jama.287.22.2973 · Cited by 478
Quality of Reporting of Randomized Trials as a Measure of Methodologic Quality
Huwiler-Müntener, K. (2002) · JAMA · doi:10.1001/jama.287.21.2801 · Cited by 397 · cited 1×
“The evaluation of the reporting quality of published MAs is very useful, as it is directly related to the study’s methodology and conclusions (Huwiler-Müntener et al., 2002; Moher et al., 1998).”
MethodGreenland, S. (2001) · Biostatistics · doi:10.1093/biostatistics/2.4.463 · Cited by 359 · cited 2×
“A wide variety of checklists and scales exist (Moher et al., 1995; Sanderson et al., 2007; Higgins et al., 2011), although the use of scales has been cautioned (Jüni et al., 1999; Greenland and O’Rourke, 2001).”
BackgroundDolovich, L. R., Addis, A., Vaillancourt, J. M. R., Power, J. D. B., Koren, G., & Einarson, T. R. (1998) · BMJ · doi:10.1136/bmj.317.7162.839 · Cited by 353
Sharpe, D. (1997) · Clinical Psychology Review · doi:10.1016/s0272-7358(97)00056-1 · Cited by 334
The Effects of Orthodontic Therapy on Periodontal Health
Bollen, A.-M., Cunha-Cruz, J., Bakko, D. W., Huang, G. J., & Hujoel, P. P. (2008) · The Journal of the American Dental Association · doi:10.14219/jada.archive.2008.0184 · Cited by 310
Miles, P. G., Vig, P. S., Weyant, R. J., Forrest, T. D., & Rockette, H. E. (1996) · American Journal of Orthodontics and Dentofacial Orthopedics · doi:10.1016/s0889-5406(96)70177-4 · Cited by 237
A systematic review of the relationship between overjet size and traumatic dental injuries
Nguyen, Q. (1999) · The European Journal of Orthodontics · doi:10.1093/ejo/21.5.503 · Cited by 224
How to assess the external validity of therapeutic trials: a conceptual approach
Dekkers, O. M., Elm, E. V., Algra, A., Romijn, J. A., & Vandenbroucke, J. P. (2010) · International Journal of Epidemiology · doi:10.1093/ije/dyp174 · Cited by 219
Adjustment of meta-analyses on the basis of quality scores should be abandoned
Herbison, P., Hay-Smith, J., & Gillespie, W. J. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.03.008 · Cited by 207
Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time
Trikalinos, T. A., Churchill, R., Ferri, M., Leucht, S., Tuunainen, A., Wahlbeck, K., & Ioannidis, J. P. A. (2004) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2004.02.018 · Cited by 177
Nieuwenhuijsen, M. J., Martinez, D., Grellier, J., Bennett, J., Best, N., Iszatt, N., Vrijheid, M., & Toledano, M. B. (2009) · Environmental Health Perspectives · doi:10.1289/ehp.0900677 · Cited by 175
Using Existing Systematic Reviews in Complex Systematic Reviews
Whitlock, E. P., Lin, J. S., Chou, R., Shekelle, P., & Robinson, K. A. (2008) · Annals of Internal Medicine · doi:10.7326/0003-4819-148-10-200805200-00010 · Cited by 144
Systematic reviews can be produced and published faster
Sampson, M., Shojania, K. G., Garritty, C., Horsley, T., Ocampo, M., & Moher, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2008.02.004 · Cited by 103 · cited 2×
“AMSTAR has been validated to some extent (Shea et al., 2007a, 2009; Sampson et al., 2008a) and proposed by the Canadian Agency for Drugs and Technologies in Health and the World Health Organization as the best tool to critically appraise MAs (Proposed Evaluation Tools for COMPUS, 2005;…”
BackgroundMethodNo consensus exists on search reporting methods for systematic reviews
Sampson, M., McGowan, J., Tetzlaff, J., Cogo, E., & Moher, D. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.10.009 · Cited by 63 · cited 2×
“AMSTAR has been validated to some extent (Shea et al., 2007a, 2009; Sampson et al., 2008a) and proposed by the Canadian Agency for Drugs and Technologies in Health and the World Health Organization as the best tool to critically appraise MAs (Proposed Evaluation Tools for COMPUS, 2005;…”
BackgroundMethod
Cited by
Sideri, S., Papageorgiou, S. N., & Eliades, T. (2018) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2018.01.003 · Cited by 388
“However, no data specific to the field of orthodontics was provided, while the inclusion of only high-impact journals might provide an overly optimistic image of protocol registration, due to considerable differences of systematic review quality according to journal characteristics [16,17].”
BackgroundResultInfluentialSelf-citationA historical review of publication bias
Marks‐Anglin, A., & Chen, Y. (2020) · Research Synthesis Methods · doi:10.1002/jrsm.1452 · Cited by 165
“Light and Pillemer's funnel plot(68) was a key development that motivated a slew of testing and correction methods for PB, one of which(69) catapulted it to become the most widely used visual tool for assessment of PB among medical researchers.(70,71) Based on the intuition that all component studies in a meta-analysi…”
MethodKoffel, J. B. (2015) · PLOS ONE · doi:10.1371/journal.pone.0125931 · Cited by 159
Basic study design influences the results of orthodontic clinical investigations
Papageorgiou, S. N., Xavier, G. M., & Cobourne, M. T. (2015) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2015.03.008 · Cited by 108
“However, high-quality clinical trials do not always exist, and non-RCTs of interventions (nonRCTs) are often included in systematic reviews and metaanalyses in orthodontics [4,5], which can potentially affect their conclusions.”
BackgroundSelf-citationKoffel, J. B., & Rethlefsen, M. L. (2016) · PLOS ONE · doi:10.1371/journal.pone.0163309 · Cited by 107
Metadata
- Authors
- Papageorgiou, S. N., Papadopoulos, M. A., & Athanasiou, A. E.
- Venue
- The European Journal of Orthodontics
- Publisher
- Oxford University Press (OUP)
- Published
- 2013-03-14
- Volume
- 36
- Issue
- 1
- Pages
- 74-85
- ISSN
- 0141-5387 (ISSN-L)
- 1460-2210
- OpenAlex
- W2099593344
- Cited by
- 51
- References
- 31
- FWCI
- 8.25
- Type
- journal article
- Language
- en
- Open access
- closed
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.