Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
The DOI matches its Crossref record; 28 of the 33 references it cites were found in a registry or an index.
Abstract
Background Few meta-analyses of randomised trials assess the quality of the studies included. Yet there is increasing evidence that trial quality can affect estimates of intervention efficacy. We investigated whether different methods of quality assessment provide different estimates of intervention efficacy evaluated in randomised controlled trials (RCTs).Methods We randomly selected 11 meta-analyses that involved 127 RCTs on the efficacy of interventions used for circulatory and digestive diseases, mental health, and pregnancy and childbirth. We replicated all the meta-analyses using published data from the primary studies. The quality of reporting of all 127 clinical trials was assessed by means of component and scale approaches. To explore the effects of quality on the quantitative results, we examined the effects of different methods of incorporating quality scores (sensitivity analysis and quality weights) on the results of the meta-analyses.Findings The quality of trials was low. Masked assessments provided significantly higher scores than unmasked assessments (mean 2.74 [SD 1.10] vs 2.55 [1.20]). Low-quality trials (score < or = 2), compared with high-quality trials (score > 2), were associated with an increased estimate of benefit of 34% (ratio of odds ratios [ROR] 0.66 [95% CI 0.52-0.83]). Trials that used inadequate allocation concealment, compared with those that used adequate methods, were also associated with an increased estimate of benefit (37%; ROR=0.63 [0.45-0.88]). The average treatment benefit was 39% (odds ratio [OR] 0.61 [0.57-0.65]) for all trials, 52% (OR 0.48 [0.43-0.54]) for low-quality trials, and 29% (OR 0.71 [0.65-0.77]) for high-quality trials. Use of all the trial scores as quality weights reduced the effects to 35% (OR 0.65 [0.59-0.71]) and resulted in the least statistical heterogeneity.Interpretation Studies of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.
Overview
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
Claims other papers make about this one
Each is checked against this paper’s own text: does it say what the citing paper says it does?
In the citing paper
“Given that highly biased studies are more likely to overestimate treatment effectiveness (Moher 1998), the quality of included studies was evaluated independently by KW and one of either TTS, EA, DG, PH, ET, or KvH using the criteria described in Higgins 2011.”
From Interventions for self-harm in children and adolescents (2021) · cites this paper as Moher 1998
In the citing paper
“The quality of each study, defined as the confidence that its design, conduct, analysis, and presentation limited bias in its results, was assessed by a validated quality scoring system (39).”
In the citing paper
“The evidence regarding the need for quality assessment of studies included in SRs is more consistent in indicating that bypassing this important step may lead to substantial bias in the review estimates [37–39, 49, 50].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [38]
In the citing paper
“A clear illustration of this phenomenon was shown in the study by Moher and colleagues, where the pooled estimate of low-quality trials, compared to high quality-trials, demonstrated 34 % greater benefit in the treatment effect [38].”
From How to conduct systematic reviews more expeditiously? (2015) · cites this paper as [38]
Citations
Cites
The Measurement of Observer Agreement for Categorical Data
Landis, J. R., & Koch, G. G. (1977) · Biometrics · doi:10.2307/2529310 · Cited by 81,148
Assessing the quality of reports of randomized clinical trials: Is blinding necessary?
Jadad, A. R., Moore, R. A., Carroll, D., Jenkinson, C., Reynolds, D. J. M., Gavaghan, D. J., & McQuay, H. J. (1996) · Controlled Clinical Trials · doi:10.1016/0197-2456(95)00134-4 · Cited by 17,582
Self-citationSchulz (1995) · doi:10.1001/jama.1995.03520290060030 · Cited by 5,634
Improving the Quality of Reporting of Randomized Controlled Trials: The CONSORT Statement
Begg, C. (1996) · JAMA · doi:10.1001/jama.1996.03540080059030 · Cited by 3,127
Moher, D., Jadad, A. R., Nichol, G., Penman, M., Tugwell, P., & Walsh, S. (1995) · Controlled Clinical Trials · doi:10.1016/0197-2456(94)00031-w · Cited by 1,308
Self-citationIncorporating variations in the quality of individual randomized trials into meta-analysis
Detsky, A. S., Naylor, C. D., O'Rourke, K., McGeer, A. J., & L'Abbé, K. A. (1992) · Journal of Clinical Epidemiology · doi:10.1016/0895-4356(92)90085-2 · Cited by 640
Better reporting of randomised controlled trials: the CONSORT statement
Altman, D. G. (1996) · BMJ · doi:10.1136/bmj.313.7057.570 · Cited by 609
Jadad, A. R., Cook, D. J., Jones, A., Klassen, T. P., Tugwell, P., Moher, M., & Moher, D. (1998) · JAMA · doi:10.1001/jama.280.3.278 · Cited by 534
Self-citationDoes blinding of readers affect the results of meta-analyses?
Berlin, J. A. (1997) · The Lancet · doi:10.1016/s0140-6736(05)62352-5 · Cited by 478
Assessing the Quality of Randomized Controlled Trials:Current Issues and Future Directions
Moher, D., Jadad, A. R., & Tugwell, P. (1996) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462300009570 · Cited by 436
Self-citation
Cited by
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Meta-analysis of Observational Studies in Epidemiology<SUBTITLE>A Proposal for Reporting</SUBTITLE>
Stroup, D. F. (2000) · JAMA · doi:10.1001/jama.283.15.2008 · Cited by 20,623
CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials
the CONSORT Group, Schulz, K. F., Altman, D. G., & Moher, D. (2010) · BMC Medicine · doi:10.1186/1741-7015-8-18 · Cited by 13,371
Self-citationHao (2015) · doi:10.1002/14651858 · Cited by 11,321
Moher, D., Schulz, K. F., & Altman, D. G. (2001) · The Lancet · doi:10.1016/s0140-6736(00)04337-3 · Cited by 5,373
Self-citation
Metadata
- Authors
- Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P.
- Affiliations
- University of Ottawa (CA)
- Children's Hospital of Eastern Ontario (CA)
- McMaster University (CA)
- University of Oxford (GB)
- Corresponding author
- Moher, David
- Venue
- The Lancet
- Publisher
- Elsevier BV
- Published
- 1998-08
- Volume
- 352
- Issue
- 9128
- Pages
- 609-613
- ISSN
- 0099-5355 (ISSN-L)
- 0140-6736
- OpenAlex
- W2134338262
- Cited by
- 3,560
- References
- 33
- Citations by year
- 2026: 11 · 2025: 26 · 2024: 55 · 2023: 75 · 2022: 68 · 2021: 87 · 2020: 91 · 2019: 106 · 2018: 110 · 2017: 124 · 2016: 144 · 2015: 148 · 2014: 186 · 2013: 232 · 2012: 186
- FWCI
- 151.25
- Type
- journal article
- Language
- en
- Open access
- closed
- Licence
- www.elsevier.com/tdm/userlicense/1.0/ (tdm)
- Locations
- The Lancet (publishedVersion)
- PubMed (publishedVersion)
- Topics
- Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
- Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Clinical Trials
- Social Sciences › Sociology and Political Science › Delphi Technique in Research
- Keywords
- trial quality assessment, reporting quality, randomized controlled trials, meta-analysis, intervention efficacy, sensitivity analysis, allocation concealment, blinding, heterogeneity
- MeSH terms
- Double-Blind Method, Humans, Publishing, Research Design, Meta-Analysis as Topic, Bias, Randomized Controlled Trials as Topic
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? https://app.cochat.ai/papers/10.1016/s0140-6736(98)01085-x
- The paper
- Moher, D., Pham, B., Jones, A., Cook, D. J., Jadad, A. R., Moher, M., Tugwell, P., & Klassen, T. P. (1998). Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses? The Lancet. https://doi.org/10.1016/s0140-6736(98)01085-x
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.