The DOI matches its Crossref record; 127 of the 127 references it cites were found in a registry or an index.
Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview
Open accessAbstract
BACKGROUND: There is considerable debate as to the relative merits of using randomised controlled trial (RCT) data as opposed to observational data in systematic reviews of adverse effects. This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of RCTs as compared to meta-analysis of observational studies. METHODS AND FINDINGS: Searches were carried out in ten databases in addition to reference checking, contacting experts, citation searches, and hand-searching key journals, conference proceedings, and Web sites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from RCTs could be directly compared, using the ratio of odds ratios, with the pooled estimate for the same adverse effect arising from observational studies. Nineteen studies, yielding 58 meta-analyses, were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% confidence interval 0.93-1.15). There was less discrepancy with larger studies. The symmetric funnel plot suggests that there is no consistent difference between risk estimates from meta-analysis of RCT data and those from meta-analysis of observational studies. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped (54/58, 93%). In terms of statistical significance, in nearly two-thirds (37/58, 64%), the results agreed (both studies showing a significant increase or significant decrease or both showing no significant difference). In only one meta-analysis about one adverse effect was there opposing statistical significance. CONCLUSIONS: Empirical evidence from this overview indicates that there is no difference on average in the risk estimate of adverse effects of an intervention derived from meta-analyses of RCTs and meta-analyses of observational studies. This suggests that systematic reviews of adverse effects should not be restricted to specific study types. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544
Analysing Data and Undertaking Meta‐Analyses
Deeks, J. J., Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch9 · Cited by 1,800
Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., Schulz, K., Moher, D., & for the CONSORT Group* (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-10-200411160-00009 · Cited by 1,593
Evidence-based medicine : how to practice and teach EBM
Straus, S. E. (2005) · Cited by 1,367
Meta-analysis of the Impact of 9 Medication Classes on Falls in Elderly Persons
Woolcott, J. C. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.357 · Cited by 1,326
Drugs and Falls in Older People: A Systematic Review and Meta‐analysis: I. Psychotropic Drugs
Leipzig, R. M., Cumming, R. G., & Tinetti, M. E. (1999) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1999.tb01898.x · Cited by 1,153
McGettigan, P., & Henry, D. (2006) · JAMA · doi:10.1001/jama.296.13.jrv60011 · Cited by 1,096
Meta-analysis Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997
Research works printed but not linked: 11
- RF Jacob, 2000, How to evaluate a dental article about harm., J Prosthet Dent, 84, 8, 10.1067/mpr.2000.108496 · doi:10.1067/mpr.2000.108496
- D Henry, 1999, Meta-analysis-its role in assessing drug safety., Pharmacoepidemiol Drug Saf, 8, 167, 10.1002/(SICI)1099-1557(199905/06)8:3<167::AID-PDS418>3.0.CO;2-F · doi:10.1002/(sici)1099-1557(199905/06)8:3<167::aid-pds418>3.0.co;2-f
- P Buekens, 2001, Invited commentary: rare side effects of obstetric interventions: are observational studies good enough?, [comment] Am J Epidemiol, 153, 108, 10.1093/aje/153.2.108 · doi:10.1093/aje/153.2.108
- KL Hyrich, 2005, Assessing the safety of biologic therapies in rheumatoid arthritis: the challenges of study design., J, Rheumatol, 48
- K Beard, 2006, Introduction., 10.1353/itx.2006.0000 · doi:10.1353/itx.2006.0000
- M McDonagh, 2006, The impact of including non-randomized studies in a systematic review: a case study [abstract].
- J Shepherd, 2006, Sometimes similar, sometimes different: a systematic review of meta-analyses of random and non-randomized policy intervention studies.
- P Jensen, 2002, Postherniorrhaphy urinary retention—effect of local, regional, and general anesthesia: a review., Reg Anesth Pain Med, 27, 612
- G Jones, 1999, Water fluoridation, bone mass and fracture: a quantitative overview of the literature., Aust N Z J Public Health, 23, 34, 10.1111/j.1467-842X.1999.tb01202.x · doi:10.1111/j.1467-842x.1999.tb01202.x
- SD Ross, 1998, Safety of GM-CSF in patients with AIDS: A review of the literature., Pharmacotherapy, 18, 1290, 10.1002/j.1875-9114.1998.tb03150.x · doi:10.1002/j.1875-9114.1998.tb03150.x
- JJ Ofman, 2002, A meta-analysis of severe upper gastrointestinal complications of non-steroidal anti-inflammatory drugs., J Rheumatol, 29, 804
Cited by
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
“21 We believe that evidence from NRSI should complement that from randomised trials, such as in providing evidence about effects on rare and adverse outcomes and long term effects to be balanced against the outcomes more readily addressed in randomised trials.”
Self-citationChang, S.-H., Stoll, C. R. T., Song, J., Varela, J. E., Eagon, C. J., & Colditz, G. A. (2014) · JAMA Surgery · doi:10.1001/jamasurg.2013.3654 · Cited by 1,688
Gloy, V. L., Briel, M., Bhatt, D. L., Kashyap, S. R., Schauer, P. R., Mingrone, G., Bucher, H. C., & Nordmann, A. J. (2013) · BMJ · doi:10.1136/bmj.f5934 · Cited by 1,351
Anglemyer, A., Horvath, H. T., & Bero, L. (2014) · Cochrane Database of Systematic Reviews · doi:10.1002/14651858.mr000034.pub2 · Cited by 534
Get real in individual participant data (IPD) meta‐analysis: a review of the methodology
Debray, T. P. A., Moons, K. G. M., van Valkenhoef, G., Efthimiou, O., Hummel, N., Groenwold, R. H. H., Reitsma, J. B., & on behalf of the GetReal methods review group (2015) · Research Synthesis Methods · doi:10.1002/jrsm.1160 · Cited by 348
Metadata
- Authors
- Golder, S., Loke, Y. K., & Bland, M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-05-03
- Volume
- 8
- Issue
- 5
- Pages
- e1001026
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- OpenAlex
- W1989467621
- Cited by
- 269
- References
- 127
- FWCI
- 30.73
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
How to cite
Cite CoChat for the checks on this page, and the paper for its own findings.
- This page
- CoChat. (2026). Citation check: Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview. https://app.cochat.ai/papers/10.1371/journal.pmed.1001026
- The paper
- Golder, S., Loke, Y. K., & Bland, M. (2011). Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview. PLoS Medicine. https://doi.org/10.1371/journal.pmed.1001026
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.
