Meta-analyses of Adverse Effects Data Derived from Randomised Controlled Trials as Compared to Observational Studies: Methodological Overview
Open accessThe DOI matches its Crossref record; 127 of the 127 references it cites were found in a registry or an index.
Abstract
BACKGROUND: There is considerable debate as to the relative merits of using randomised controlled trial (RCT) data as opposed to observational data in systematic reviews of adverse effects. This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of RCTs as compared to meta-analysis of observational studies. METHODS AND FINDINGS: Searches were carried out in ten databases in addition to reference checking, contacting experts, citation searches, and hand-searching key journals, conference proceedings, and Web sites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from RCTs could be directly compared, using the ratio of odds ratios, with the pooled estimate for the same adverse effect arising from observational studies. Nineteen studies, yielding 58 meta-analyses, were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% confidence interval 0.93-1.15). There was less discrepancy with larger studies. The symmetric funnel plot suggests that there is no consistent difference between risk estimates from meta-analysis of RCT data and those from meta-analysis of observational studies. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped (54/58, 93%). In terms of statistical significance, in nearly two-thirds (37/58, 64%), the results agreed (both studies showing a significant increase or significant decrease or both showing no significant difference). In only one meta-analysis about one adverse effect was there opposing statistical significance. CONCLUSIONS: Empirical evidence from this overview indicates that there is no difference on average in the risk estimate of adverse effects of an intervention derived from meta-analyses of RCTs and meta-analyses of observational studies. This suggests that systematic reviews of adverse effects should not be restricted to specific study types. Please see later in the article for the Editors' Summary.
Overview
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Integrity signals
Registry
Cites
Statistics
Verification
Claims the paper makes
The claims and statistics of this paper haven’t been checked yet.
Free: it reads the open-access text and checks the first 5 claims that cite a source we can check. Create a free account to check every claim.
Self-check
Percentages, sums and reported differences re-computed from the paper’s own numbers.
The self-check needs the paper’s full text.
Citations
Cites
Cochrane Handbook for Systematic Reviews of Interventions: Cochrane Book Series
(2008) · doi:10.1002/9780470712184 · Cited by 25,048
Randomized, Controlled Trials, Observational Studies, and the Hierarchy of Research Designs
Concato, J., Shah, N., & Horwitz, R. I. (2000) · New England Journal of Medicine · doi:10.1056/nejm200006223422507 · Cited by 3,563
External validity of randomised controlled trials: “To whom do the results of this trial apply?”
Rothwell, P. M. (2005) · The Lancet · doi:10.1016/s0140-6736(04)17670-8 · Cited by 2,544 · cited 1×
“For instance, if one set of studies was carried out on a younger cohort of patients, with a lower drug dosage, or with shorter duration of use, or relied on passive ascertainment of adverse effects data [6,17,52,134], it might be expected that the magnitude of any adverse effects recorded would be lower.”
ResultAnalysing Data and Undertaking Meta‐Analyses
Deeks, J. J., Higgins, J. P., & Altman, D. G. (2008) · Cochrane Handbook for Systematic Reviews of Interventions · doi:10.1002/9780470712184.ch9 · Cited by 1,800
Better Reporting of Harms in Randomized Trials: An Extension of the CONSORT Statement
Ioannidis, J. P. A., Evans, S. J. W., Gøtzsche, P. C., O'Neill, R. T., Altman, D. G., Schulz, K., Moher, D., & for the CONSORT Group* (2004) · Annals of Internal Medicine · doi:10.7326/0003-4819-141-10-200411160-00009 · Cited by 1,593
Evidence-based medicine : how to practice and teach EBM
Straus, S. E. (2005) · Cited by 1,367
Meta-analysis of the Impact of 9 Medication Classes on Falls in Elderly Persons
Woolcott, J. C. (2009) · Archives of Internal Medicine · doi:10.1001/archinternmed.2009.357 · Cited by 1,326
Drugs and Falls in Older People: A Systematic Review and Meta‐analysis: I. Psychotropic Drugs
Leipzig, R. M., Cumming, R. G., & Tinetti, M. E. (1999) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1999.tb01898.x · Cited by 1,153
McGettigan, P., & Henry, D. (2006) · JAMA · doi:10.1001/jama.296.13.jrv60011 · Cited by 1,096
Meta-analysis Spurious precision? Meta-analysis of observational studies
Egger, M., Schneider, M., & Smith, G. D. (1998) · BMJ · doi:10.1136/bmj.316.7125.140 · Cited by 997
Fitzpatrick, Shortall, Sculpher, Murray, Morris, Lodge, Dawson, Carr, Britton, & Briggs (1998) · Health Technology Assessment · doi:10.3310/hta2200 · Cited by 969
Hormone replacement therapy and endometrial cancer risk: A meta-analysis
GRADY, D., GEBRETSADIK, T., KERLIKOWSKE, K., ERNSTER, V, & PETITTI, D. (1995) · Obstetrics & Gynecology · doi:10.1016/0029-7844(94)00383-o · Cited by 962
Comparison of Evidence of Treatment Effects in Randomized and Nonrandomized Studies
Ioannidis, J. P. A. (2001) · JAMA · doi:10.1001/jama.286.7.821 · Cited by 919
Leipzig, R. M., Cumming, R. G., & Tinetti, M. E. (1999) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1999.tb01899.x · Cited by 633
Canonico, M., Plu-Bureau, G., Lowe, G. D. O., & Scarabin, P.-Y. (2008) · BMJ · doi:10.1136/bmj.39555.441944.be · Cited by 569 · cited 1×
“indication of heterogeneity; only five (15%) [6,117,130,131] of the”
BackgroundLong-term use of thiazolidinediones and fractures in type 2 diabetes: a meta-analysis
Loke, Y. K., Singh, S., & Furberg, C. D. (2009) · Canadian Medical Association Journal · doi:10.1503/cmaj.080486 · Cited by 551
Self-citationCervical cancer and use of hormonal contraceptives: a systematic review
Smith, J. S., Green, J., de Gonzalez, A. B., Appleby, P., Peto, J., Plummer, M., Franceschi, S., & Beral, V. (2003) · The Lancet · doi:10.1016/s0140-6736(03)12949-2 · Cited by 544
When are observational studies as credible as randomised trials?
Vandenbroucke, J. P. (2004) · The Lancet · doi:10.1016/s0140-6736(04)16261-2 · Cited by 538 · cited 1×
“As the intended effects of treatment are more likely to occur than adverse effects and to occur within the trial time frame, RCTs may not be large enough, or have a sufficient follow-up to identify rare, long-term adverse effects, or adverse effects that occur after the drug has been discontinued [1–3,9,13,15,16,18,19…”
BackgroundKeltner, N. L. (1993) · Cited by 530
Siegel, C. A., Marden, S. M., Persing, S. M., Larson, R. J., & Sands, B. E. (2009) · Clinical Gastroenterology and Hepatology · doi:10.1016/j.cgh.2009.01.004 · Cited by 511
Caesarean delivery and risk of atopy and allergic disesase: meta‐analyses
Bager, P., Wohlfahrt, J., & Westergaard, T. (2008) · Clinical & Experimental Allergy · doi:10.1111/j.1365-2222.2008.02939.x · Cited by 501
Cutler, C., Giri, S., Jeyapalan, S., Paniagua, D., Viswanathan, A., & Antin, J. H. (2001) · Journal of Clinical Oncology · doi:10.1200/jco.2001.19.16.3685 · Cited by 432
Observational research, randomised trials, and two views of medical science
Vandenbroucke, J. P. (2008) · PLoS Medicine · doi:10.1371/journal.pmed.0050067 · Cited by 416
Tramèr, M. R., Moore, A. R., Reynolds, J. M. D., & McQuay, H. J. (2000) · Pain · doi:10.1016/s0304-3959(99)00267-5 · Cited by 402
Ischemic Stroke Risk With Oral Contraceptives: A Meta-analysis
Gillum, L. A., Mamidipudi, S. K., & Johnston, S. C. (2000) · JAMA · doi:10.1001/jama.284.1.72 · Cited by 402
Assessment and Control for Confounding by Indication in Observational Studies
Psaty, B. M., Koepsell, T. D., Lin, D., Weiss, N. S., Siscovick, D. S., Rosendaal, F. R., Pahor, M., & Furberg, C. D. (1999) · Journal of the American Geriatrics Society · doi:10.1111/j.1532-5415.1999.tb01603.x · Cited by 363 · cited 1×
“Nevertheless, observational study designs may sometimes be the only available source of data for a particular adverse effect, and are commonly used in evaluating adverse effects [1,9,13,52,58,59].”
BackgroundAdverse effects of medical cannabinoids: a systematic review
Wang, T., Collet, J.-P., Shapiro, S., & Ware, M. A. (2008) · Canadian Medical Association Journal · doi:10.1503/cmaj.071178 · Cited by 357 · cited 1×
“On further detailed evaluation, 33 of these articles either compared different types of observational studies to one another (for example, cohort studies versus case control studies) or compared only the incidence of adverse effects (without reporting the RR/OR) in those receiving the intervention according to type of…”
BackgroundDolovich, L. R., Addis, A., Vaillancourt, J. M. R., Power, J. D. B., Koren, G., & Einarson, T. R. (1998) · BMJ · doi:10.1136/bmj.317.7162.839 · Cited by 353
Postmarketing Surveillance and Adverse Drug Reactions: Current Perspectives and Future Needs
Brewer, T., & Colditz, G. A. (1999) · JAMA · doi:10.1001/jama.281.9.824 · Cited by 327
MacLehose, Reeves, Harvey, Sheldon, Russell, & Black (2000) · Health Technology Assessment · doi:10.3310/hta4340 · Cited by 311
Users' Guides to the Medical Literature: IV. How to Use an Article About Harm
Levine, M. (1994) · JAMA · doi:10.1001/jama.1994.03510440075037 · Cited by 311
Choosing between randomised and non-randomised studies: a systematic review.
Britton, McKee, Black, McPherson, Sanderson, & Bain (1998) · Health Technology Assessment · doi:10.3310/hta2130 · Cited by 308
Thiazolidinediones and Heart Failure
Singh, S., Loke, Y. K., & Furberg, C. D. (2007) · Diabetes Care · doi:10.2337/dc07-0141 · Cited by 262
Self-citationSystematic reviews of adverse effects: framework for a structured approach
Loke, Y. K., Price, D., Herxheimer, A., & the Cochrane Adverse Effects Methods Group (2007) · BMC Medical Research Methodology · doi:10.1186/1471-2288-7-32 · Cited by 253
Self-citationAdverse drug event monitoring at the food and drug administration: Your report can make a difference
Ahmad, S. R. (2003) · Journal of General Internal Medicine · doi:10.1046/j.1525-1497.2003.20130.x · Cited by 249
Ofman, J. J., MacLean, C. H., Straus, W. L., Morton, S. C., Berger, M. L., Röth, E., & Shekelle, P. (2002) · PubMed · Cited by 239
Levine, M. (1994) · JAMA · doi:10.1001/jama.271.20.1615 · Cited by 233
Propofol and bradycardia: causation, frequency and severity
Tramèr, M. R., Moore, R. A., & McQuay, H. J. (1997) · British Journal of Anaesthesia · doi:10.1093/bja/78.6.642 · Cited by 230
Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
Papanikolaou, P. N. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.050873 · Cited by 229 · cited 9×
“) Two of the 19 articles were methodological evaluations with the main aim of assessing the influence of study characteristics (including study design) on the measurement of adverse effects [6,127], whereas the remaining 17 were systematic reviews within which subgroup analysis by study design was embedded [114– 126,1…”
BackgroundResultInfluentialStatins and risk of cancer: A systematic review and metaanalysis
Browning, D. R. L., & Martin, R. M. (2007) · International Journal of Cancer · doi:10.1002/ijc.22366 · Cited by 228
Safety of ultrasonography in pregnancy: WHO systematic review of the literature and meta‐analysis
Torloni, M. R., Vedmedovska, N., Merialdi, M., Betrán, A. P., Allen, T., González, R., & Platt, L. D. (2009) · Ultrasound in Obstetrics & Gynecology · doi:10.1002/uog.6328 · Cited by 222
Detection, verification, and quantification of adverse drug reactions
Stricker, B. H., & Psaty, B. M. (2004) · BMJ · doi:10.1136/bmj.329.7456.44 · Cited by 221
Adverse events in coronary artery bypass graft (CABG) trials: a systematic review and analysis
Nalysnyk, L., Fahrbach, K., Reynolds, M. W., Zhao, S. Z., & Ross, S. (2003) · Heart · doi:10.1136/heart.89.7.767 · Cited by 213
Psychotropic Medications and the Risk of Fracture: A Meta-Analysis
Takkouche, B., Montes-Mart??nez, A., Gill, S. S., & Etminan, M. (2007) · Drug Safety · doi:10.2165/00002018-200730020-00006 · Cited by 204 · cited 1×
“On further detailed evaluation, 33 of these articles either compared different types of observational studies to one another (for example, cohort studies versus case control studies) or compared only the incidence of adverse effects (without reporting the RR/OR) in those receiving the intervention according to type of…”
BackgroundRief, W., Nestoriuc, Y., von Lilienfeld-Toal, A., Dogan, I., Schreiber, F., Hofmann, S. G., Barsky, A. J., & Avorn, J. (2009) · Drug Safety · doi:10.2165/11316580-000000000-00000 · Cited by 202 · cited 1×
“Empirical studies have shown that many RCTs fail to provide detailed adverse effects data, that the quality of those that do report adverse effects is poor [6,17–31], and that the reporting may be strongly influenced by expectations of investigators and patients [32].”
BackgroundLOKE, Y. K., TRIVEDI, A. N., & SINGH, S. (2008) · Alimentary Pharmacology & Therapeutics · doi:10.1111/j.1365-2036.2007.03541.x · Cited by 193
Self-citationHormone replacement therapy and the risk of epithelial ovarian carcinoma: a meta-analysis
Garg, P. (1998) · Obstetrics & Gynecology · doi:10.1016/s0029-7844(98)00139-2 · Cited by 182
SACKETT, D. L. (1969) · American Journal of Epidemiology · doi:10.1093/oxfordjournals.aje.a120921 · Cited by 173
Chan, W.-S., Ray, J., Wai, E. K., Ginsburg, S., Hannah, M. E., Corey, P. N., & Ginsberg, J. S. (2004) · Archives of Internal Medicine · doi:10.1001/archinte.164.7.741 · Cited by 171
Reporting of Adverse Effects in Clinical Trials Should Be Improved
Edwards, J. E., McQuay, H. J., Moore, R. A., & Collins, S. L. (1999) · Journal of Pain and Symptom Management · doi:10.1016/s0885-3924(99)00093-7 · Cited by 167
Potshuma, W. F. M., Westendorp, R. G. J., & Vandenbroucke, J. P. (1994) · BMJ · doi:10.1136/bmj.308.6939.1268 · Cited by 162
Adverse Events Associated With Pediatric Spinal Manipulation: A Systematic Review
Vohra, S., Johnston, B. C., Cramer, K., & Humphreys, K. (2006) · PEDIATRICS · doi:10.1542/peds.2006-1392 · Cited by 159
Challenges in Systematic Reviews That Assess Treatment Harms
Chou, R., & Helfand, M. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-142-12_part_2-200506211-00009 · Cited by 147
Prentice, R. L., Chlebowski, R. T., Stefanick, M. L., Manson, J. E., Langer, R. D., Pettinger, M. B., Hendrix, S. L., Hubbell, F. A., Kooperberg, C. L., Kuller, L. H., et al. (2008) · American Journal of Epidemiology · doi:10.1093/aje/kwn090 · Cited by 142
Meta-analysis of rare and adverse event data
Sutton, A. J., Cooper, N. J., Lambert, P. C., Jones, D. R., Abrams, K. R., & Sweeting, M. J. (2002) · Expert Review of Pharmacoeconomics & Outcomes Research · doi:10.1586/14737167.2.4.367 · Cited by 141
Reporting of adverse drug reactions in randomised controlled trials – a systematic survey
Loke, Y. K., & Derry, S. (2001) · BMC Clinical Pharmacology · doi:10.1186/1472-6904-1-3 · Cited by 138
Self-citationHordijk-Trion, M., Lenzen, M., Wijns, W., de Jaegere, P., Simoons, M. L., Scholte op Reimer, W. J. M., Bertrand, M. E., Mercado, N., & Boersma, E. (2006) · European Heart Journal · doi:10.1093/eurheartj/ehi731 · Cited by 136
McAlister, F. A., Clark, H. D., Wells, P. S., & Laupacis, A. (1998) · Journal of British Surgery · doi:10.1046/j.1365-2168.1998.00698.x · Cited by 131
Bollini, P. (1992) · Archives of Internal Medicine · doi:10.1001/archinte.152.6.1289 · Cited by 120
Assessing the Neonatal Safety of Indomethacin Tocolysis: A Systematic Review With Meta-Analysis
Loe, S. M., Sanchez‐Ramos, L., & Kaunitz, A. M. (2005) · Obstetrics and Gynecology · doi:10.1097/01.aog.0000168622.56478.df · Cited by 109
Oral contraceptives and the risk of subarachnoid hemorrhage
Johnston, S. C., Colford, J. M., & Gress, D. R. (1998) · Neurology · doi:10.1212/wnl.51.2.411 · Cited by 109
Petitti, D. B. (1994) · Annals of Epidemiology · doi:10.1016/1047-2797(94)90056-6 · Cited by 106
Col, N. F., Kim, J. A., & Chlebowski, R. T. (2005) · Breast Cancer Research · doi:10.1186/bcr1035 · Cited by 105
Balancing benefits and harms in health care
Cuervo, L. G., & Clarke, M. (2003) · BMJ · doi:10.1136/bmj.327.7406.65 · Cited by 104
Alghamdi, A. A., Moussa, F., & Fremes, S. E. (2007) · Journal of Cardiac Surgery · doi:10.1111/j.1540-8191.2007.00402.x · Cited by 104
Scott, P. A., Kingsley, G. H., Smith, C. M., Choy, E., & Scott, D. L. (2007) · Annals of the Rheumatic Diseases · doi:10.1136/ard.2006.068650 · Cited by 104
The Role of Meta-analysis in the Regulatory Process for Foods, Drugs, and Devices
Berlin, J. A. (1999) · JAMA · doi:10.1001/jama.281.9.830 · Cited by 101
Zuluaga, A. F., Agudelo, M., Rodríguez, C. A., & Vesga, Ó. (2009) · BMC Clinical Pharmacology · doi:10.1186/1472-6904-9-1 · Cited by 100
Population-Based Studies of Adverse Drug Effects
Ray, W. A. (2003) · New England Journal of Medicine · doi:10.1056/nejmp038145 · Cited by 98
Reporting of safety data from randomised trials
Ioannidis, J. P. A., & Contopoulos‐Ioannidis, D. G. (1998) · The Lancet · doi:10.1016/s0140-6736(05)79825-1 · Cited by 97
Shikata, S., Nakayama, T., Noguchi, Y., Taji, Y., & Yamagishi, H. (2006) · Annals of Surgery · doi:10.1097/01.sla.0000225356.04304.bc · Cited by 91
Principles of epidemiological research on adverse and beneficial drug effects
Jick, H., Rodríguez, L. A. G., & Perez‐Gutthann, S. (1998) · The Lancet · doi:10.1016/s0140-6736(98)04350-5 · Cited by 90
Epidemiology overview of gastrointestinal and renal toxicity of NSAIDs.
Henry, D. A., & McGettigan, P. (2003) · PubMed · Cited by 88
The HRT controversy: observational studies and RCTs fall in line
Vandenbroucke, J. P. (2009) · The Lancet · doi:10.1016/s0140-6736(09)60708-x · Cited by 88
Balancing benefits and harms: the example of non-steroidal anti-inflammatory drugs
Dieppe, P. A., Bartlett, C., Davey, P., Doyal, L., & Ebrahim, S. B. J. (2004) · BMJ · doi:10.1136/bmj.329.7456.31 · Cited by 87
Safety Reporting in Randomized Trials of Mental Health Interventions
Papanikolaou, P. N., Churchill, R., Wahlbeck, K., Ioannidis, J. P. A., & The EU-PSI Project (2004) · American Journal of Psychiatry · doi:10.1176/appi.ajp.161.9.1692 · Cited by 84
What is the best evidence for determining harms of medical treatment?
Vandenbroucke, J. P. (2006) · Canadian Medical Association Journal · doi:10.1503/cmaj.051484 · Cited by 84
Scott, P. A., Kingsley, G. H., & Scott, D. L. (2008) · European Journal of Heart Failure · doi:10.1016/j.ejheart.2008.07.013 · Cited by 83
Douketis, J. D., Ginsberg, J. S., Holbrook, A., Crowther, M., Duku, E. K., & Burrows, R. F. (1997) · Archives of Internal Medicine · doi:10.1001/archinte.1997.00440350022002 · Cited by 80
Availability of large-scale evidence on specific harms from systematic reviews of randomized trials.
Papanikolaou, P. N., & Ioannidis, J. P. A. (2004) · The American Journal of Medicine · doi:10.1016/j.amjmed.2004.04.026 · Cited by 79
Randomised controlled trials for policy interventions: a review of reviews and meta-regression
Oliver, S., Bagnall, A., Thomas, J., Shepherd, J., Sowden, A., White, I., Dinnes, J., Rees, R., Colquitt, J., Oliver, K., et al. (2010) · Health Technology Assessment · doi:10.3310/hta14160 · Cited by 76
Epidemiological assessment of drug-induced disease
Kaufman, D. W., & Shapiro, S. (2000) · The Lancet · doi:10.1016/s0140-6736(00)02826-9 · Cited by 72
Adverse events: the more you search, the more you find
Ioannidis, J. P. A., Mulrow, C. D., & Goodman, S. N. (2006) · Annals of Internal Medicine · doi:10.7326/0003-4819-144-4-200602210-00013 · Cited by 69
Clarke, A., Deeks, J. J., & Shakir, S. (2006) · Drug Safety · doi:10.2165/00002018-200629020-00008 · Cited by 69
How have the Selective Serotonin Reuptake Inhibitor Antidepressants Affected Suicide Mortality?
Hall, W. D., & Lucke, J. (2006) · Australian & New Zealand Journal of Psychiatry · doi:10.1080/j.1440-1614.2006.01917.x · Cited by 67
Schwarz, E. B., Moretti, M. E., Nayak, S., & Koren, G. (2008) · Drug Safety · doi:10.2165/00002018-200831090-00006 · Cited by 63
Aagaard, L., & Hansen, E. H. (2009) · BMC Clinical Pharmacology · doi:10.1186/1472-6904-9-4 · Cited by 62
Loke, Y. K., Derry, S., & Aronson, J. K. (2004) · British Journal of Clinical Pharmacology · doi:10.1111/j.0306-5251.2003.02055.x · Cited by 59
Self-citationHow Can We Improve the Assessment of Safety in Child and Adolescent Psychopharmacology?
Vitiello, B., Riddle, M. A., Walkup, J. T., Labellarte, M. J., VARIPATIS, E., Greenhill, L. L., Davies, M., Capasso, L., March, J. S., Levine, J., et al. (2003) · Journal of the American Academy of Child & Adolescent Psychiatry · doi:10.1097/01.chi.0000046840.90931.36 · Cited by 58
Methodological issues in the epidemiological study of the teratogenicity of drugs
Källén, B. A. J. (2005) · Congenital Anomalies · doi:10.1111/j.1741-4520.2005.00062.x · Cited by 56
Benefits and harms of drug treatments
Vandenbroucke, J. P. (2004) · BMJ · doi:10.1136/bmj.329.7456.2 · Cited by 53
Colorectal cancer and oestrogen replacement therapy: A meta‐analysis of epidemiological studies
MacLennan, S. C., MacLennan, A. H., & Ryan, P. (1995) · The Medical Journal of Australia · doi:10.5694/j.1326-5377.1995.tb140014.x · Cited by 52
Ethgen, M., Boutron, I., Baron, G., Giraudeau, B., Sibilia, J., & Ravaud, P. (2005) · Annals of Internal Medicine · doi:10.7326/0003-4819-143-1-200507050-00006 · Cited by 49 · cited 1×
“Empirical studies have shown that many RCTs fail to provide detailed adverse effects data, that the quality of those that do report adverse effects is poor [6,17–31], and that the reporting may be strongly influenced by expectations of investigators and patients [32].”
BackgroundInterpreting meta-analysis in systematic reviews
Perera, R., & Heneghan, C. (2008) · Evidence-Based Medicine · doi:10.1136/ebm.13.3.67 · Cited by 49
Limited knowledge on progestogen‐only contraception and risk of venous thromboembolism
BERGENDAL, A., ODLIND, V., PERSSON, I., & KIELER, H. (2009) · Acta Obstetricia et Gynecologica Scandinavica · doi:10.1080/00016340902730375 · Cited by 49
Drug-Related Adverse Events: A Readers' Guide to Assessing Literature Reviews and Meta-analyses
Ross, S. D. (2001) · Archives of Internal Medicine · doi:10.1001/archinte.161.8.1041 · Cited by 47
Capurso, G., Schünemann, H. J., Terrenato, I., MORETTI, A., Koch, M., Muti, P. C., Capurso, L., & Fave, G. D. (2007) · Alimentary Pharmacology & Therapeutics · doi:10.1111/j.1365-2036.2007.03495.x · Cited by 45
Oral contraceptives and venous thromboembolism: a quantitative discussion of the uncertainties
Koster, T., SMALL, R. ‐., Rosendaal, F. R., & Helmerhorst, F. M. (1995) · Journal of Internal Medicine · doi:10.1111/j.1365-2796.1995.tb00896.x · Cited by 43
Assessment of the Risk for Venous Thromboembolism Among Users of Hormone Replacement Therapy
Oger, E., & Scarabin, P. (1999) · Drugs & Aging · doi:10.2165/00002512-199914010-00004 · Cited by 43
In vitro fertilization and breast cancer risk: a review.
Salhab, M., Al Sarakbi, W., & Mokbel, K. (2005) · PubMed · Cited by 43
How to use articles about harm
Meade, M. O., Cook, D. J., Kernerman, P., & Bernard, G. R. (1997) · Critical Care Medicine · doi:10.1097/00003246-199711000-00034 · Cited by 41
Challenges in Using Observational Studies to Evaluate Adverse Effects of Treatment
Hughes, M. D., & Williams, P. L. (2007) · New England Journal of Medicine · doi:10.1056/nejmp078038 · Cited by 37
Chou, R., Fu, R. F., Carson, S., Saha, S. S., & Helfand, M. (2006) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2006.02.021 · Cited by 33
Quantifying Adverse Drug Events: Are Systematic Reviews the Answer?
Etminan, M., Carleton, B., & Rochon, P. A. (2004) · Drug Safety · doi:10.2165/00002018-200427110-00001 · Cited by 30
Evaluating the safety of medicines, with particular reference to contraception
Skegg, D. C. G. (2001) · Statistics in Medicine · doi:10.1002/sim.1077 · Cited by 29
Getting It Right: Being Smarter about Clinical Trials
Kramer, B. S., Wilentz, J., Alexander, D. F., Burklow, J. T., Friedman, L. M., Hodes, R. J., Kirschstein, R. L., Patterson, A., Rodgers, G. P., & Straus, S. E. (2006) · PLoS Medicine · doi:10.1371/journal.pmed.0030144 · Cited by 29
Pedersen, A. T. (2003) · Human Reproduction · doi:10.1093/humrep/deg435 · Cited by 29
Some concerns about adverse event reporting in randomized clinical trials.
Yazıcı, Y. (2008) · PubMed · Cited by 29
Published randomized controlled trials of drug therapy for dementia often lack complete data on harm
Lee, P. E., Fischer, H. D., Rochon, P. A., Gill, S. S., Herrmann, N., Bell, C. M., Sykora, K., & Anderson, G. M. (2008) · Journal of Clinical Epidemiology · doi:10.1016/j.jclinepi.2007.09.012 · Cited by 27
Synthesising licensing data to assess drug safety
McPherson, K., & Hemminki, E. (2004) · BMJ · doi:10.1136/bmj.328.7438.518 · Cited by 27
Methodological issues arising from systematic reviews of the evidence of safety of vaccines
Price, D., Jefferson, T., & Demicheli, V (2004) · Vaccine · doi:10.1016/j.vaccine.2004.01.009 · Cited by 23
Adverse Drug Reactions, 2nd edn
Franklin, P. (2007) · Journal of Advanced Nursing · doi:10.1111/j.1365-2648.2007.04185.x · Cited by 21
Reporting adverse events in randomized controlled trials
Nuovo, J., & Sather, C. (2006) · Pharmacoepidemiology and Drug Safety · doi:10.1002/pds.1310 · Cited by 20
Interpreting measures of treatment effect.
Davies, H. T. O. (1998) · PubMed · Cited by 19
Relation between experimental and non-experimental study designs. HB vaccines: a case study.
Jefferson, T., & Demicheli, V. (1999) · Journal of Epidemiology & Community Health · doi:10.1136/jech.53.1.51 · Cited by 17
Hormone Replacement Therapy and Risk of Venous Thromboembolism
Miller, J. M., Bks, C., & Nelson, H. D. (2002) · Cited by 16
Anello, C., & O’Neill, R. T. (1996) · Clinical Research and Regulatory Affairs · doi:10.3109/10601339609019625 · Cited by 15
Principles of research: limitations of non-randomized studies
Reeves, B. C. (2006) · Surgery (Oxford) · doi:10.1053/j.mpsur.2006.06.003 · Cited by 15
Kim, J., Evans, S. J. W., Smeeth, L. L., & Pocock, S. (2006) · International Journal of Epidemiology · doi:10.1093/ije/dyi285 · Cited by 15
Inadequate reporting of trials compromises the applicability of systematic reviews
Gartlehner, G., Thieda, P., Hansen, R. A., Morgan, L. C., Shumate, J. A., & Nissman, D. B. (2009) · International Journal of Technology Assessment in Health Care · doi:10.1017/s0266462309990122 · Cited by 11
Meta-analysis and adverse drug reactions.
Mittmann, N., Liu, B. A., Knowles, S. R., & Shear, N. H. (1999) · Europe PMC (PubMed Central) · Cited by 9
Jefferson, T., & Traversa, G. (2002) · Journal of Medical Virology · doi:10.1002/jmv.10095 · Cited by 9
Ravaud, P., & Tubach, F. (2005) · Joint Bone Spine · doi:10.1016/j.jbspin.2005.10.003 · Cited by 9
Conflicting clinical trial data: a lesson from albumin.
Martin, G. S. (2005) · Critical Care · doi:10.1186/cc3931 · Cited by 4
Levine, M. (1997) · American Journal of Hypertension · doi:10.1016/s0895-7061(97)00227-6 · Cited by 4
Pharmacoepidemiology in Safety Evaluations of Newly Approved Medications
Gutterman, E. M. (2004) · Drug Information Journal · doi:10.1177/009286150403800109 · Cited by 4
Completeness of Safety Reporting in Randomized Trials: An Evaluation of 7 Medical Areas
Ioannidis, J. P. A., & Lau, J. (2001) · JAMA · doi:10.1001/jama.285.4.437 · Cited by 3
Research works printed but not linked: 11
- RF Jacob, 2000, How to evaluate a dental article about harm., J Prosthet Dent, 84, 8, 10.1067/mpr.2000.108496 · doi:10.1067/mpr.2000.108496
- D Henry, 1999, Meta-analysis-its role in assessing drug safety., Pharmacoepidemiol Drug Saf, 8, 167, 10.1002/(SICI)1099-1557(199905/06)8:3<167::AID-PDS418>3.0.CO;2-F · doi:10.1002/(sici)1099-1557(199905/06)8:3<167::aid-pds418>3.0.co;2-f
- P Buekens, 2001, Invited commentary: rare side effects of obstetric interventions: are observational studies good enough?, [comment] Am J Epidemiol, 153, 108, 10.1093/aje/153.2.108 · doi:10.1093/aje/153.2.108
- KL Hyrich, 2005, Assessing the safety of biologic therapies in rheumatoid arthritis: the challenges of study design., J, Rheumatol, 48
- K Beard, 2006, Introduction., 10.1353/itx.2006.0000 · doi:10.1353/itx.2006.0000
- M McDonagh, 2006, The impact of including non-randomized studies in a systematic review: a case study [abstract].
- J Shepherd, 2006, Sometimes similar, sometimes different: a systematic review of meta-analyses of random and non-randomized policy intervention studies.
- P Jensen, 2002, Postherniorrhaphy urinary retention—effect of local, regional, and general anesthesia: a review., Reg Anesth Pain Med, 27, 612
- G Jones, 1999, Water fluoridation, bone mass and fracture: a quantitative overview of the literature., Aust N Z J Public Health, 23, 34, 10.1111/j.1467-842X.1999.tb01202.x · doi:10.1111/j.1467-842x.1999.tb01202.x
- SD Ross, 1998, Safety of GM-CSF in patients with AIDS: A review of the literature., Pharmacotherapy, 18, 1290, 10.1002/j.1875-9114.1998.tb03150.x · DOI
- JJ Ofman, 2002, A meta-analysis of severe upper gastrointestinal complications of non-steroidal anti-inflammatory drugs., J Rheumatol, 29, 804
Cited by
ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions
Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., Boutron, I., et al. (2016) · BMJ · doi:10.1136/bmj.i4919 · Cited by 20,329
“21 We believe that evidence from NRSI should complement that from randomised trials, such as in providing evidence about effects on rare and adverse outcomes and long term effects to be balanced against the outcomes more readily addressed in randomised trials.”
Self-citationChang, S.-H., Stoll, C. R. T., Song, J., Varela, J. E., Eagon, C. J., & Colditz, G. A. (2014) · JAMA Surgery · doi:10.1001/jamasurg.2013.3654 · Cited by 1,688
Gloy, V. L., Briel, M., Bhatt, D. L., Kashyap, S. R., Schauer, P. R., Mingrone, G., Bucher, H. C., & Nordmann, A. J. (2013) · BMJ · doi:10.1136/bmj.f5934 · Cited by 1,351
Interpretation and Impact of Real-World Clinical Data for the Practicing Clinician
Blonde, L., Khunti, K., Harris, S. B., Meizinger, C., & Skolnik, N. S. (2018) · Advances in Therapy · doi:10.1007/s12325-018-0805-y · Cited by 714
McGettigan, P., & Henry, D. (2011) · PLoS Medicine · doi:10.1371/journal.pmed.1001098 · Cited by 571
Metadata
- Authors
- Golder, S., Loke, Y. K., & Bland, M.
- Venue
- PLoS Medicine
- Publisher
- Public Library of Science (PLoS)
- Published
- 2011-05-03
- Volume
- 8
- Issue
- 5
- Pages
- e1001026
- ISSN
- 1549-1277 (ISSN-L)
- 1549-1676
- OpenAlex
- W1989467621
- Cited by
- 269
- References
- 127
- FWCI
- 30.73
- Type
- journal article
- Language
- en
- Open access
- gold
About these checks
Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.
Statistics and arithmetic are recomputed, not judged.
Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.
What counts as a claim ›How we measure confidence ›How we check this ›
Are you an author, or do you see a problem with this page? Request a review
Data sources
Bibliographic data: Semantic Scholar, OpenAlex, Crossref.
Contains information from the Semantic Scholar Academic Graph Datasets, made available under the ODC Attribution License (ODC-BY 1.0).
Semantic Scholar’s data is described in: Kinney, R., et al. (2023). The Semantic Scholar Open Data Platform. arXiv:2301.10140.