CoChat
Sign Up
Cited by 229Meta-analysis and systematic reviewsAdvanced Causal Inference TechniquesPharmacovigilance and Adverse Drug Reactions

Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies

Open access
Papanikolaou, P. N.·2006·Canadian Medical Association Journal
DOI 10.1503/cmaj.050873PMC PMC1389826
Open this paper in CoChat — free accountOpen PDF

Abstract

BACKGROUND: Information on major harms of medical interventions comes primarily from epidemiologic studies performed after licensing and marketing. Comparison with data from large-scale randomized trials is occasionally feasible. We compared evidence from randomized trials with that from epidemiologic studies to determine whether they give different estimates of risk for important harms of medical interventions. METHODS: We targeted well-defined, specific harms of various medical interventions for which data were already available from large-scale randomized trials (> 4000 subjects). Nonrandomized studies involving at least 4000 subjects addressing these same harms were retrieved through a search of MEDLINE. We compared the relative risks and absolute risk differences for specific harms in the randomized and nonrandomized studies. RESULTS: Eligible nonrandomized studies were found for 15 harms for which data were available from randomized trials addressing the same harms. Comparisons of relative risks between the study types were feasible for 13 of the 15 topics, and of absolute risk differences for 8 topics. The estimated increase in relative risk differed more than 2-fold between the randomized and nonrandomized studies for 7 (54%) of the 13 topics; the estimated increase in absolute risk differed more than 2-fold for 5 (62%) of the 8 topics. There was no clear predilection for randomized or nonrandomized studies to estimate greater relative risks, but usually (75% [6/8]) the randomized trials estimated larger absolute excess risks of harm than the nonrandomized studies did. INTERPRETATION: Nonrandomized studies are often conservative in estimating absolute risks of harms. It would be useful to compare and scrutinize the evidence on harms obtained from both randomized and nonrandomized studies.

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Integrity signals

Registry

Crossref · 10.1503/cmaj.050873

Cites

57 of 57 references verified0 citation errors · 0 not checkable

Statistics

Claims the paper makes

The claims and statistics of this paper haven’t been checked yet.

Run the check — free, no account needed. It reads the open-access text and checks the sources it cites.

A few free checks a day · papers already verified open instantly · results are public

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Observational studies are more suitable to detect rare or late adverse effects of treatments.

    “For example, observational studies are more suitable to detect rare or late adverse effects of treatments, and are more likely to provide an indication of what is achieved in daily medical practice [3].”

    From The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies (2007) · cites this paper as [3]

    The Interpretation (page unavailable) says observational studies have been the standard for identifying serious but uncommon harms, supporting the claim about rare adverse effects. The paper discusses harms detected after marketing, but does not establish that observational studies are more suitable for detecting late-onset effects. It also does not establish the citing sentence’s comparison about indicating what is achieved in daily practice. (LLM, conf=0.85)

    Experimental — accuracy not yet measured

  • Observational studies are more likely to provide an indication of what is achieved in daily medical practice.

    “For example, observational studies are more suitable to detect rare or late adverse effects of treatments, and are more likely to provide an indication of what is achieved in daily medical practice [3].”

    From The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for Reporting Observational Studies (2007) · cites this paper as [3]

    The Interpretation section describes observational studies as a standard way to identify serious, uncommon harms, but does not say they are more likely to show what is achieved in daily medical practice. It cautions that some harms in everyday practice may be more common than observational studies suggest. Searches for daily practice and real-world effectiveness found no passage supporting the comparative claim. (LLM, conf=0.74)

    Experimental — accuracy not yet measured

  • Observational studies are more suitable to detect rare or late adverse effects of treatments, and are more likely to provide an indication of what is achieved in daily medical practice.

    “For example, observational studies are more suitable to detect rare or late adverse effects of treatments, and are more likely to provide an indication of what is achieved in daily medical practice [3].”

    From The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: Guidelines for reporting observational studies (2014) · cites this paper as [3]

Self-check

Percentages, sums and reported differences re-computed from the paper’s own numbers.

The self-check needs the paper’s full text.

Cites

Flagged first, then by citations

Research works printed but not linked: 3

  1. Stephens MD, Talbot JC, Routledge PA, editors. The detection of new adverse reac- tions. 4th ed. London (UK): Macmillan Reference; 1998.
  2. Windham GC, Shaw GM, Todoroff K, et al. Miscarriage and use of multi-vitamins or folic acid. Am J Med Genet 2000;90:261-2.
  3. Werler MM, Cragan JD, Wasserman CR, et al. Multivitamin supplementation and multiple births. Am J Med Genet 1997;71:93-6.

Cited by

Sorted by citations

Related

Authors
Papanikolaou, P. N.
Affiliations
University of Ioannina (GR)
Corresponding author
Papanikolaou, P. N.
Venue
Canadian Medical Association Journal
Publisher
CMA Joule Inc.
Published
2006-02-28
Volume
174
Issue
5
Pages
635-641
ISSN
  • 0820-3946 (ISSN-L)
  • 1488-2329
DOI
10.1503/cmaj.050873
PMC
PMC1389826
OpenAlex
W2156294597
Cited by
229
References
57
Citations by year
2026: 1 · 2025: 2 · 2024: 8 · 2023: 4 · 2022: 6 · 2021: 8 · 2020: 3 · 2019: 8 · 2018: 7 · 2017: 7 · 2016: 12 · 2015: 12 · 2014: 16 · 2013: 30 · 2012: 18
FWCI
24.29
Type
journal article
Language
en
Open access
Locations
  • Canadian Medical Association Journal (publishedVersion)
  • Europe PMC (PubMed Central) (publishedVersion)
  • PubMed Central (publishedVersion)
  • PubMed (publishedVersion)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Physical Sciences › Mathematics › Statistics and Probability › Advanced Causal Inference Techniques
  • Life Sciences › Pharmacology, Toxicology and Pharmaceutics › Toxicology › Pharmacovigilance and Adverse Drug Reactions
Keywords
non-randomized studies, relative risk, risk difference, risk estimation, post-marketing surveillance
MeSH terms
Humans, Risk, Surgical Procedures, Operative, Vaccines, Vitamins, Bias, Epidemiologic Studies, Randomized Controlled Trials as Topic, Evidence-Based Medicine, Drug-Related Side Effects and Adverse Reactions

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review