CoChat
Sign Up
Cited by 70Meta-analysis and systematic reviewsClinical practice guidelines implementationStatistical Methods in Epidemiology

Systematic reviews experience major limitations in reporting absolute effects

Open access
Alonso-Coello, Pablo, Carrasco-Labra, Alonso, Brignardello-Petersen, Romina, Neumann, Ignacio, Akl, Elie A., Vernooij, Robin W.M., Johnston, Brad C., Sun, Xin +19 more·2016·Journal of Clinical Epidemiology
DOI 10.1016/j.jclinepi.2015.11.002
Open this paper in CoChat — free account

Abstract

Objectives Expressing treatment effects in relative terms yields larger numbers than expressions in absolute terms, affecting the judgment of the clinicians and patients regarding the treatment options. It is uncertain how authors of systematic reviews (SRs) absolute effect estimates are reported in. We therefore undertook a systematic survey to identify and describe the reporting and methods for calculating absolute effect estimates in SRs. Study design and setting Two reviewers independently screened title, abstract, and full text and extracted data from a sample of Cochrane and non-Cochrane SRs. We used regression analyses to examine the association between study characteristics and the reporting of absolute estimates for the most patient-important outcome. Results We included 202 SRs (98 Cochrane and 104 non-Cochrane), most of which (92.1%) included standard meta-analyses including relative estimates of effect. Of the 202 SRs, 73 (36.1%) reported absolute effect estimates for the most patient-important outcome. SRs with statistically significant effects were more likely to report absolute estimates (odds ratio, 2.26; 95% confidence interval: 1.08, 4.74). The most commonly reported absolute estimates were: for each intervention, risk of adverse outcomes expressed as a percentage (41.1%); number needed to treat (26.0%); and risk for each intervention expressed as natural units or natural frequencies (24.7%). In 12.3% of the SRs that reported absolute effect estimates for both benefit and harm outcomes, harm outcomes were reported exclusively as absolute estimates. Exclusively reporting of beneficial outcomes as absolute estimates occurred in 6.8% of the SRs. Conclusions Most SRs do not report absolute effects. Those that do often report them inadequately, thus requiring users of SRs to generate their own estimates of absolute effects. For any apparently effective or harmful intervention, SR authors should report both absolute and relative estimates to optimize the interpretation of their findings.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2015.11.002

Cites

35 of 35 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

We couldn’t retrieve this paper’s full text, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

Cites

Flagged first, then by citations

2 of the 37 references couldn’t be matched to a known paper.

Cited by

Sorted by citations

Related

Authors
Alonso-Coello, P., Carrasco-Labra, A., Brignardello-Petersen, R., Neumann, I., Akl, E. A., Vernooij, R. W. M., Johnston, B. C., Sun, X., Briel, M., Busse, J. W., Ebrahim, S., Granados, C. E., Iorio, A., Irfan, A., Martínez García, L., Mustafa, R. A., Ramírez-Morera, A., Selva, A., Solà, I., Sanabria, A. J., Tikkinen, K. A. O., Vandvik, P. O., Zazueta, O. E., Zhang, Y., Zhou, Q., Schünemann, H., & Guyatt, G. H.
Affiliations
  • Pontificia Universidad Católica de Chile (CL)
  • American University of Beirut Medical Center (LB)
  • Iberoamerican Cochrane Centre (ES)
  • University of Toronto (CA)
  • Sichuan University (CN)
  • University Hospital of Basel (CH)
  • McMaster University (CA)
  • Stanford University (US)
  • Universidad de La Sabana (CO)
  • University of Louisville (US)
  • University of Missouri–Kansas City (US)
  • Costa Rican Department of Social Security (CR)
  • University of Helsinki (FI)
  • University of Oslo (NO)
  • China Academy of Chinese Medical Sciences (CN)
ORCID
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2016-04
Volume
72
Pages
16-26
ISSN
0895-4356 (ISSN-L)
DOI
10.1016/j.jclinepi.2015.11.002
OpenAlex
W2195700003
Cited by
70
References
37
Citations by year
2026: 6 · 2025: 2 · 2024: 9 · 2023: 7 · 2022: 6 · 2021: 10 · 2020: 8 · 2019: 7 · 2018: 3 · 2017: 6 · 2016: 6
FWCI
0.77
Type
journal article
Language
en
Open access
Licence
Locations
  • Journal of Clinical Epidemiology (publishedVersion)
  • PubMed (publishedVersion)
  • Työväentutkimus Vuosikirja (submittedVersion, other-oa)
  • Universidad de Chile (submittedVersion, cc-by-nc-nd)
  • Repositorio Institucional (Pontificia Universidad Javeriana) (submittedVersion, cc-by-nc)
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Physical Sciences › Mathematics › Statistics and Probability › Statistical Methods in Epidemiology
Keywords
reporting quality, systematic reviews, number needed to treat, relative risk, natural frequencies, benefit-harm balance, meta-analysis reporting
MeSH terms
Data Interpretation, Statistical, Delivery of Health Care, Humans, Quality Control, Research Design, Odds Ratio, Epidemiologic Studies, Research Report, Patient Outcome Assessment

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review