CoChat
Sign Up
Cited by 513Meta-analysis and systematic reviewsClinical practice guidelines implementationDelphi Technique in Research

AHRQ Series Paper 5: Grading the strength of a body of evidence when comparing medical interventions—Agency for Healthcare Research and Quality and the Effective Health-Care Program

Owens, Douglas K., Lohr, Kathleen N., Atkins, David, Treadwell, Jonathan R., Reston, James T., Bass, Eric B., Chang, Stephanie, Helfand, Mark·2010·Journal of Clinical Epidemiology
DOI 10.1016/j.jclinepi.2009.03.009
Open this paper in CoChat — free account

Abstract

Objective To establish guidance on grading strength of evidence for the Evidence-based Practice Center (EPC) program of the U.S. Agency for Healthcare Research and Quality. Study design and setting Authors reviewed authoritative systems for grading strength of evidence, identified domains and methods that should be considered when grading bodies of evidence in systematic reviews, considered public comments on an earlier draft, and discussed the approach with representatives of the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) working group. Results The EPC approach is conceptually similar to the GRADE system of evidence rating; it requires assessment of four domains: risk of bias, consistency, directness, and precision. Additional domains to be used when appropriate include dose-response association, presence of confounders that would diminish an observed effect, strength of association, and publication bias. Strength of evidence receives a single grade: high, moderate, low, or insufficient. We give definitions, examples, mechanisms for scoring domains, and an approach for assigning strength of evidence. Conclusion EPCs should grade strength of evidence separately for each major outcome and, for comparative effectiveness reviews, all major comparisons. We will collaborate with the GRADE group to address ongoing challenges in assessing the strength of evidence.

Integrity signals

Registry

Crossref · 10.1016/j.jclinepi.2009.03.009

Cites

5 of 5 references verified0 citation errors · 0 not checkable

Statistics

Recomputing p-values needs the paper’s full text.

Claims the paper makes

This paper’s full text isn’t openly available, so its claims and self-check weren’t run. The registry and citation checks still apply.

Claims other papers make about this one

Each is checked against this paper’s own text: does it say what the citing paper says it does?

  • Methodological quality was assessed using the Cochrane Collaboration's risk-of-bias tool, considering random sequence generation, advertisement as a meditation intervention, control for confound variables, blinding of outcome assessment, and selective reporting of statistical results.

    “Methodological quality was assed using the Cochrane Collaboration’s risk-of-bias tool32,33 which included considering if the study: used random sequence generation; was advertised as a meditation intervention; controlled for confound variables (e.g. demand characteristics); included blinding of outcome assessment; and showed selective reporting of statistical results.”

    From The limited prosocial effects of meditation: A systematic review and meta-analysis (2018) · cites this paper as [33]

Cites

Flagged first, then by citations

Cited by

Sorted by citations

Authors
Owens, D. K., Lohr, K. N., Atkins, D., Treadwell, J. R., Reston, J. T., Bass, E. B., Chang, S., & Helfand, M.
Affiliations
  • University of California, San Francisco (US)
  • RTI International (US)
  • United States Department of Veterans Affairs (US)
  • ECRI Institute (US)
  • Johns Hopkins University (US)
  • Agency for Healthcare Research and Quality (US)
  • Oregon Health & Science University (US)
Corresponding author
Owens, Douglas K.
ORCID
Venue
Journal of Clinical Epidemiology
Publisher
Elsevier BV
Published
2010-05
Volume
63
Issue
5
Pages
513-523
ISSN
0895-4356 (ISSN-L)
DOI
10.1016/j.jclinepi.2009.03.009
OpenAlex
W2004489444
Cited by
513
References
5
Citations by year
2026: 5 · 2025: 21 · 2024: 8 · 2023: 11 · 2022: 19 · 2021: 20 · 2020: 28 · 2019: 22 · 2018: 24 · 2017: 29 · 2016: 25 · 2015: 38 · 2014: 50 · 2013: 93 · 2012: 61
FWCI
49.21
Type
journal article
Language
en
Open access
closed
Licence
www.elsevier.com/tdm/userlicense/1.0/ (tdm)
Locations
  • Journal of Clinical Epidemiology (publishedVersion)
  • PubMed (publishedVersion)
Funders
  • U.S. Department of Veterans Affairs
  • Agency for Healthcare Research and Quality
Topics
  • Social Sciences › Decision Sciences › Statistics, Probability and Uncertainty › Meta-analysis and systematic reviews
  • Health Sciences › Medicine › Public Health, Environmental and Occupational Health › Clinical practice guidelines implementation
  • Social Sciences › Sociology and Political Science › Delphi Technique in Research
Keywords
quality of evidence, level of evidence, evidence-based practice centers, Agency for Healthcare Research and Quality, GRADE, risk of bias, consistency, precision, dose-response relationship, confounding, publication bias, systematic reviews, medical interventions
MeSH terms
Humans, Review Literature as Topic, United States, Reproducibility of Results, Bias, Outcome Assessment, Health Care, Evidence-Based Medicine

About these checks

Registry, retraction, Cites and statistics results are computed, not judged, with one exception: a printed reference that no index lists is read once by a model before it counts as a citation error. Claims are extracted by a model and kept only when their quote appears verbatim in the text; each cited source is then checked, with a model’s help, for whether it says what the claim relies on. Those verdicts can be wrong — read the claim’s quote and the cited source before relying on one.

Statistics and arithmetic are recomputed, not judged.

Confidence bands are shown only for fields calibrated on our annotated test set. None is calibrated yet, so no band is shown.

What counts as a claim ›How we measure confidence ›How we check this ›

Are you an author, or do you see a problem with this page? Request a review